Photographing method and device, related equipment and computer program product

Abstract

The invention discloses a photographing method and device, related equipment and a computer program product, and relates to the technical field of image processing. The current frame and the historical frame are obtained through the camera, the pixel difference value of the same position between the current frame and the historical frame is calculated, whether the current frame is in the stable state or not is determined based on the pixel difference value, and under the condition that the current frame is in the stable state, the shooting function is executed, and the current frame is captured rapidly and automatically. According to the method and the device, the picture can be automatically captured under the condition that a user does not need to issue a shooting instruction, so that the timeliness of picture acquisition and the definition of the shot picture are ensured, and the convenience of shooting is improved. And the phenomena that the picture is unstable and the shot picture is blurred due to the fact that the user shakes the equipment again in the process of issuing the photographing instruction are prevented.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and more specifically, to a photographing method, apparatus, related device, and computer program product. Background Art

[0002] Many intelligent terminals have a photographing function. The intelligent terminal can capture pictures through the camera for users to use or for subsequent task processing. For example, some shopping software supports users to take pictures of items of interest, and then search for corresponding products with the captured pictures; some learning software supports users to take pictures of physical objects such as paper texts and billboards, and identify the content of the captured text, images, etc. through OCR (Optical Character Recognition) or other image recognition technologies, and then realize functions such as word lookup, translation, or question search.

[0003] During the process of taking pictures with a handheld terminal device, if the device shakes when issuing a shooting instruction (such as clicking the shooting button or issuing a shooting voice instruction, etc.), the captured picture will be blurred, which will affect subsequent use. Usually, users need to take pictures multiple times to obtain a clear picture, which is inconvenient to use. Summary of the Invention

[0004] In view of the above problems, the present application is proposed to provide a photographing method, apparatus, related device, and computer program product to improve the convenience of the photographing process and ensure the quality of the captured pictures. The specific solutions are as follows:

[0005] In the first aspect of the present application, a photographing method is provided, including:

[0006] Obtain preview video frame data captured by a camera, where the preview video frame data includes a current frame and at least one historical frame before the current frame;

[0007] Calculate the pixel difference at the same position between the current frame and the historical frame, and determine whether the current frame is in a stable state based on the pixel difference;

[0008] When it is determined that the current frame is in a stable state, use the current frame as the captured picture.

[0009] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, the process of calculating the pixel difference at the same position between the current frame and the historical frame and determining whether the current frame is in a stable state based on the pixel difference includes:

[0010] Perform grayscale processing on the current frame image and the historical frame image respectively, and take the difference between the grayscale image of the current frame and the grayscale image of the historical frame to obtain at least one difference image;

[0011] Perform binarization operation on the difference image, and determine the difference pixel points based on the binarization result;

[0012] Determine whether the current frame image is in a stable state based on the number of the difference pixel points.

[0013] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, the process of determining whether the current frame image is in a stable state based on the number of the difference pixel points includes:

[0014] Calculate the proportion of the difference pixel points in the current frame image. If the proportion is lower than the set proportion threshold, determine that the current frame image is in a stable state. If the proportion is not lower than the proportion threshold, determine that the current frame image is in an unstable state;

[0015] Or,

[0016] If the number of the difference pixel points is lower than the set number threshold, determine that the current frame image is in a stable state. If the number of the difference pixel points is not lower than the number threshold, determine that the current frame image is in an unstable state.

[0017] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, the process of calculating the pixel difference at the same position between the current frame image and the historical frame image and determining whether the current frame image is in a stable state based on the pixel difference includes:

[0018] Perform grayscale processing on the current frame image and the historical frame image respectively, and detect the feature points of the grayscale image of the current frame and the grayscale image of the historical frame;

[0019] Calculate the brightness difference of each pixel point within the window area where the same feature point is located in the grayscale image of the current frame and the grayscale image of the historical frame;

[0020] Calculate the optical flow amplitude of the feature point based on the brightness difference of each pixel point within the window area where the same feature point is located;

[0021] Determine whether the current frame image is in a stable state based on the optical flow amplitude of the feature point.

[0022] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, the process of determining whether the current frame image is in a stable state based on the optical flow amplitude of the feature point includes:

[0023] Perform a first statistical operation on the optical flow amplitudes of the feature points in the grayscale image of the current frame and the grayscale image of the historical frame to obtain an optical flow amplitude statistical value;

[0024] If the optical flow amplitude statistical value is lower than a set statistical threshold, it is determined that the current frame image is in a stable state. If the optical flow amplitude statistical value is not lower than the statistical threshold, it is determined that the current frame image is in an unstable state.

[0025] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, the first statistical operation includes an averaging operation, and the corresponding optical flow amplitude statistical value includes an average optical flow amplitude.

[0026] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, based on the number of the differential pixel points, the result of determining whether the current frame image is in a stable state is used as a first stability detection result;

[0027] Then, calculating the pixel difference at the same position between the current frame image and the historical frame image, and the process of determining whether the current frame image is in a stable state based on the pixel difference further includes:

[0028] Perform grayscale processing on the current frame image and the historical frame image respectively, and detect the feature points of the grayscale image of the current frame and the grayscale image of the historical frame;

[0029] Calculate the brightness difference of each pixel point within the window area where the same feature point is located in the grayscale image of the current frame and the grayscale image of the historical frame;

[0030] Based on the brightness difference of each pixel point within the window area where the same feature point is located, calculate the optical flow amplitude of the feature point;

[0031] Based on the optical flow amplitude of the feature point, determine whether the current frame image is in a stable state to obtain a second stability detection result;

[0032] Refer to the first stability detection result and the second stability detection result to obtain the final result of whether the current frame image is in a stable state.

[0033] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, before calculating the pixel difference at the same position between the current frame image and the historical frame image, it further includes:

[0034] Perform target object detection on the current frame image to obtain the coordinate information of the area where the target object is located;

[0035] According to the coordinate information of the area where the target object is located, set the pixel values of the area in the current frame image that is not the area where the target object is located to 0;

[0036] Crop the current frame image according to the area where the target object is located to obtain the cropped current frame image.

[0037] In a possible design, in another implementation manner of the first aspect of the embodiments of the present application, when it is determined that the current frame image is in a stable state, the process of using the current frame image as a captured picture includes:

[0038] When it is determined that the current frame image is in a stable state, perform image rotation and / or trapezoidal correction processing on the current frame image, and use the result as the captured picture.

[0039] In the second aspect of the present application, a photographing device is provided, including:

[0040] A data acquisition unit, configured to acquire preview video frame data captured by a camera, where the preview video frame data includes a current frame image and at least one historical frame image before the current frame image;

[0041] A stability detection unit, configured to calculate the pixel difference at the same position between the current frame image and the historical frame image, and determine whether the current frame image is in a stable state based on the pixel difference;

[0042] A photographing unit, configured to use the current frame image as a captured picture when it is determined that the current frame image is in a stable state.

[0043] In the third aspect of the present application, an electronic device is provided, including: a memory and a processor;

[0044] The memory is used to store a program;

[0045] The processor is configured to execute the program to implement each step of the photographing method described in any one of the first aspects of the present application.

[0046] In the fourth aspect of the present application, a readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, each step of the photographing method described in any one of the first aspects of the present application is implemented.

[0047] In the fifth aspect, a computer program product is provided, including a computer program. When the computer program is executed by a processor, each step of the photographing method described in any one of the first aspects of the present application is implemented.

[0048] With the above technical solution, the present application obtains the current frame and historical frame images through a camera, calculates the pixel difference at the same position between the current frame image and the historical frame image, determines whether the current frame image is in a stable state based on the pixel difference, and executes the shooting function in the case where the current frame image is in a stable state to quickly and automatically capture the current frame image (using the current frame image as the captured picture). The present application supports automatically capturing pictures without the user issuing a shooting instruction, ensuring the timeliness of picture acquisition, the clarity of the captured picture, and improving the convenience of taking pictures. It prevents the user from shaking the device again during the process of issuing the shooting instruction, resulting in unstable pictures and blurred captured pictures. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present application. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0050] Figure 1 It is a schematic flowchart of a photographing method provided by an embodiment of the present application;

[0051] Figure 2 It exemplifies a schematic flowchart of a method for determining whether the current frame image is in a stable state by the frame difference method;

[0052] Figure 3 It exemplifies a schematic flowchart of a method for determining whether the current frame image is in a stable state by the optical flow method;

[0053] Figure 4 It exemplifies a schematic diagram of the pixel point distribution in the window area where the feature points are located;

[0054] Figure 5 It exemplifies a schematic diagram of the current frame image after cropping;

[0055] Figure 6 It is another schematic flowchart of a photographing method provided by an embodiment of the present application;

[0056] Figure 7 It is a schematic structural diagram of a photographing device provided by an embodiment of the present application;

[0057] Figure 8 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0059] It can be understood that before using the technical solutions disclosed in the embodiments of the present application, the types, usage scopes, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained through appropriate means in accordance with relevant laws and regulations.

[0060] Most intelligent terminals have a camera function. They can record life by taking pictures, or take pictures of books, road signs, billboards, etc., and obtain the text and images in the pictures through image recognition, so as to realize functions such as word query, translation, and question search.

[0061] In the related art, generally, the user manually clicks the shooting button or issues a shooting command in the form of voice, and the terminal device responds to the user's command and takes the current frame image as the captured picture. In actual situations, it often occurs that when the user holds the terminal device and issues a shooting command, the device shakes, resulting in an unstable picture captured by the device and blurring, which affects subsequent use. For example, it is impossible to recognize the text, images, etc. contained in the blurred picture, and thus it is impossible to complete subsequent functions such as word query, translation, and search.

[0062] In the embodiments of the present application, an automatic shooting method is provided. It can compare the pixel differences at the same positions in the current frame and the historical frame based on the preview video frame data captured by the camera, and detect whether the current frame is in a stable state based on this. In the case of a stable state, the current frame is automatically captured to ensure the clarity of the captured picture and improve the convenience of shooting.

[0063] The embodiments of the present application provide a shooting method, which can be applied to an intelligent terminal with a shooting function, or to a shooting system composed of an intelligent terminal and a server. That is, the intelligent terminal can execute the shooting method of the embodiments of the present application alone, or the intelligent terminal and the server can cooperate to execute the shooting method of the embodiments of the present application.

[0064] The smart terminal in the embodiments of the present application can be a mobile phone, a dictionary pen, a translation machine, a learning machine, a tablet computer, a wearable device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and the embodiments of the present application do not impose any restrictions thereon.

[0065] The photographing method in the embodiments of the present application can be applied to various photographing scenarios. Next, several possible applicable scenarios are exemplified:

[0066] In the daily life scenario, the user can take selfies or take pictures of other people, scenery, etc. through the smart terminal. By applying the photographing method of the present application, the smart terminal automatically detects whether the current frame image captured by the camera is in a stable state. If it is determined that the current frame image is in a stable state, the photographing function is executed to automatically capture the current frame image. The automatically captured current frame image can be saved, or the automatically captured current frame image can be shown to the user, and it is up to the user to decide whether to save or delete it. If the user performs a deletion operation, the photographing method can be repeated until a picture satisfactory to the user is captured, or until the user exits the photographing mode.

[0067] In the scenario of traveling abroad, the user can take pictures of roadside billboards, signboards, menus, etc. through the smart terminal. By applying the photographing method of the present application, the smart terminal automatically detects whether the current frame image captured by the camera is in a stable state. If it is determined that the current frame image is in a stable state, the photographing function is executed to automatically capture the current frame image. Further, image recognition technology (such as OCR) is called to recognize information such as text in the captured picture, and the text in the picture is translated into a specified language and output.

[0068] In the learning scenario, the user can take pictures of words, sentences or questions in textbooks through the smart terminal. By applying the photographing method of the present application, the smart terminal automatically detects whether the current frame image captured by the camera is in a stable state. If it is determined that the current frame image is in a stable state, the photographing function is executed to automatically capture the current frame image. Further, image recognition technology (such as OCR) is called to recognize information such as text in the captured picture, and word information related to the captured text information (such as the meaning and pronunciation of words), or the translation result of the text, or question information related to the captured text information (such as question analysis, answers, etc.) is queried, and the query result is output to assist the user in independent learning.

[0069] Of course, only some possible application scenarios are exemplified above. The photographing method according to the embodiments of the present application can be applied to but is not limited to the several scenarios exemplified above.

[0070] Next, in combination with the foregoing application scenarios, the process of the photographing method provided by the embodiments of the present application will be introduced.

[0071] Referring to Figure 1 , the photographing method specifically includes the following steps:

[0072] Step S100: Obtain preview video frame data captured by a camera, where the preview video frame data includes a current frame and at least one historical frame before the current frame.

[0073] Specifically, after the intelligent terminal turns on the photographing mode, it can obtain real-time preview video frame data captured by the camera, that is, a series of consecutive frames captured by the camera. In this embodiment, in order to detect the stability of the current frame, a strategy of comparing the pixels of the current frame with the historical frames before is adopted. Therefore, the preview video frame data obtained in this step includes the current frame and at least one historical frame before the current frame.

[0074] The number of historical frames obtained in this step can be 1 frame, 2 frames or multiple frames. The more the number of historical frames, the higher the accuracy of determining whether the current frame is stable, but it will also increase the calculation time. In the subsequent embodiments of the present application, it is exemplified that the historical frames include 2 frames.

[0075] Generally, the historical frames are one or more consecutive frames immediately before the current frame. Taking the current frame as the i-th frame, the historical frames obtained in this step can be the (i - 1)-th frame and the (i - 2)-th frame (taking the example of obtaining 2 historical frames). In some possible implementations, the obtained historical frames may not be completely consecutive. Still taking the current frame as the i-th frame, the historical frames obtained in this step can be the (i - 1)-th frame and the (i - 3)-th frame (taking the example of obtaining 2 historical frames).

[0076] Only some possible selection strategies of the historical frames are exemplified above. Those skilled in the art can expand and deform on the basis of the above ideas to obtain other selection strategies, which all fall within the protection scope of the present application.

[0077] In addition, according to the frame rate of the camera module carried by the intelligent terminal, the maximum time threshold of the time interval between the obtained historical frames and the current frame can be reasonably set to avoid comparing the historical frames too far from the current frame, which affects the stability detection effect.

[0078] Step S110: Calculate the pixel difference at the same position between the current frame image and the historical frame image, and determine whether the current frame image is in a stable state based on the pixel difference.

[0079] Specifically, in this step, the detection of whether the camera shakes is based on the difference between the current frame image and the historical frame image. During the process of camera shooting, the movement of the camera will cause the pixel values at the corresponding positions between adjacent frames to change. In this step, by calculating the pixel difference at the same position between the current frame image and the historical frame image, the stability of the current frame image can be perceived, that is, it can be perceived whether the camera shakes / moves during the process of shooting the current frame image.

[0080] Step S120: When it is determined that the current frame image is in a stable state, use the current frame image as the captured picture.

[0081] Based on the stability detection result obtained in the previous step, when it is determined that the current frame image is in a stable state, the current frame image can be captured in time and used as the captured picture.

[0082] After obtaining the captured picture, it can be automatically saved or output to the user for confirmation.

[0083] In some possible implementations, the captured picture can be saved after the user confirms the save. After the user confirms the deletion, the captured picture can be discarded, and then each step of the above shooting method is repeated. When the user exits the camera mode, the execution process of the camera method ends.

[0084] The camera method provided by the embodiments of the present application obtains the current frame and the historical frame image through the camera, calculates the pixel difference at the same position between the current frame image and the historical frame image, determines whether the current frame image is in a stable state based on the pixel difference, and when the current frame image is in a stable state, executes the shooting function to quickly and automatically capture the current frame image (use the current frame image as the captured picture) without the user issuing a shooting instruction, ensuring the timeliness of picture acquisition, the clarity of the captured picture, and improving the convenience of taking pictures. It prevents the user from shaking the device again during the process of issuing the shooting instruction, resulting in unstable pictures and blurred captured pictures.

[0085] For step S110 in the above embodiments, in the process of calculating the pixel difference at the same position between the current frame image and the historical frame image and determining whether the current frame image is in a stable state based on the pixel difference, the stability of the current frame image can be detected by analyzing the pixel difference. Several optional implementation methods are provided in this embodiment.

[0086] In a possible implementation, this embodiment provides a "frame difference method" to implement the detection process of whether the current frame image is stable.

[0087] Specifically:

[0088] S11. Perform grayscale processing on the current frame image and the historical frame image respectively, and calculate the difference between the grayscale image of the current frame and the grayscale image of the historical frame to obtain at least one difference image.

[0089] By performing grayscale processing on the current frame and the historical frame images, RGB images or other types of image frames can be converted into grayscale images for processing, which simplifies the computational complexity.

[0090] Optionally, in this step, on the basis of performing grayscale processing on the current frame image and the historical frame image, an operation of filtering the current frame image and the historical frame image can be added. The filtering operation can be performed after grayscale processing or before grayscale processing.

[0091] Since there is Gaussian noise (such as slight blurring, graininess, etc.) in the image frame due to reasons such as shooting, Gaussian filtering can be used to perform weighted averaging on the image pixels to smooth the image and make the image frame more natural. Therefore, the Gaussian filtering algorithm can be selected in this embodiment. Of course, in other possible implementations, other filtering algorithms can also be used.

[0092] Combined with Figure 2 As shown, define the current frame as the i-th frame. Taking the historical frame image including the (i - 1)-th frame and the (i - 2)-th frame as an example for illustrative purposes.

[0093] After obtaining the grayscale images of each frame, Gaussian filtering processing can be further performed. Further, the difference (subtraction operation) can be calculated between the grayscale image of the current frame and the grayscale image of the historical frame to obtain two difference images: diff1 and diff2.

[0094] S12. Perform binarization on the difference image, and determine the difference pixel points based on the binarization result.

[0095] Combined with Figure 2 As shown, the binarization operation process can set the binarization image thresholds thre1 and thre2. When performing binarization on the difference image, each pixel point in the difference image can be compared with the binarization image threshold respectively. The pixel value (grayscale value) of the pixel point greater than the threshold is set to 1, and the pixel value of the pixel point less than the threshold is set to 0, thereby obtaining the binarization result.

[0096] Through the binarization operation, it is possible to prevent interference from image noise, image regions blocked by spatial structure shadows (such as shadows caused by hand light occlusion, etc.), and minor local changes in the image. At the same time, it can also ensure that the pixel points of the different parts in the difference image are extracted.

[0097] It can be understood that the number of difference images is the same as the number of historical frame images. A set of different pixel points can be calculated from one difference image. Figure 2 In the case of the two historical frame images shown, two sets of different pixel points can be obtained.

[0098] S13. Determine whether the current frame image is in a stable state based on the number of the different pixel points.

[0099] Among them, the different pixel points indicate that a change has occurred at the position of this pixel point in the current frame image and the historical frame image. The more the number of different pixel points, the greater the difference between the current frame image and the historical frame image, corresponding to the less stable the current frame image. On the contrary, the fewer the number of different pixel points, the more stable the current frame image.

[0100] In a possible implementation, the proportion p of the different pixel points in the current frame image can be calculated. If the proportion p is lower than the set proportion threshold T1, it can be determined that the current frame image is in a stable state. On the contrary, if the proportion p is not lower than the proportion threshold T1, it is determined that the current frame image is in an unstable state.

[0101] In another possible implementation, it is possible to directly determine whether the current frame image is stable based on the number of different pixel points, that is: if the number of different pixel points is lower than the set number threshold T2, it is determined that the current frame image is in a stable state. If the number of different pixel points is not lower than the number threshold T2, it is determined that the current frame image is in an unstable state.

[0102] Since a set of different pixel points can be calculated from one difference image, and multiple sets of different pixel points can be obtained from multiple difference images. Then, in the process of calculating the proportion or number of different pixel points in the above embodiments, it can be to calculate the proportion or number of a corresponding set of different pixel points for each difference image respectively, and compare with the corresponding threshold (T1 or T2) to obtain a comparison result. Finally, based on the comparison results of each difference image, it is determined whether the current frame image is in a stable state. As shown in combination Figure 2 For the difference image diff1, the proportion p1 of the number of different pixel points can be calculated. For the difference image diff2, the proportion p2 of the number of different pixel points can be calculated. If both p1 and p2 are less than the proportion threshold T1, it can be determined that the current frame image is in a stable state. Otherwise, it is determined that the current frame image is in an unstable state.

[0103] Of course, for multiple sets of differential pixel points obtained from multiple differential images, they can also be merged, and the proportion or quantity of the differential pixel points after merging can be calculated and compared with the corresponding threshold (T1 or T2) to obtain a comparison result, and based on this comparison result, it is determined whether the current frame of the picture is in a stable state.

[0104] The "frame difference method" stable frame detection method provided in this embodiment can determine whether there are obvious motion traces in the image from the perspective of the overall image by comparing the quantity (proportion) of differential pixel points in two frames of pictures, and obtain a determination result on whether the current frame of the picture is in a stable state.

[0105] In another possible implementation, this embodiment provides an "optical flow method" to implement the detection process of whether the current frame of the picture is stable.

[0106] Specifically:

[0107] S21. Perform grayscale processing on the current frame of the picture and the historical frame of the picture respectively, and detect the feature points of the grayscale image of the current frame and the grayscale image of the historical frame.

[0108] By performing grayscale processing on the current frame and the historical frame of the picture, an RGB image or other types of image pictures can be converted into a grayscale image for processing, simplifying the computational complexity.

[0109] Optionally, in this step, on the basis of performing grayscale processing on the current frame of the picture and the historical frame of the picture, an operation of performing filtering processing on the current frame of the picture and the historical frame of the picture can also be added. The operation of filtering processing can be after the grayscale processing or before the grayscale processing.

[0110] Since there is Gaussian noise (such as slight blurring, graininess, etc.) in the image picture due to reasons such as shooting, the Gaussian filtering process can perform weighted averaging on the image pixels to smooth the image and make the image picture more natural. Therefore, the Gaussian filtering algorithm can be selected in this embodiment. Of course, in other possible implementations, other filtering algorithms can also be used.

[0111] Combined Figure 3 As shown, define the current frame as the i-th frame, and take the historical frame of the picture including the (i - 1)-th frame and the (i - 2)-th frame of the picture as an example for exemplary illustration.

[0112] After obtaining the grayscale images of each frame, Gaussian filtering processing can be further performed. Further, detect the feature points of the grayscale image of the current frame and the grayscale image of the historical frame.

[0113] Among them, the feature points can be some representative specific pixel points in the image, such as the corner points of the target object in the image. Taking the image of a book as an example, the feature points can be the four vertices of the book. In this step, a feature point detection algorithm can be used to detect the feature points in the grayscale images of the current frame and the historical frame.

[0114] In addition, the feature points can also be a fixed-position pixel point set in advance, such as several pixel points with preset position coordinates as feature points. Of course, the number of feature points can be one or more. The more the number of feature points, the more accurate the stable frame detection result, and the greater the corresponding computational amount.

[0115] In a possible implementation, a fixed number N of pixel points can be selected from all the pixel points in the image as feature points. Or, all the pixel points in the image can also be set as feature points. The specific feature point selection method can be set according to business needs.

[0116] S22. Calculate the brightness difference of each pixel point in the window area where the same feature point is located in the grayscale image of the current frame and the grayscale image of the historical frame.

[0117] Among them, with the feature point as the center, each pixel point in the area with a fixed window size can be determined, and the brightness difference of each pixel point in this window area in the grayscale images of the current frame and the historical frame can be calculated.

[0118] Refer to Figure 4 As shown, for the pixel point X as a feature point, the pixel points in the window area centered on the pixel point X (which can also be other window sizes) can be selected. The pixel points are pixel points 1-8 respectively, and together with the pixel point X, a total of 9 pixel points. Calculate the brightness difference of each of these 9 pixel points in the grayscale images of the current frame and the historical frame respectively.

[0119] S23. Calculate the optical flow amplitude of the feature point based on the brightness difference of each pixel point in the window area where the same feature point is located.

[0120] Specifically, the optical flow equation can be expressed as:

[0121] ;

[0122] Among them, , are the spatial gradients of the image in the x and y directions respectively, which can be calculated by the Sobel operator or other edge detection operators; is the gradient of the image in the time direction, which can be obtained by calculating the brightness difference between the pixel point in the current frame and the pixel point in the historical frame ; , are the components of the optical flow velocity in the x and y directions.​

[0123] Within a small neighborhood (the window area centered on the feature point), it is assumed that the optical flow is constant. For each pixel within the neighborhood there is an optical flow equation, which can be written in matrix form:

[0124] ;

[0125] In the above formula, n represents the number of pixels within the neighborhood, , can be calculated through the Sobel operator or other edge detection operators, and I on the right side of the equal sign t can be determined based on the calculation result of the previous step S22. Therefore, , values can be solved, that is, the optical flow amplitude in the horizontal direction (x direction) of the feature point is obtained , and the optical flow amplitude in the vertical direction (y direction) . According to , the optical flow amplitude of the feature point can be calculated.

[0126] S24. Based on the optical flow amplitude of the feature point, determine whether the current frame image is in a stable state.

[0127] Specifically, the larger the optical flow amplitude of the feature point, the greater the difference between the corresponding image regions in the front and rear two-frame images, and the more unstable the current frame image.

[0128] In a possible implementation, in this embodiment, a first statistical operation can be performed on the optical flow amplitudes of each feature point in the grayscale image of the current frame and the grayscale image of the historical frame to obtain an optical flow amplitude statistical value. If the optical flow amplitude statistical value is lower than the set statistical threshold T3, it is determined that the current frame image is in a stable state. If the optical flow amplitude statistical value is not lower than the statistical threshold T3, it is determined that the current frame image is in an unstable state.

[0129] Among them, the first statistical operation includes but is not limited to: average operation, maximum value operation, etc. Taking the average operation as an example, the average of the optical flow amplitudes of each feature point can be calculated to obtain the average optical flow amplitude. If the average optical flow amplitude is lower than the set threshold T3, it is determined that the current frame image is in a stable state. Otherwise, it is determined that the current frame image is in an unstable state.

[0130] Combined with Figure 3 as shown, the combination of the current frame image and any historical frame image obtains a set of feature points, and the optical flow amplitude of each feature point in the set of feature points. Figure 3 Taking two historical frame images as an example in

[0131] For each set of feature points, the average value of the optical flow amplitudes of the feature points in the set of feature points can be calculated respectively (such as Figure 3 the average values v1 and v2 in). On this basis, in an optional implementation, an average operation can be further performed on the average values of the optical flow amplitudes to obtain the overall average value of the optical flow amplitudes (such as Figure 3 the average value v3 in), and then the magnitude relationship between the overall average value of the optical flow amplitudes and the set threshold T3 can be determined to determine whether the current frame of the picture is in a stable state. In another optional implementation, for the average value of the optical flow amplitudes obtained for each set of feature points, it can be compared with the set threshold T3 respectively. If each average value of the optical flow amplitudes is less than the set threshold T3, it can be determined that the current frame of the picture is in a stable state; otherwise, it is determined that the current frame of the picture is in an unstable state.

[0132] The "optical flow method" stable frame detection method provided in this embodiment calculates the optical flow amplitude of the feature points, averages the optical flow amplitudes of the feature points, and determines whether the current frame is stable according to the average optical flow amplitude, accurately calculates the pixel displacement and the moving direction, and can realize the stability judgment at the pixel level.

[0133] In the foregoing embodiments, two methods are respectively exemplified to implement the detection of whether the current frame of the picture is stable, that is, the "frame difference method" and the "optical flow method" introduced in the foregoing embodiments respectively.

[0134] In some embodiments of the present application, the above "frame difference method" and "optical flow method" can be further combined to implement the detection of whether the current frame of the picture is stable. Define the result of whether the current frame of the picture is in a stable state obtained based on the "frame difference method" as the first stability detection result, and define the result of whether the current frame of the picture is in a stable state obtained based on the "optical flow method" as the second stability detection result.

[0135] In an optional example, the "frame difference method" and the "optical flow method" can be respectively used to determine whether the current frame of the picture is in a stable state, and based on the first stability detection result and the second stability detection result, the final result of whether the current frame of the picture is in a stable state can be obtained. For example, when both the first stability detection result and the second stability detection result indicate that the current frame of the picture is in a stable state, it is determined that the final result is that the current frame of the picture is in a stable state; when any one of the first stability detection result and the second stability detection result indicates that the current frame of the picture is in an unstable state, it is determined that the final result is that the current frame of the picture is in an unstable state.

[0136] Among them, in the process of using the "frame difference method" and the "optical flow method" to determine whether the current frame image is in a stable state, the two methods can be executed in parallel or in any order. It can be understood that when the two methods are executed successively, if the detection result obtained by the previous method indicates that the current frame image is in an unstable state, the execution of the latter method can be stopped at this time, and the final result that the current frame image is in an unstable state can be directly obtained.

[0137] By combining the "frame difference method" and the "optical flow method", the determination result of whether the current frame image is in a stable state can be improved, and the algorithm accuracy can be improved.

[0138] In some possible application scenarios, the user's intention is to photograph a target object, and there may be some interfering objects in the environment. When applying the foregoing photographing method of the present application, the movement of the interfering objects may affect the stability detection of the current frame image. Exemplarily, in the scenario where the user takes a photo of textbook content to search for questions, the movement of irrelevant objects such as hands and pens outside the textbook content area will affect the stability detection of the current frame image.

[0139] In some embodiments of the present application, another photographing method is provided. Before the foregoing step S110, an operation of cropping the current frame image can be further added. Specifically:

[0140] Perform target object detection on the current frame image to obtain the coordinate information of the area where the target object is located.

[0141] According to the coordinate information of the area where the target object is located, set the pixel values of the areas in the current frame image other than the area where the target object is located to 0.

[0142] Crop the current frame image according to the area where the target object is located to obtain the cropped current frame image.

[0143] Among them, the target object is the object that the user is interested in, that is, the object that the user expects to photograph. The user can specify the target object in the preview video image. In addition, a target object detection algorithm can also be used to automatically detect the target object in the current frame image. Among them, the target object detection algorithm can use a pre-trained object detection model, and the object detection model can be trained using pre-collected image training data marked with target object labels. Exemplarily, the object detection model can use the yoluv11 convolutional neural network or a neural network model with other structures.

[0144] Taking the target object as a textbook as an example, the coordinate information of the area where the target object is located can be represented by the four corner coordinates of the target object.

[0145] By performing object detection on the current frame of the image, the coordinate information of the area where the target object is located can be obtained. For example, it can be the contour information of the target object.

[0146] Since the current frame of the image captured by the camera may be rotated or deformed, in order to avoid interference from objects in the area of the current frame other than the target object, in this embodiment, the pixel values of the area of the current frame other than the target object can be set to 0. On this basis, the current frame is cropped according to the area where the target object is located, and the cropped current frame containing the target object is obtained.

[0147] Refer to Figure 5 As shown, Figure 5 An example of the cropped current frame A is illustrated. Among them, the area where the target object is located is S1. When cropping, it is cropped according to the minimum bounding rectangle of the area where the target object is located, and the cropped current frame A is obtained.

[0148] It can be seen that in addition to the area S1 where the target object is located, the cropped current frame A also includes some other areas S2 - S5. The pixel values of these other areas have been set to 0, so as to ensure that they will not interfere with the target object.

[0149] In the method of this embodiment, before detecting the stability of the current frame, object detection is performed on the current frame, the pixel values of the non-target object area are set to 0, and the area where the target object is located is cropped to obtain the cropped current frame. Then, the stability of the cropped current frame is detected, which can avoid the influence of the movement of non-target objects in the current frame before cropping on the stability detection of the target object, and improve the accuracy of the stability detection result of the current frame.

[0150] In some possible implementations, in step S120 of the foregoing embodiment, when it is determined that the current frame is in a stable state, the process of using the current frame as a captured picture may include:

[0151] When it is determined that the current frame is in a stable state, the current frame is subjected to image rotation and / or trapezoidal correction processing, and the result is used as the captured picture.

[0152] Considering that the current frame may have problems such as rotation and deformation, by performing image rotation, trapezoidal correction, etc. on the current frame, the quality of the final captured picture can be improved.

[0153] Refer to Figure 6 , Figure 6 An example of the implementation process of a photographing method for determining whether the current frame is in a stable state by combining the "frame difference method" and the "optical flow method" is illustrated. Specifically, it may include the following steps:

[0154] Step S200: Obtain the preview video frame data captured by the camera.

[0155] Among them, the preview video frame data includes the current frame and at least one historical frame before the current frame.

[0156] Step S210: Detect the target object in the current frame to obtain the coordinate information of the area where the target object is located.

[0157] Step S220: According to the coordinate information of the area where the target object is located, set the pixel values of the areas in the current frame other than the target object to 0.

[0158] Step S230: Crop the current frame according to the area where the target object is located to obtain the cropped current frame.

[0159] Step S240: Use the frame difference method to detect the stability of the current frame.

[0160] Among them, the detailed calculation process of the frame difference method can refer to the relevant introduction above and will not be elaborated here.

[0161] Step S250: Determine whether the current frame is a stable frame. If it is determined that the current frame is a stable frame, execute Step S260.

[0162] Step S260: Use the optical flow method to detect the stability of the current frame.

[0163] Among them, the detailed calculation process of the optical flow method can refer to the relevant introduction above and will not be elaborated here.

[0164] Step S270: Determine whether the current frame is a stable frame. If it is determined that the current frame is a stable frame, execute Step S280.

[0165] Step S280: Rotate and trapezoidally correct the current frame to obtain a captured picture.

[0166] The photographing method provided in this embodiment performs cropping on the current frame in the preview video frame data to remove the interference of non-target objects. At the same time, the frame difference method and the optical flow method are combined to detect the stability of the current frame. When it is determined that the detection results of both methods indicate that the current frame is in a stable state, the current frame is rotated and trapezoidally corrected to obtain a captured picture, which ensures the clarity of the finally captured picture and improves the picture quality. Moreover, this application supports automatically capturing pictures without the user issuing a shooting instruction (such as manually pressing the shooting button or issuing a shooting voice instruction), which ensures the timeliness of picture acquisition and improves the convenience of photographing.

[0167] The photographing device provided by the embodiments of the present application will be described below. The photographing device described below can be correspondingly referred to the photographing method described above.

[0168] Referring to Figure 7 , Figure 7 which is a schematic structural diagram of a photographing device disclosed in an embodiment of the present application.

[0169] As Figure 7 shown, the device may include:

[0170] A data acquisition unit 11, configured to acquire preview video frame data captured by a camera, where the preview video frame data includes a current frame and at least one historical frame before the current frame;

[0171] A stability detection unit 12, configured to calculate a pixel difference at the same position between the current frame and the historical frame, and determine whether the current frame is in a stable state based on the pixel difference;

[0172] A photographing unit 13, configured to use the current frame as a captured picture when it is determined that the current frame is in a stable state.

[0173] In a possible implementation, the process of the stability detection unit calculating the pixel difference at the same position between the current frame and the historical frame and determining whether the current frame is in a stable state based on the pixel difference includes:

[0174] Performing grayscale processing on the current frame and the historical frame respectively, and taking the difference between the grayscale image of the current frame and the grayscale image of the historical frame to obtain at least one difference image;

[0175] Performing binarization operation on the difference image, and determining difference pixel points based on the binarization result;

[0176] Determining whether the current frame is in a stable state based on the number of the difference pixel points.

[0177] In a possible implementation, the process of the stability detection unit determining whether the current frame is in a stable state based on the number of the difference pixel points includes:

[0178] Calculating the proportion of the difference pixel points in the current frame. If the proportion is lower than a set proportion threshold, it is determined that the current frame is in a stable state. If the proportion is not lower than the proportion threshold, it is determined that the current frame is in an unstable state;

[0179] Or,

[0180] If the number of the differential pixel points is lower than a set number threshold, it is determined that the current frame picture is in a stable state; if the number of the differential pixel points is not lower than the number threshold, it is determined that the current frame picture is in an unstable state.

[0181] In a possible implementation, the process that the stability detection unit calculates the pixel difference at the same position between the current frame picture and the historical frame picture and determines whether the current frame picture is in a stable state based on the pixel difference includes:

[0182] Perform grayscale processing on the current frame picture and the historical frame picture respectively, and detect the feature points of the grayscale map of the current frame and the grayscale map of the historical frame;

[0183] Calculate the brightness difference of each pixel point within the window area where the same feature point is located in the grayscale map of the current frame and the grayscale map of the historical frame;

[0184] Based on the brightness difference of each pixel point within the window area where the same feature point is located, calculate the optical flow amplitude of the feature point;

[0185] Based on the optical flow amplitude of the feature point, determine whether the current frame picture is in a stable state.

[0186] In a possible implementation, the process that the stability detection unit determines whether the current frame picture is in a stable state based on the optical flow amplitude of the feature point includes:

[0187] Perform a first statistical operation on the optical flow amplitudes of the feature points in the grayscale map of the current frame and the grayscale map of the historical frame to obtain an optical flow amplitude statistical value;

[0188] If the optical flow amplitude statistical value is lower than a set statistical threshold, it is determined that the current frame picture is in a stable state; if the optical flow amplitude statistical value is not lower than the statistical threshold, it is determined that the current frame picture is in an unstable state.

[0189] In a possible implementation, the first statistical operation includes an averaging operation, and the corresponding optical flow amplitude statistical value includes an average optical flow amplitude.

[0190] In a possible implementation, if the result that the stability detection unit determines whether the current frame picture is in a stable state based on the number of the differential pixel points is used as a first stability detection result, then the process that the stability detection unit calculates the pixel difference at the same position between the current frame picture and the historical frame picture and determines whether the current frame picture is in a stable state based on the pixel difference further includes:

[0191] Perform grayscale processing on the current frame picture and the historical frame picture respectively, and detect the feature points of the grayscale map of the current frame and the grayscale map of the historical frame;

[0192] Calculate the brightness difference between each pixel in the window area where the same feature point is located in the grayscale image of the current frame and the grayscale image of the historical frame;

[0193] Based on the brightness difference between each pixel in the window area where the same feature point is located, calculate the optical flow amplitude of the feature point;

[0194] Based on the optical flow amplitude of the feature point, determine whether the current frame image is in a stable state to obtain a second stability detection result;

[0195] Refer to the first stability detection result and the second stability detection result to obtain the final result of whether the current frame image is in a stable state.

[0196] In a possible implementation, the device of the present application may further include: an image cropping unit, configured to perform target object detection on the current frame image before the stability detection unit processes it, to obtain coordinate information of the area where the target object is located; according to the coordinate information of the area where the target object is located, set the pixel values of the area in the current frame image that is not the area where the target object is located to 0; crop the current frame image according to the area where the target object is located to obtain the cropped current frame image.

[0197] In a possible implementation, the process of the shooting unit taking the current frame image as a captured picture when determining that the current frame image is in a stable state includes:

[0198] When it is determined that the current frame image is in a stable state, perform image rotation and / or trapezoidal correction processing on the current frame image, and use the result as the captured picture.

[0199] In the embodiments of the present application, an electronic device is further provided. Refer to Figure 8 As shown, it shows a schematic structural diagram of an electronic device suitable for implementing the electronic device in the embodiments of the present application. The electronic device in the embodiments of the present application may include, but is not limited to, intelligent terminals such as mobile phones, dictionary pens, translators, tablet computers, wearable devices, and the like. Figure 8 The electronic device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0200] As Figure 8As shown in the figure, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage device 608 into the random access memory (RAM) 603, so as to implement the photographing method of the foregoing embodiments of the present application. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0201] Generally, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a memory card, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 8 an electronic device with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0202] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions run on an electronic device, the electronic device is enabled to implement any one of the photographing methods provided by the embodiments of the present application.

[0203] An embodiment of the present application also provides a computer-readable storage medium. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can be enabled to implement any one of the photographing methods provided by the embodiments of the present application.

[0204] In addition, it should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines.

[0205] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware. Of course, it can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures for implementing the same function can also be diverse, such as analog circuits, digital circuits or dedicated circuits, etc. However, for the present application, in more cases, software program implementation is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disc of a computer, etc., and includes several instructions for causing a computer device (which can be a personal computer, training device, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0206] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0207] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training device or data center to another website, computer, training device or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0208] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

Claims

1. A photographing method, characterized in that, Including: Obtain the preview video frame data captured by a camera, where the preview video frame data includes the current frame and at least one historical frame before the current frame; Calculate the pixel difference at the same position between the current frame and the historical frame, and determine whether the current frame is in a stable state based on the pixel difference; When it is determined that the current frame is in a stable state, use the current frame as the captured picture.

2. The method according to claim 1, wherein The process of calculating the pixel difference at the same position between the current frame and the historical frame and determining whether the current frame is in a stable state based on the pixel difference includes: Perform grayscale processing on the current frame and the historical frame respectively, and take the difference between the grayscale image of the current frame and the grayscale image of the historical frame to obtain at least one difference map; Perform a binarization operation on the difference map, and determine the difference pixel points based on the binarization result; Determine whether the current frame is in a stable state based on the number of the difference pixel points.

3. The method according to claim 2, wherein The process of determining whether the current frame is in a stable state based on the number of the difference pixel points includes: Calculate the proportion of the difference pixel points in the current frame. If the proportion is lower than the set proportion threshold, it is determined that the current frame is in a stable state. If the proportion is not lower than the proportion threshold, it is determined that the current frame is in an unstable state; Or, If the number of the difference pixel points is lower than the set number threshold, it is determined that the current frame is in a stable state. If the number of the difference pixel points is not lower than the number threshold, it is determined that the current frame is in an unstable state.

4. The method according to claim 1, characterized in that, The process of calculating the pixel difference at the same position between the current frame and the historical frame and determining whether the current frame is in a stable state based on the pixel difference includes: Perform grayscale processing on the current frame and the historical frame respectively, and detect the feature points of the grayscale image of the current frame and the grayscale image of the historical frame; Calculate the brightness difference of each pixel point within the window area where the same feature point is located in the grayscale image of the current frame and the grayscale image of the historical frame; Calculate the optical flow amplitude of the feature point based on the brightness difference of each pixel point within the window area where the same feature point is located; Determine whether the current frame is in a stable state based on the optical flow amplitude of the feature point.

5. The method according to claim 4, characterized in that, The process of determining whether the current frame is in a stable state based on the optical flow amplitude of the feature point includes: Perform a first statistical operation on the optical flow amplitudes of the feature points in the grayscale image of the current frame and the grayscale image of the historical frame to obtain an optical flow amplitude statistical value; If the optical flow amplitude statistical value is lower than the set statistical threshold, it is determined that the current frame is in a stable state. If the optical flow amplitude statistical value is not lower than the statistical threshold, it is determined that the current frame is in an unstable state.

6. The method according to claim 5, characterized in that, The first statistical operation includes an averaging operation, and the corresponding optical flow amplitude statistical value includes an average optical flow amplitude.

7. The method according to claim 2, wherein The result of determining whether the current frame is in a stable state based on the number of the difference pixel points is used as the first stability detection result; Then, the process of calculating the pixel difference at the same position between the current frame image and the historical frame image and determining whether the current frame image is in a stable state based on the pixel difference further includes: Perform grayscale processing on the current frame image and the historical frame image respectively, and detect the feature points of the grayscale image of the current frame and the grayscale image of the historical frame; Calculate the brightness difference of each pixel point within the window area where the same feature point is located in the grayscale image of the current frame and the grayscale image of the historical frame; Calculate the optical flow amplitude of the feature point based on the brightness difference of each pixel point within the window area where the same feature point is located; Determine whether the current frame image is in a stable state based on the optical flow amplitude of the feature point to obtain a second stability detection result; Refer to the first stability detection result and the second stability detection result to obtain the final result of whether the current frame image is in a stable state.

8. The method according to any one of claims 1 to 7, characterized in that, Before calculating the pixel difference at the same position between the current frame image and the historical frame image, it further includes: Perform target object detection on the current frame image to obtain the coordinate information of the area where the target object is located; According to the coordinate information of the area where the target object is located, set the pixel values of the area in the current frame image other than the area where the target object is located to 0; Crop the current frame image according to the area where the target object is located to obtain the cropped current frame image.

9. The method according to any one of claims 1-7, characterized in that In the case of determining that the current frame image is in a stable state, the process of using the current frame image as a captured picture includes: In the case of determining that the current frame image is in a stable state, perform image rotation and / or trapezoidal correction processing on the current frame image, and use the result as the captured picture.

10. A photographing device, characterized in that, It includes: A data acquisition unit for acquiring preview video frame data captured by a camera, where the preview video frame data includes a current frame image and at least one historical frame image before the current frame image; A stability detection unit for calculating the pixel difference at the same position between the current frame image and the historical frame image and determining whether the current frame image is in a stable state based on the pixel difference; A shooting unit for using the current frame image as a captured picture in the case of determining that the current frame image is in a stable state.

11. An electronic device, characterized in that, It includes: A memory and a processor; The memory is used for storing programs; The processor is used for executing the program to implement each step of the photographing method as described in any one of claims 1 to 9.

12. A readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the photographing method as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When this computer program is executed by the processor, it implements each step of the photographing method as described in any one of claims 1 to 9.

Images