Shooting method and device, electronic equipment and storage medium

Abstract

The invention relates to a shooting method and device, electronic equipment and a storage medium. The shooting method comprises the following steps: in response to shooting of shooting equipment in a motion process, obtaining an image frame sequence collected by the shooting equipment; performing motion estimation on feature points of adjacent frames in the image frame sequence to obtain inter-frame motion vectors of the adjacent frames, and determining the length of a filtering window based on the inter-frame motion vectors; the inter-frame motion vector of the adjacent frames comprises an inter-frame horizontal motion vector and an inter-frame vertical motion vector, and the filtering window length comprises a horizontal direction filtering window length and a vertical direction filtering window length; based on the horizontal direction filtering window length and the vertical direction filtering window length, determining a target filtering window, and based on the target filtering window, performing smooth filtering on the displacement change vector of the shooting equipment between the adjacent frames; and carrying out image stabilization processing on the adjacent frame images based on the displacement change vector after smooth filtering, and outputting the image after image stabilization processing as a shot image.

Description

Technical Field

[0001] The present disclosure relates to the field of terminal photography, and in particular to a photography method, apparatus, electronic device, and storage medium. Background Art

[0002] Image stabilization is a camera feature that helps compensate for minor movements that may occur during a photo. Two types of image stabilization have long been used: OIS (optical image stabilization) and DIS (digital image stabilization). OIS uses floating elements within the lens structure and physically adjusts their position to offset any camera movement during exposure. This helps maintain a stable image and produce sharp photos. DIS (digital image stabilization), on the other hand, is a software-based solution that electronically adjusts the image to compensate for camera movement. It does this by cropping the image and using image processing algorithms to stabilize the remaining portion. Digital image stabilization is widely used in various digital cameras, mobile phones, and video cameras to provide clearer and more stable images and videos. Whether shooting everyday, sports, or low-light conditions, digital image stabilization effectively reduces blur and shake.

[0003] As the weight of the lens module increases, the difficulty of controlling the OIS module increases. In high-frequency jitter scenarios, the OIS may experience abnormal position changes, causing the reported OIS data to be inconsistent with the actual movement, resulting in abnormal image jitter. The DIS solution in related technologies also suffers from abnormal jitter and image distortion in high-frequency jitter scenarios. Summary of the Invention

[0004] To overcome the problems existing in the related art, the present disclosure provides a shooting method, device, electronic device and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a shooting method is provided, comprising: in response to a shooting device shooting while in motion, acquiring a sequence of image frames captured by the shooting device; performing motion estimation on feature points of adjacent frames in the sequence of image frames to obtain inter-frame motion vectors of the adjacent frames, and determining a filtering window length based on the inter-frame motion vectors; the inter-frame motion vectors of the adjacent frames include an inter-frame horizontal motion vector and an inter-frame vertical motion vector, and the filtering window length includes a horizontal filtering window length and a vertical filtering window length; determining a target filtering window based on the horizontal filtering window length and the vertical filtering window length, and performing smoothing filtering on a displacement change vector of the shooting device between the adjacent frames based on the target filtering window; performing image stabilization processing on the adjacent frame images based on the displacement change vector after the smoothing filtering, and outputting the stabilized image as a shot image;

[0006] In one embodiment, based on the inter-frame motion vector, the filter window length is determined, including: integrating the inter-frame motion vector to obtain the original path of the shooting device and the inter-frame displacement data length; determining the dominant frequency of motion within the inter-frame displacement data length, and determining the target filter window length based on the dominant frequency of motion.

[0007] In one embodiment, determining the dominant frequency of motion within the inter-frame displacement data length includes: performing Fourier transform on the original path within the inter-frame displacement data length to obtain each frequency motion component within the inter-frame displacement length; determining the energy of each frequency motion component and the spectral component of each frequency motion component; the spectral component is the ratio of the energy of each frequency motion component to the inter-frame displacement data length; determining the frequency with the largest spectral component value among the frequency motion components as the frequency with the largest energy proportion, and using the frequency with the largest energy proportion as the dominant frequency of motion.

[0008] In one embodiment, determining the frequency corresponding to the spectral component with the largest energy proportion in each frequency motion component includes: determining the integer frequency component with the largest energy value within the inter-frame displacement data length as the first frequency; fitting the energy values ​​of the first frequency, the second frequency and the third frequency to obtain a fitting curve, the fitting curve representing the correspondence between frequency and energy, the second frequency is the maximum integer value less than the first frequency, and the third frequency is greater than the minimum integer value of the first frequency; using the frequency corresponding to the maximum energy value in the fitting curve as the frequency corresponding to the spectral component with the largest energy proportion in each frequency motion component.

[0009] In one embodiment, the target filter window length is determined based on the motion dominant frequency, including: determining the left neighbor frequency and the right neighbor frequency of the motion dominant frequency, the left neighbor frequency being the maximum integer value frequency less than the dominant frequency, and the right neighbor frequency being the minimum integer value frequency greater than the dominant frequency; determining the filter window length corresponding to the left neighbor frequency and the filter window length of the right neighbor frequency through a pre-calibrated correspondence between the integer value frequency and the filter window length within the inter-frame displacement data length; determining the dominant frequency filter window length based on a first weight, the left neighbor frequency filter window length, a second weight, and the right neighbor frequency filter window length; wherein the first weight is the ratio of the difference between the motion dominant frequency and the left neighbor to the difference between the right neighbor and the left neighbor, and the second weight is the ratio of the difference between the motion dominant frequency and the right neighbor to the difference between the right neighbor and the left neighbor.

[0010] In one embodiment, performing smoothing filtering on the displacement change vector of the shooting device between adjacent frames based on the target filtering window includes:

[0011] In the target filtering window, determining a current position of a frame and positions of at least two past time points, the positions of the at least two past time points including a first position and a second position;

[0012] Wherein, the current position of the frame is the position of a frame in the image frame sequence at the current time point, wherein the time point of the first position is earlier than the time point of the second position; the first path is mean filtered to obtain a first smooth point, and the second path is mean filtered to obtain a second smooth point; wherein, the first path is the path passed by the image in the target time period, the midpoint of the first path is the first position, the second path is the path passed by the image in the target time period, and the midpoint of the second path is the second position; the target time period is twice the target filtering window; based on the line connecting the first smooth point and the second smooth point, a third smooth point is obtained by linear interpolation calculation, and the third smooth point is used as the smoothing value of the displacement change vector of the current frame between the adjacent frames.

[0013] According to a second aspect of an embodiment of the present disclosure, a shooting device is provided, comprising an acquisition unit for acquiring, in response to shooting by a shooting device during motion, a sequence of image frames captured by the shooting device; a determination unit for performing motion estimation on feature points of adjacent frames in the sequence of image frames to obtain inter-frame motion vectors of the adjacent frames, and determining a filter window length based on the inter-frame motion vectors; the inter-frame motion vectors of the adjacent frames include inter-frame horizontal motion vectors and inter-frame vertical motion vectors, and the filter window length includes a horizontal direction filter window length and a vertical direction filter window length; a target filter window is determined based on the horizontal direction filter window length and the vertical direction filter window length, and a displacement change vector of the shooting device between the adjacent frames is smoothed and filtered based on the target filter window; a stabilization unit for performing image stabilization processing on the adjacent frame images based on the displacement change vectors after smoothing filtering, and outputting the image after image stabilization processing as a shot image.

[0014] In one embodiment, the determination unit determines the filter window length based on the inter-frame motion vector in the following manner: integrating the inter-frame motion vector to obtain the original path of the shooting device and the inter-frame displacement data length; determining the dominant frequency of motion within the inter-frame displacement data length, and determining the target filter window length based on the dominant frequency of motion.

[0015] In one embodiment, the determination unit determines the filter window length based on the inter-frame motion vector in the following manner: performing Fourier transform on the original path within the inter-frame displacement data length to obtain each frequency motion component within the inter-frame displacement length; determining the energy of each frequency motion component and the spectral component of each frequency motion component; the spectral component is the ratio of the energy of each frequency motion component to the inter-frame displacement data length; determining the frequency with the largest spectral component value among the frequency motion components as the frequency with the largest energy proportion, and taking the frequency with the largest energy proportion as the motion dominant frequency.

[0016] In one embodiment, the determination unit determines the frequency corresponding to the spectral component with the largest energy proportion in each frequency motion component in the following manner: determining the integer frequency component with the largest energy value within the inter-frame displacement data length as the first frequency; fitting the energy values ​​of the first frequency, the second frequency and the third frequency to obtain a fitting curve, wherein the fitting curve represents the correspondence between frequency and energy, the second frequency is the maximum integer value less than the first frequency, and the third frequency is greater than the minimum integer value of the first frequency; the frequency corresponding to the maximum energy value in the fitting curve is used as the frequency corresponding to the spectral component with the largest energy proportion in each frequency motion component.

[0017] In one embodiment, the determination unit determines the target filter window length based on the motion dominant frequency in the following manner: determine the left neighbor frequency and the right neighbor frequency of the motion dominant frequency, the left neighbor frequency being the maximum integer value frequency less than the dominant frequency, and the right neighbor frequency being the minimum integer value frequency greater than the dominant frequency; determine the filter window length corresponding to the left neighbor frequency and the filter window length of the right neighbor frequency through a pre-calibrated correspondence between the integer value frequency and the filter window length within the inter-frame displacement data length; determine the dominant frequency filter window length based on a first weight, the left neighbor frequency filter window length, a second weight, and the right neighbor frequency filter window length; wherein the first weight is the ratio of the difference between the motion dominant frequency and the left neighbor to the difference between the right neighbor and the left neighbor, and the second weight is the ratio of the difference between the motion dominant frequency and the right neighbor to the difference between the right neighbor and the left neighbor.

[0018] In one embodiment, the determination unit performs smoothing filtering on the displacement change vector of the shooting device between adjacent frames based on the target filtering window in the following manner: in the target filtering window, determining the current position of the frame and the positions of at least two past time points, the positions of the at least two past time points including a first position and a second position; wherein the current position of the frame is the position of a frame in the image frame sequence at the current time point, wherein the time point of the first position is earlier than the time point of the second position; performing mean filtering on the first path to obtain a first smoothing point, and performing mean filtering on the second path to obtain a second smoothing point; wherein the first path is the path that the image passes through during the target time period, the midpoint of the first path is the first position, the second path is the path that the image passes through during the target time period, and the midpoint of the second path is the second position; the target time period is twice the target filtering window; based on the line connecting the first smoothing point and the second smoothing point, linear interpolation is performed to obtain a third smoothing point, and the third smoothing point is used as the smoothing value of the displacement change vector of the current frame between the adjacent frames.

[0019] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: in response to the shooting device shooting in motion, an image frame sequence is acquired, and motion estimation is performed on the feature points of adjacent frames to obtain a two-axis inter-frame motion vector and, based on the inter-frame motion vector, determine the length of the filter window in the horizontal and vertical directions and obtain a target filter window, perform smoothing filtering based on the determined target filter window, and finally perform image stabilization processing based on the filtered displacement change vector to output a stabilized image. By accurately estimating the two-axis motion vector and determining the appropriate filter window size, a smoother image stabilization effect is achieved, the stability of the captured image is improved, and the filtering method combined with pre-interpolation filtering reduces the lag of the filtering result, thereby improving the user experience when shooting handheld, especially having significant advantages under unstable conditions such as handheld shooting.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0022] Figure 1 The figure is a flowchart of a shooting method according to an exemplary embodiment.

[0023] Figure 2 The present invention is a flowchart showing a method of determining a filtering window length based on an inter-frame motion vector according to an exemplary embodiment.

[0024] Figure 3 The present invention is a flowchart showing a method for determining a motion dominant frequency within an inter-frame displacement data length according to an exemplary embodiment.

[0025] Figure 4 The present invention is a flowchart showing a method for determining a frequency corresponding to a spectral component having the largest energy proportion among frequency motion components according to an exemplary embodiment.

[0026] Figure 5 The present invention is a flowchart showing a method of determining a target filtering window length based on a motion dominant frequency according to an exemplary embodiment.

[0027] Figure 6 The present invention is a flowchart showing a method of performing smoothing filtering on a displacement change vector of a shooting device between adjacent frames based on a target filtering window according to an exemplary embodiment.

[0028] Figure 7 The present invention is a flowchart showing a method for optimizing a DIS dual-axis filtering path based on a global motion vector according to an exemplary embodiment.

[0029] Figure 8 The figure is an effect diagram of a pre-interpolation filtering method according to an exemplary embodiment.

[0030] Figure 9 is a block diagram of a photographing device 100 according to an exemplary embodiment.

[0031] Figure 10 is a block diagram of an electronic device 200 according to an exemplary embodiment. DETAILED DESCRIPTION

[0032] Exemplary embodiments are described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different drawings represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present disclosure.

[0033] The shooting method provided by the present disclosure is applicable to scenarios where a camera shakes during shooting and anti-shake processing is performed. For example, it is applicable to scenarios where a camera is shooting while in motion. For example, a user is using a camera in a running or high-frequency shaking scenario.

[0034] In related technologies, DIS (Digital Image Stabilization) or electronic image stabilization technology is used to solve the footstep problem caused by incorrect path filtering in running and high-frequency shaking scenes.

[0035] DIS (Digital Image Stabilization) technology is designed to reduce blur and shake in photos and videos caused by camera shake through software or hardware. This technology is crucial for capturing stable, clear images. Here's how DIS works: Digital image stabilization uses image detection to detect the motion of previous and next frames of video, then applies motion-based smoothing to counteract camera shake.

[0036] Electronic image stabilization technology: This method relies on software algorithms to fix image blur caused by camera movement during post-processing. It analyzes the displacement changes between consecutive frames and corrects them into a smooth and stable picture.

[0037] Among them: Digital image stabilization is widely used in various digital cameras, mobile phone photography and video equipment to provide clearer and more stable pictures and video effects. Whether it is daily shooting, sports photography or shooting in low-light conditions, digital image stabilization can effectively reduce blur and shaking problems. However, as the weight of the lens module increases, the control difficulty of the OIS (optical image stabilizer) module increases. In high-frequency jitter scenarios, the OIS has abnormal position changes (mainly translation, but also tilt), resulting in the OIS reported data not matching the actual movement, causing abnormal shaking in the picture, such as the appearance of footsteps. If the DIS filtering path is not smooth enough, the footsteps cannot be completely eliminated.

[0038] In view of this, the embodiments of the present disclosure provide a shooting method. When performing anti-shake processing, the smoothing filter window is calculated for each of the X and Y directions to solve the problem of stability in one direction and instability in the other direction caused by the use of a unified window filter for both axes. The dominant frequency of the motion is detected by Fast Fourier Transform (FFT), and the corresponding filter window size is calculated to solve the path jitter of the filtering result caused by an inappropriate filter window. Combined with the pre-interpolation filtering method, the lag of the filtering in the preview mode is reduced.

[0039] Figure 1 FIG. 1 is a flow chart showing a shooting method according to an exemplary embodiment. Figure 1 As shown, the shooting method is used in a terminal, comprising the following steps:

[0040] In step S11 , in response to the shooting device shooting during movement, a sequence of image frames captured by the shooting device is acquired.

[0041] In step S12, motion estimation is performed on feature points of adjacent frames in the image frame sequence to obtain inter-frame motion vectors of the adjacent frames, and the length of the filtering window is determined based on the inter-frame motion vectors.

[0042] In one embodiment, a filter window refers to the size or shape of the filter applied to a local area of ​​an image. When processing images with high noise density, a smaller window can be used to preserve more image detail; when processing images with low noise density, a larger window can be selected to filter out more noise. The window size determines the amount of information the filter can utilize when smoothing noise or extracting features. A larger window can include more signal points, providing a more accurate spectral representation, helping to filter out high-frequency components and making the filtered signal closer to the low-frequency portion of the original signal. However, an overly large window may also cause the entire image to become blurred, resulting in loss and distortion of image information.

[0043] In one embodiment, the inter-frame motion vectors of adjacent frames include an inter-frame horizontal motion vector and an inter-frame vertical motion vector, and the filtering window length includes a horizontal filtering window length and a vertical filtering window length.

[0044] In step S13, a target filtering window is determined based on the horizontal filtering window length and the vertical filtering window length, and a displacement change vector of the shooting device between adjacent frames is smoothed and filtered based on the target filtering window.

[0045] In one embodiment, the smoothing filter reduces noise by taking the average or median value of pixels within a window.

[0046] In one embodiment, the displacement vector calculation refers to calculating the displacement vectors between consecutive frames based on the motion estimation results of the preceding and following frames. These displacement vectors represent the shaking or jittering of the camera during the shooting process.

[0047] In step S14 , image stabilization is performed on adjacent frame images based on the displacement change vector after smoothing filtering, and the image after image stabilization is output as a captured image.

[0048] In one embodiment, based on the displacement change vector after smooth filtering, stabilization processing is performed on adjacent frame images, including calculating a compensation vector for each frame relative to a reference frame (e.g., the first frame in a sequence or a specifically selected frame) based on the displacement change vector after smooth filtering, simulating an opposite motion, that is, the reverse motion of the terminal camera jitter, thereby offsetting the impact of the original jitter on the image.

[0049] Continuing with the above embodiment, the calculated compensation vector is used to perform a geometric transformation on each image frame, including operations such as translation, rotation, or scaling, to eliminate or reduce unstable motion between frames. During the image transformation process, some pixel regions may extend beyond the image boundaries after the transformation. To address this issue, the transformed image must be interpolated to fill in the missing pixel regions. Furthermore, any portions extending beyond the image boundaries must be cropped to ensure that the output image meets the required size and proportions.

[0050] The present invention analyzes the motion data of the shooting device during the shooting process, performs filtering on both axes, and uses smoothing filtering technology to reduce the noise and fluctuation of the displacement change vector, which can better adapt to different types of shooting actions in reality, thereby improving the stability of shooting and the quality of the resulting video.

[0051] Figure 2 FIG. 1 is a flow chart showing a method for determining a filter window length based on an inter-frame motion vector according to an exemplary embodiment. Figure 2 As shown, the shooting method is used in a terminal, comprising the following steps:

[0052] In step S21, the inter-frame motion vector is integrated to obtain the original path of the shooting device and the length of the inter-frame displacement data.

[0053] In one exemplary implementation, dense optical flow is used to calculate the global motion vectors of the preceding and following frames, and the inter-frame motion vectors on the X and Y axes are integrated separately to calculate the original motion path along both DIS axes. Optical flow is the instantaneous speed of pixel motion of a moving object on the observation imaging plane, reflecting image changes caused by motion during a time interval. Dense optical flow involves calculating the optical flow of every pixel in the image, meaning that each pixel has a corresponding motion vector. The dense optical flow algorithm estimates the motion of each pixel in the video by calculating the motion vector of each pixel along both axes between adjacent frames, thus deriving the global motion trend of the entire image.

[0054] In step S22 , a motion dominant frequency within the inter-frame displacement data length is determined, and a target filtering window length is determined based on the motion dominant frequency.

[0055] In an exemplary implementation, the motion dominant frequency refers to a frequency component that is dominant or most significant in a motion signal or image sequence, that is, the main frequency of the camera motion in the time period.

[0056] The present disclosure calculates the original path of the shooting device in a certain period of time and calculates the dominant frequency of motion in this path to determine the length of the filtering window, thereby achieving a filtering effect that is more in line with the actual situation and obtaining a smoother visual effect.

[0057] Figure 3 FIG. 1 is a flow chart showing a method for determining a motion dominant frequency within an inter-frame displacement data length according to an exemplary embodiment. Figure 3 As shown, the shooting method is used in a terminal, comprising the following steps:

[0058] In step S31, Fourier transform is performed on the original path within the inter-frame displacement data length to obtain the frequency motion components within the inter-frame displacement length.

[0059] In one embodiment, performing Fourier transform on the original path within the length of the inter-frame displacement data includes performing calculation using a fast Fourier transform.

[0060] Following the above embodiment, DFT (Discrete Fourier Transform) may be used instead of FFT, and its output result is a complex number.

[0061] In an exemplary embodiment, for an N-point sequence {x[n] 0≤n≤N}, its discrete Fourier transform is:

[0062] where 0≤k≤N-1

[0063] The sampled signal x[n] of the original signal x(t) can be expressed as:

[0064] where 0≤n≤N-1

[0065] Where e is the base of natural logarithms and i is the imaginary unit.

[0066] In step S32 , the energy of each frequency motion component and the spectral component of each frequency motion component are determined.

[0067] In one embodiment, the spectral component is the ratio of the energy of each frequency motion component to the length of the inter-frame displacement data;

[0068] In an exemplary embodiment, the energy and spectrum of each frequency motion component are calculated based on the FFT result using the following formula:

[0069]

[0070]

[0071] where X real X is the real part of the DFT transformation result of the past FFTLen data calculated based on the current moment, imag is the imaginary part of the DFT result, and FFTLen is the length of the inter-frame displacement data.

[0072] In step S33, the frequency with the largest spectral component value among the frequency motion components is determined as the frequency with the largest energy proportion, and the frequency with the largest energy proportion is used as the motion dominant frequency.

[0073] The present disclosure improves the accuracy and stability of motion analysis by calculating the motion components of each frequency within a certain time length and their energy and spectral components, and obtaining the dominant frequency of motion within the time length based on the motion components of each frequency and their energy and spectral components.

[0074] Figure 4 FIG. 1 is a flow chart showing a method for determining the frequency corresponding to the spectral component with the largest energy proportion in each frequency motion component according to an exemplary embodiment. Figure 4 As shown, the shooting method is used in a terminal, comprising the following steps:

[0075] In step S41, an integer frequency component with the maximum energy value within the inter-frame displacement data length is determined as the first frequency.

[0076] In an exemplary embodiment, the energy values ​​of the components may be directly compared to find the frequency with the maximum energy, i.e., the spectrum peak coarse location maxID.

[0077] In step S42 , the energy values ​​of the first frequency, the second frequency, and the third frequency are fitted to obtain a fitting curve.

[0078] In one embodiment, the fitting curve represents the corresponding relationship between frequency and energy, the second frequency is a maximum integer value smaller than the first frequency, and the third frequency is a minimum integer value greater than the first frequency.

[0079] In an exemplary embodiment, when the first frequency is 2.6 Hz, the second frequency is 2 Hz, and the third frequency is 3 Hz.

[0080] Following the above embodiment, a quadratic polynomial is used to fit the energy values ​​of maxID and the left and right frequency components (E maxID-1 、E maxID 、E maxID+1 ), calculate the maximum position of the spectrum peak according to the fitting quadratic curve, that is, the spectrum peak precise positioning softID.

[0081] In step S43 , the frequency corresponding to the maximum energy value in the fitting curve is used as the frequency corresponding to the spectral component with the largest energy proportion in each frequency motion component.

[0082] The present disclosure determines the integer frequency component with the largest energy value and combines it with the curve fitting method to determine the frequency corresponding to the spectral component with the largest energy proportion in each frequency motion component, which can improve the accuracy and smoothness of the filtering results, enhance the generalization ability and optimize the computational efficiency.

[0083] Figure 5 FIG. 1 is a flow chart showing a method for determining a target filter window length based on a motion dominant frequency according to an exemplary embodiment. Figure 5 As shown, the shooting method is used in a terminal, comprising the following steps:

[0084] In step S51 , the left neighbor frequency and the right neighbor frequency of the motion dominant frequency are determined.

[0085] In one embodiment, the left neighbor frequency is the maximum integer frequency smaller than the dominant frequency, and the right neighbor frequency is the minimum integer frequency larger than the dominant frequency.

[0086] In an exemplary embodiment, when the dominant frequency is 1.2, the left neighbor frequency is 1 and the right neighbor frequency is 2.

[0087] In step S52, the filter window length corresponding to the left nearest neighbor frequency and the filter window length corresponding to the right nearest neighbor frequency are determined based on the pre-calibrated correspondence between the integer value frequency within the inter-frame displacement data length and the filter window length.

[0088] In one embodiment, when the terminal makes regular reciprocating motion at a frequency W, a window of width win is used for mean filtering, which can just smooth the sine curve into a straight line. The selection of the filter window size needs to comprehensively consider factors such as the time used for filtering and the filtering effect. The larger the window, the more obvious the filtering effect may be, but it may also cause loss and distortion of image information, making the image details blurred and the image boundaries and edge contours unclear. Therefore, it is necessary to select a suitable filter window length according to the specific image processing requirements. When the terminal is shooting, the more intense the movement, the larger the window required for anti-shake processing, that is, the higher the dominant frequency, the larger the filter window. For example, when the frame rate is 60fps, 1HZ represents 60 frames, and the pre-calibrated correspondence between the dominant frequency and the filter window is shown in the following table:

[0089] Table 1

[0090] Dominant frequency / HZ 1 2 3 4 … Window size / frame Win1 Win2 Win3 Win4 …

[0091] Continuing with the above example, if FFTLen is selected too small, sufficient motion information may not be captured, resulting in inaccurate FFT analysis results. If FFTLen is selected too large, unnecessary computational complexity and time delay may be introduced. For example, when the video frame rate is 60fps, the length of the inter-frame displacement data used in the FFT calculation can be 128, while when the video frame rate is 30fps, the length of the inter-frame displacement data used in the FFT calculation is 64.

[0092] In step S53, the dominant frequency filter window length is determined based on the first weight, the left neighbor frequency filter window length, the second weight, and the right neighbor frequency filter window length.

[0093] In one embodiment, the first weight is the ratio of the difference between the dominant frequency of motion and the left neighbor to the difference between the right neighbor and the left neighbor, and the second weight is the ratio of the difference between the dominant frequency of motion and the right neighbor to the difference between the right neighbor and the left neighbor, that is, the dominant frequency filter window length is determined by linear interpolation method.

[0094] In an exemplary embodiment, the code implementation logic of the linear interpolation calculation corresponding to the filter window length win is as follows:

[0095] startID=floor(softID)

[0096] endID=ceil(softID)

[0097] ratio=(softID-startID) / (endID-startID)

[0098] win=(1-ratio)*Win[startID]+ratio*Win[endID]

[0099] Where startID represents the left neighbor of the true spectrum peak softID, endID represents the right neighbor of the true spectrum peak softID, ratio represents the ratio of the distance from the true spectrum peak to the left neighbor to the distance from the right neighbor to the left neighbor, and Win is the calibrated filter window length corresponding to different frequencies.

[0100] The present invention solves the problem of filtering result path jitter caused by inappropriate filtering window by determining the target filtering window length based on the dominant frequency of motion, improves the filtering effect, is more adaptable, and enhances the stability and quality of the captured image.

[0101] Figure 6 FIG. 1 is a flow chart showing a method for performing smooth filtering on a displacement change vector of a shooting device between adjacent frames based on a target filtering window according to an exemplary embodiment. Figure 6 As shown, the shooting method is used in a terminal, comprising the following steps:

[0102] In step S61 , in the target filtering window, the current position of the frame and the positions of at least two past time points are determined.

[0103] In one embodiment, the positions of the at least two past time points include a first position and a second position. To facilitate subsequent calculations, the specific number of time points must be greater than 2.

[0104] In one embodiment, the current position of the frame is the position of a frame in the image frame sequence at the current time point, wherein the time point of the first position is earlier than the time point of the second position.

[0105] In step S62, mean filtering is performed on the first path to obtain a first smooth point, and mean filtering is performed on the second path to obtain a second smooth point.

[0106] In one embodiment, the first path is the path that the image passes through during the target period, the midpoint of the first path is the first position, and the second path is the path that the image passes through during the target period, the midpoint of the second path is the second position.

[0107] Continuing with the above embodiment, the target time period may be twice the target filtering window length, or may be set as needed, and this disclosure does not make any specific limitation here.

[0108] In step S63, a third smooth point is obtained by linear interpolation based on the line connecting the first smooth point and the second smooth point, and the third smooth point is used as the smoothed value of the displacement change vector of the current frame between adjacent frames.

[0109] The present invention reduces the lag of filtering in preview mode by collecting motion data of the current frame within a period of time in the past, separating the frame motion paths within this period and performing path mean filtering on each of them, connecting the filtering results and performing linear interpolation to calculate the obtained filtering results as the smoothing value of the displacement change vector of the current frame.

[0110] Figure 7 FIG. 1 is a flow chart showing a method for optimizing a DIS dual-axis filtering path based on a global motion vector according to an exemplary embodiment. Figure 7 As shown, the following steps are included:

[0111] In step S71, the global motion vectors of the previous and next frame images are calculated based on the dense optical flow.

[0112] In one embodiment, dense optical flow refers to an image registration method that performs point-by-point matching on an image or a specified region. It calculates the offsets of all points in the image, forming a dense optical flow field. In other words, each pixel has a corresponding motion vector. The dense optical flow algorithm estimates the motion of each pixel in a video by calculating the motion vector of each pixel along two axes between adjacent frames, thereby determining the global motion trend of the entire image.

[0113] In step S72, the inter-frame motion vectors of the image on the two axes are integrated respectively, and the original paths of the image on the two axes are calculated respectively.

[0114] In one embodiment, the dual axes include X / Y axes.

[0115] In step S73, the filter window length of each frequency motion is calibrated and the inter-frame displacement data length FFTLen involved in FFT (Fast Fourier Transform) calculation is determined.

[0116] In step S74, FFT is used to calculate the frequency motion components within the FFTLen length.

[0117] In step S75 , the energy and spectrum components of each frequency motion component are calculated based on the FFT result.

[0118] In step S76 , the frequency with the largest energy proportion is recorded as the dominant frequency.

[0119] In one embodiment, the energy values ​​of each component are directly compared to find the frequency with the maximum energy, and then a quadratic polynomial is used to fit the energy values ​​of the frequency with the maximum energy and the left and right frequency components, and the position of the maximum value is calculated based on the quadratic curve obtained by fitting.

[0120] In an exemplary embodiment, the energy values ​​of each component are directly compared, and the frequency with the maximum energy is found to roughly locate the spectrum peak. The energy values ​​of the frequency with the maximum energy and the left and right frequency components are fitted, and the position of the maximum value is calculated based on the quadratic curve obtained by fitting, which is the precise positioning of the spectrum peak.

[0121] In step S77, the corresponding filter window length is calculated by linear interpolation according to the dominant frequency at the current moment.

[0122] In step S78, a first calculation point and a second calculation point are obtained.

[0123] In one embodiment, the number of calculation points is greater than or equal to 2.

[0124] In one embodiment, the first calculation point and the second calculation point are both earlier than the current time, and the first calculation point is earlier than the second calculation point.

[0125] In step S79, mean filtering is performed on the first path to obtain a first smooth point, and mean filtering is performed on the second path to obtain a second smooth point.

[0126] In one embodiment, the first path is a frame motion path centered on the first calculation point and passing through twice the filtering window length, and the second path is a frame motion path centered on the second calculation point and passing through twice the filtering window length.

[0127] In step S80 , the first smooth point and the second smooth point are connected and linear interpolation is performed to obtain the current smooth point.

[0128] In this disclosed embodiment, a dense optical flow algorithm is used to calculate pixel-level motion vectors between consecutive frames to form a global motion field. These motion vectors are then biaxially integrated to obtain the original path of the pixels in the time dimension. The inter-frame displacement data length used for FFT calculations is set, and FFT transformation is used to calculate motion components at different frequencies, their energy, and spectral distribution. The dominant frequency with the largest energy contribution is determined, which helps identify the primary motion pattern in the video. Dynamically adjusting the filter window length based on the current dominant frequency demonstrates the characteristics of adaptive filtering, enabling more accurate capture of local information that matches the current motion characteristics. Two or more calculation points are selected, representing different time points prior to the current moment. Mean filtering is performed on the frame motion path centered around these two points and spanning the corresponding filter window lengths to obtain a first smoothing point and a second smoothing point, respectively. This effectively removes the effects of irregular motion, such as noise and sudden changes, and extracts a stable motion trend. By connecting the first and second smoothing points and performing linear extrapolation, the smoothing point at the current moment is obtained, achieving accurate prediction and smooth estimation of the motion state of the current frame.

[0129] Figure 8FIG. 1 is a schematic diagram showing a method of obtaining a filtering result by using a pre-interpolation filtering method according to an exemplary embodiment. Figure 8 As shown, the Y axis is the posture (i.e., frame position), the X axis is time, the curve is the original motion path of the frame, the current frame is recorded as t, and the filter window length win corresponding to the current moment is found based on FFT.

[0130] like Figure 8 As shown, the first calculation point and the second calculation point are obtained. The time at which the first calculation point id25Per is located is t-3win; the time at which the second calculation point id75Per is located is t-win; the path with the starting time t-4win, the midpoint time t-3win, and the end time t-2win is recorded as the first path, and the path with the starting time t-2win, the midpoint time t-win, and the end time t is recorded as the second path.

[0131] Calculate the mean filtering result of the path (first path) with a left and right window width of Win at time id25Per, and record it as firQt25Per. Calculate the mean filtering result of the path (second path) with a left and right window width of Win at time id75Per, and record it as firQt75Per.

[0132] According to the line connecting firQt25Per and firQt75Per, candQt at time t is calculated by linear interpolation as the filtering result at that moment, that is, the smoothing point at the current moment.

[0133] The disclosed embodiment performs path filtering based on time series data, extracts relatively stable and smooth data features, effectively reduces the impact of random noise, and improves data quality. It can not only ensure the efficiency of filtering processing, but also better preserve the main features of the original data.

[0134] Based on the same concept, an embodiment of the present disclosure also provides a photographing device.

[0135] It is understandable that the shooting device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function in order to realize the above functions. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0136] Figure 9 FIG. 1 is a block diagram of a photographing device 100 according to an exemplary embodiment. Figure 9The device includes an acquisition unit 101, a determination unit 102 and a stabilization unit 103.

[0137] The acquisition unit 101 is configured to acquire a sequence of image frames captured by the shooting device in response to the shooting device shooting during movement;

[0138] The determining unit 102 is configured to perform motion estimation on feature points of adjacent frames in the image frame sequence to obtain inter-frame motion vectors of the adjacent frames, and determine a filtering window length based on the inter-frame motion vectors; wherein the inter-frame motion vectors of the adjacent frames include an inter-frame horizontal motion vector and an inter-frame vertical motion vector, and wherein the filtering window length includes a horizontal filtering window length and a vertical filtering window length; determine a target filtering window based on the horizontal filtering window length and the vertical filtering window length, and perform smoothing filtering on a displacement change vector of the shooting device between adjacent frames based on the target filtering window;

[0139] The stabilization unit 103 is configured to perform image stabilization processing on adjacent frame images based on the displacement change vector after smoothing filtering, and output the image after image stabilization processing as the captured image.

[0140] In one embodiment, the determination unit 102 determines the filter window length based on the inter-frame motion vector in the following manner: integrating the inter-frame motion vector to obtain the original path of the shooting device and the inter-frame displacement data length; determining the dominant frequency of motion within the inter-frame displacement data length, and determining the target filter window length based on the dominant frequency of motion.

[0141] In one embodiment, the determination unit 102 determines the filter window length based on the inter-frame motion vector in the following manner: performing Fourier transform on the original path within the inter-frame displacement data length to obtain each frequency motion component within the inter-frame displacement length; determining the energy of each frequency motion component and the spectral component of each frequency motion component; the spectral component is the ratio of the energy of each frequency motion component to the inter-frame displacement data length; determining the frequency with the largest spectral component value among each frequency motion component as the frequency with the largest energy proportion, and taking the frequency with the largest energy proportion as the motion dominant frequency.

[0142] In one embodiment, the determination unit 102 determines the frequency corresponding to the spectral component with the largest energy proportion in each frequency motion component in the following manner: determine the integer frequency component with the largest energy value within the inter-frame displacement data length as the first frequency; fit the energy values ​​of the first frequency, the second frequency and the third frequency to obtain a fitting curve, the fitting curve represents the correspondence between frequency and energy, the second frequency is the maximum integer value less than the first frequency, and the third frequency is greater than the minimum integer value of the first frequency; the frequency corresponding to the maximum energy value in the fitting curve is used as the frequency corresponding to the spectral component with the largest energy proportion in each frequency motion component.

[0143] In one embodiment, the determination unit 102 determines the target filter window length based on the dominant frequency of motion in the following manner: determine the left neighbor frequency and the right neighbor frequency of the dominant frequency of motion, where the left neighbor frequency is the maximum integer value frequency less than the dominant frequency, and the right neighbor frequency is the minimum integer value frequency greater than the dominant frequency; determine the filter window length corresponding to the left neighbor frequency and the filter window length of the right neighbor frequency through a pre-calibrated correspondence between the integer value frequency and the filter window length within the inter-frame displacement data length; determine the dominant frequency filter window length based on a first weight, the left neighbor frequency filter window length, a second weight, and the right neighbor frequency filter window length; wherein the first weight is the ratio of the difference between the dominant frequency of motion and the left neighbor to the difference between the right neighbor and the left neighbor, and the second weight is the ratio of the difference between the dominant frequency of motion and the right neighbor to the difference between the right neighbor and the left neighbor.

[0144] In one embodiment, the determination unit 102 performs smoothing filtering on the displacement change vector of the shooting device between adjacent frames based on the target filtering window in the following manner: in the target filtering window, the current position of the frame and the positions of at least two past time points are determined, and the positions of the at least two past time points include a first position and a second position; wherein the current position of the frame is the position of a frame in the image frame sequence at the current time point, wherein the time point of the first position is earlier than the time point of the second position; mean filtering is performed on the first path to obtain a first smoothing point, and mean filtering is performed on the second path to obtain a second smoothing point; wherein the first path is the path that the image passes through in the target time period, and the midpoint of the first path is the first position, and the second path is the path that the image passes through in the target time period, and the midpoint of the second path is the second position; the target time period is twice the target filtering window; based on the line connecting the first smoothing point and the second smoothing point, linear interpolation is performed to obtain a third smoothing point, and the third smoothing point is used as the smoothing value of the displacement change vector of the current frame between adjacent frames.

[0145] Figure 10 2 is a block diagram of an electronic device 200 according to an exemplary embodiment. For example, the electronic device 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0146] Reference Figure 10 , electronic device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0147] The processing component 202 generally controls the overall operation of the electronic device 200, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0148] The memory 204 is configured to store various types of data to support operations on the electronic device 200. Examples of such data include instructions for any application or method operating on the electronic device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0149] The power component 206 provides power to the various components of the electronic device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 200.

[0150] The multimedia component 208 includes a screen that provides an output interface between the electronic device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the electronic device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0151] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 200 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 204 or transmitted via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0152] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0153] The sensor assembly 214 includes one or more sensors for providing various aspects of status assessment for the electronic device 200. For example, the sensor assembly 214 can detect the open / closed state of the electronic device 200, the relative positioning of components, such as the display and keypad of the electronic device 200. The sensor assembly 214 can also detect changes in the position of the electronic device 200 or a component of the electronic device 200, the presence or absence of user contact with the electronic device 200, the orientation or acceleration / deceleration of the electronic device 200, and temperature changes of the electronic device 200. The sensor assembly 214 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 214 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0154] The communication component 216 is configured to facilitate wired or wireless communication between the electronic device 200 and other devices. The electronic device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0155] In an exemplary embodiment, the electronic device 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described methods.

[0156] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the electronic device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0157] It is understood that in this disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of related objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0158] It will be further understood that the terms "first," "second," and the like are used to describe various types of information, but such information should not be limited to these terms. These terms are used solely to distinguish information of the same type from one another and do not indicate a particular order or level of importance. In fact, the terms "first," "second," and the like are fully interchangeable. For example, first information could be referred to as second information, and similarly, second information could be referred to as first information without departing from the scope of this disclosure.

[0159] It can be further understood that the terms "center", "longitudinal", "lateral", "down", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing this embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation.

[0160] It is further understood that, unless otherwise specified, “connection” includes a direct connection where there are no other components between the two elements, and also includes an indirect connection where there are other elements between the two elements.

[0161] It is further understood that although operations are described in a particular order in the drawings in the embodiments of the present disclosure, this should not be construed as requiring that the operations be performed in the particular order shown or in a serial order, or that all of the operations shown be performed to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous.

[0162] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0163] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. A shooting method, characterized in that: include: In response to a shooting device shooting during movement, acquiring a sequence of image frames captured by the shooting device; Performing motion estimation on feature points of adjacent frames in the image frame sequence to obtain inter-frame motion vectors of the adjacent frames, and determining a filtering window length based on the inter-frame motion vectors; The inter-frame motion vectors of the adjacent frames include an inter-frame horizontal motion vector and an inter-frame vertical motion vector, and the filtering window length includes a horizontal filtering window length and a vertical filtering window length; Determining a target filtering window based on the horizontal filtering window length and the vertical filtering window length, and performing smoothing filtering on a displacement change vector of the shooting device between adjacent frames based on the target filtering window; Based on the displacement change vector after smoothing filtering, image stabilization processing is performed on the adjacent frame images, and the image after image stabilization processing is output as the captured image.

2. The shooting method according to claim 1, wherein determining the filter window length based on the inter-frame motion vector comprises: Performing integration processing on the inter-frame motion vector to obtain the original path of the shooting device and the length of the inter-frame displacement data; A motion dominant frequency within the inter-frame displacement data length is determined, and a target filtering window length is determined based on the motion dominant frequency.

3. The shooting method according to claim 2, characterized in that: The determining of the motion dominant frequency within the inter-frame displacement data length comprises: Performing Fourier transform on the original path within the inter-frame displacement data length to obtain each frequency motion component within the inter-frame displacement length; Determining the energy of each frequency motion component and the spectral component of each frequency motion component; The spectral component is the ratio of the energy of each frequency motion component to the length of the inter-frame displacement data; The frequency with the largest spectral component value among the frequency motion components is determined as the frequency with the largest energy proportion, and the frequency with the largest energy proportion is used as the motion dominant frequency.

4. The shooting method according to claim 3, characterized in that: The determining of the frequency corresponding to the spectral component with the largest energy proportion in each frequency motion component includes: Determine an integer frequency component with the largest energy value within the inter-frame displacement data length as the first frequency; Fitting the energy values ​​of the first frequency, the second frequency, and the third frequency to obtain a fitting curve, wherein the fitting curve represents a corresponding relationship between frequency and energy, wherein the second frequency is a maximum integer value less than the first frequency, and the third frequency is greater than a minimum integer value of the first frequency; The frequency corresponding to the maximum energy value in the fitting curve is used as the frequency corresponding to the spectral component with the largest energy proportion in the frequency motion components.

5. The shooting method according to claim 2, wherein: Determining a target filtering window length based on the motion dominant frequency includes: Determine a left neighbor frequency and a right neighbor frequency of the motion dominant frequency, wherein the left neighbor frequency is a maximum integer value frequency less than the dominant frequency, and the right neighbor frequency is a minimum integer value frequency greater than the dominant frequency; Determine the filter window length corresponding to the left nearest neighbor frequency and the filter window length corresponding to the right nearest neighbor frequency through the pre-calibrated correspondence between the integer value frequency within the inter-frame displacement data length and the filter window length; Determining a dominant frequency filter window length based on the first weight, the left neighbor frequency filter window length, the second weight, and the right neighbor frequency filter window length; Among them, the first weight is the ratio of the difference between the dominant frequency of the motion and the left neighbor to the difference between the right neighbor and the left neighbor, and the second weight is the ratio of the difference between the dominant frequency of the motion and the right neighbor to the difference between the right neighbor and the left neighbor.

6. The shooting method according to claim 1, wherein: The performing smoothing filtering on the displacement change vector of the shooting device between adjacent frames based on the target filtering window includes: In the target filtering window, determining a current position of a frame and positions of at least two past time points, the positions of the at least two past time points including a first position and a second position; The current frame position is the position of a frame in the image frame sequence at the current time point, wherein the time point of the first position is earlier than the time point of the second position; Performing mean filtering on the first path to obtain a first smoothing point, and performing mean filtering on the second path to obtain a second smoothing point; The first path is the path that the image passes through during the target period, the midpoint of the first path is the first position, the second path is the path that the image passes through during the target period, and the midpoint of the second path is the second position; The target time period is twice the target filter window length; Based on the line connecting the first smooth point and the second smooth point, a third smooth point is obtained by linear interpolation calculation, and the third smooth point is used as the smooth value of the displacement change vector of the current frame between the adjacent frames.

7. A photographing device, characterized in that: include: an acquisition unit, configured to acquire a sequence of image frames captured by the shooting device in response to the shooting device shooting during movement; a determining unit, configured to perform motion estimation on feature points of adjacent frames in the image frame sequence to obtain inter-frame motion vectors of the adjacent frames, and determine a filtering window length based on the inter-frame motion vectors; The inter-frame motion vectors of the adjacent frames include an inter-frame horizontal motion vector and an inter-frame vertical motion vector, and the filtering window length includes a horizontal filtering window length and a vertical filtering window length; Determining a target filtering window based on the horizontal filtering window length and the vertical filtering window length, and performing smoothing filtering on a displacement change vector of the shooting device between adjacent frames based on the target filtering window; The stabilization unit is used to perform image stabilization processing on the adjacent frame images based on the displacement change vector after smoothing filtering, and output the image after image stabilization processing as the captured image.

8. The photographing device according to claim 7, wherein: The determining unit determines the filtering window length based on the inter-frame motion vector in the following manner: Performing integration processing on the inter-frame motion vector to obtain the original path of the shooting device and the length of the inter-frame displacement data; A motion dominant frequency within the inter-frame displacement data length is determined, and a target filtering window length is determined based on the motion dominant frequency.

9. The photographing device according to claim 7, wherein: The determining unit determines the filtering window length based on the inter-frame motion vector in the following manner: Performing Fourier transform on the original path within the inter-frame displacement data length to obtain each frequency motion component within the inter-frame displacement length; Determining the energy of each frequency motion component and the spectral component of each frequency motion component; The spectral component is the ratio of the energy of each frequency motion component to the length of the inter-frame displacement data; The frequency with the largest spectral component value among the frequency motion components is determined as the frequency with the largest energy proportion, and the frequency with the largest energy proportion is used as the motion dominant frequency.

10. The photographing device according to claim 7, wherein: The determining unit determines the frequency corresponding to the spectral component with the largest energy proportion in each frequency motion component in the following manner: Determine an integer frequency component with the largest energy value within the inter-frame displacement data length as the first frequency; Fitting the energy values ​​of the first frequency, the second frequency, and the third frequency to obtain a fitting curve, wherein the fitting curve represents a corresponding relationship between frequency and energy, wherein the second frequency is a maximum integer value less than the first frequency, and the third frequency is greater than a minimum integer value of the first frequency; The frequency corresponding to the maximum energy value in the fitting curve is used as the frequency corresponding to the spectral component with the largest energy proportion in the frequency motion components.

11. The photographing device according to claim 8, wherein: The determining unit determines the target filtering window length based on the motion dominant frequency in the following manner: Determine a left neighbor frequency and a right neighbor frequency of the motion dominant frequency, wherein the left neighbor frequency is a maximum integer value frequency less than the dominant frequency, and the right neighbor frequency is a minimum integer value frequency greater than the dominant frequency; Determine the filter window length corresponding to the left nearest neighbor frequency and the filter window length corresponding to the right nearest neighbor frequency through the pre-calibrated correspondence between the integer value frequency within the inter-frame displacement data length and the filter window length; Determining a dominant frequency filter window length based on the first weight, the left neighbor frequency filter window length, the second weight, and the right neighbor frequency filter window length; Among them, the first weight is the ratio of the difference between the dominant frequency of the motion and the left neighbor to the difference between the right neighbor and the left neighbor, and the second weight is the ratio of the difference between the dominant frequency of the motion and the right neighbor to the difference between the right neighbor and the left neighbor.

12. The photographing device according to claim 7, wherein: The determining unit performs smooth filtering on the displacement change vector of the shooting device between adjacent frames based on the target filtering window in the following manner: In the target filtering window, determining a current position of a frame and positions of at least two past time points, the positions of the at least two past time points including a first position and a second position; The current frame position is the position of a frame in the image frame sequence at the current time point, wherein the time point of the first position is earlier than the time point of the second position; Performing mean filtering on the first path to obtain a first smoothing point, and performing mean filtering on the second path to obtain a second smoothing point; The first path is the path that the image passes through during the target period, the midpoint of the first path is the first position, the second path is the path that the image passes through during the target period, and the midpoint of the second path is the second position; The target time period is twice the target filter window length; Based on the line connecting the first smooth point and the second smooth point, a third smooth point is obtained by linear interpolation calculation, and the third smooth point is used as the smooth value of the displacement change vector of the current frame between the adjacent frames.

13. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the shooting method according to any one of claims 1 to 6.

14. A storage medium, characterized in that The storage medium stores instructions, and when the instructions in the storage medium are executed by the processor, the shooting method according to any one of claims 1 to 6 can be executed.

Images