Image anti-shake method and system for mobile phone camera combined with telescope shooting

Abstract

The application provides a kind of mobile phone camera combination telescope shooting image anti-shake method and system, belong to image acquisition and processing field, method includes: the multiple of telescope, the field of view of camera, the shaking angular velocity data of mobile phone are obtained;Telescope is set in the front of mobile phone camera;According to the shaking angular velocity data, the shaking angle of camera is determined;The shaking angle is filtered, and first shaking component and second shaking component are obtained;According to first shaking component and the multiple of telescope, the action amount of the optical anti-shake execution component of camera is controlled;Image is collected by the camera after optical anti-shake;According to second shaking component, the field of view of camera and the multiple of telescope, the equivalent offset proportion of remaining shaking is determined;According to the equivalent offset proportion of remaining shaking, image is moved to the opposite direction of shaking, and the image after anti-shake is obtained.The application is aimed at the zooming shooting scene of telescope, combines optical anti-shake with digital anti-shake, and improves the anti-shake performance of image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image acquisition and processing, in particular to an image anti-shake method and system for photographing by a mobile phone camera combined with a telescope. BACKGROUND

[0002] Long-focus application of mobile phone photography is very wide, and long-focus cameras are configured in mobile phone products of many manufacturers. However, due to the small volume of mobile phone lenses, the lens aperture is also small, resulting in that the effect of long-focus camera photographing is limited by optical limit and the definition is not enough. Therefore, many photographers increase a telescope in front of the mobile phone camera as a long-focus lens to expand the long-focus photographing capability. When photographing, the mobile phone is affected by hand holding, and the motion is complex. Generally, high-frequency small-angle shaking must be eliminated, and low-frequency large-angle movement is considered as panning action of photographing and needs to retain its dynamics. The medium-frequency shaking between the two is generally unintentional movement of the hand.

[0003] The telescope itself has no anti-shake function, and the small shaking is magnified, resulting in that the photographed video jumps violently and the photo is very blurred. Although some mobile phone cameras have optical anti-shake function, they still cannot eliminate the shaking caused by the application mode of the external telescope.

[0004] Based on the above problems, there is an urgent need for a method capable of eliminating the shaking generated when a mobile phone camera combined with a telescope photographs a video or a photo. SUMMARY

[0005] The purpose of the present application is to provide an image anti-shake method and system for photographing by a mobile phone camera combined with a telescope, which can eliminate the shaking generated when a mobile phone camera combined with a telescope photographs a video or a photo, so as to improve the stability of the image.

[0006] To achieve the above purpose, the present application provides the following solutions:

[0007] The image anti-shake method for photographing by a mobile phone camera combined with a telescope comprises the following steps:

[0008] obtaining the magnification of the telescope, the field of view angle of the camera and the shaking angular velocity data of the mobile phone; the telescope is arranged in front of the mobile phone camera;

[0009] integrating the shaking angular velocity data to obtain the shaking angle of the camera;

[0010] filtering the shaking angle to obtain a first shaking component and a second shaking component; the frequency of the first shaking component is greater than the frequency of the second shaking component; the shaking amplitude of the first shaking component is smaller than the shaking amplitude of the second shaking component;

[0011] According to the first jitter component and the multiple of the telescope, the action amount of the optical image stabilization execution component of the camera is controlled to perform optical image stabilization on the camera.

[0012] An image is captured by the camera after optical image stabilization.

[0013] According to the second jitter component, the field of view angle of the camera, and the multiple of the telescope, the equivalent offset ratio of the residual jitter is determined.

[0014] According to the equivalent offset ratio of the residual jitter, the image is moved in the opposite direction of the jitter to obtain an image after image stabilization.

[0015] Optionally, the frequency range of the first jitter component and the frequency range of the second jitter component are automatically adjusted according to the hand-held jitter characteristics of the operator, and the principle of adjustment is to fully utilize the maximum stroke of the optical image stabilization execution component.

[0016] Optionally, the frequency of the first jitter component is 3-5 Hz, and the frequency of the second jitter component is 1-3 Hz.

[0017] Optionally, the shaking angle is filtered to obtain the first jitter component and the second jitter component, specifically including:

[0018] The shaking angle is filtered by a low-pass filtering algorithm to obtain a preliminary jitter amount;

[0019] The preliminary jitter amount is filtered by a high-pass filtering algorithm to obtain the first jitter component;

[0020] According to the preliminary jitter amount and the first jitter component, the second jitter component is determined:

[0021] DM=D-DH;

[0022] Wherein, DM is the second jitter component, D is the preliminary jitter amount, and DH is the first jitter component.

[0023] Optionally, according to the first jitter component and the multiple of the telescope, the action amount of the optical image stabilization execution component of the camera is controlled to perform optical image stabilization on the camera, specifically including:

[0024] According to the first jitter component and the multiple of the telescope, the optical image stabilization target amount is determined; the optical image stabilization target amount is less than or equal to the maximum stroke of the optical image stabilization execution component; if the optical image stabilization target amount is greater than the maximum stroke of the optical image stabilization execution component, the amplitude of the first jitter component is reduced, and the reduced part is combined into the second jitter component;

[0025] According to the optical image stabilization target amount, the camera is optically stabilized by the optical image stabilization execution component.

[0026] Optionally, the equivalent offset ratio of the residual jitter is determined by the following formula:

[0027] EOR=N*DM / M;

[0028] wherein EOR is the equivalent offset ratio of the residual jitter, N is the multiple of the telescope, DM is the second jitter component, and M is the field of view of the camera.

[0029] Optionally, when the number of cameras is multiple, the image anti-shake method of the camera combined with the telescope further comprises:

[0030] selecting a camera from the multiple cameras to collect images combined with the telescope.

[0031] To achieve the above object, the application further provides the following scheme:

[0032] An image anti-shake method of a camera combined with a telescope, comprising: a mobile phone and a telescope; the mobile phone comprises a camera, a gyroscope, an optical anti-shake execution component and a processor; the camera, the gyroscope and the optical anti-shake execution component are connected with the processor; the telescope is arranged in front of the camera;

[0033] The gyroscope is used to acquire shaking angular velocity data of the camera;

[0034] The processor is used to acquire a multiple of the telescope and a field of view of the camera, to perform integral operation on the shaking angular velocity data to obtain a shaking angle of the camera, to filter the shaking angle to obtain a first jitter component and a second jitter component; the frequency of the first jitter component is greater than the frequency of the second jitter component; the jitter amplitude of the first jitter component is smaller than the jitter amplitude of the second jitter component;

[0035] The optical anti-shake execution component is used to perform optical anti-shake on the camera according to the first jitter component and the multiple of the telescope;

[0036] The camera is used to collect images;

[0037] The processor is further used to determine an equivalent offset ratio of residual jitter according to the second jitter component, the field of view of the camera and the multiple of the telescope, to move the images to the opposite direction of the jitter according to the equivalent offset ratio of the residual jitter to obtain anti-shake images. Optionally, the image anti-shake system of the camera combined with the telescope further comprises an adapter; the telescope is fixed on the camera through the adapter.

[0038] According to the embodiments of the application, the following technical effects are achieved:

[0039] The present application combines optical image stabilization and digital image stabilization, determines the shaking angle according to the angular velocity data of the camera, filters the shaking angle to obtain a first shaking component (high frequency) and a second shaking component (low frequency), then performs optical image stabilization on the camera according to the first shaking component and the magnification of the telescope to eliminate high frequency shaking and obtain a clear image, and then eliminates low frequency shaking by means of digital image stabilization (moves the image in the direction opposite to the shaking direction of the mobile phone according to the low frequency shaking component, the field of view angle and the magnification of the telescope), thereby increasing the effective range and improving the stability of the image. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0041] Figure 1 The overall flowchart of the image stabilization method for the image captured by the mobile phone camera combined with the telescope provided by the present application;

[0042] Figure 2 The detailed flowchart of the image stabilization method for the image captured by the mobile phone camera combined with the telescope provided by the present application;

[0043] Figure 3 The schematic diagram of the image stabilization system for the image captured by the mobile phone camera combined with the telescope provided by the present application;

[0044] Figure 4 The schematic diagram of the connection relationship of the components in the mobile phone.

[0045] Symbol explanation:

[0046] 1 - mobile phone, 2 - telescope, 3 - adapter clamp, 4 - camera, 5 - gyroscope, 6 - optical image stabilization execution component, 7 - processor. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0048] The application aims to provide an image anti-shake method and system for mobile phone camera combined with telescope shooting, which combines optical anti-shake and digital anti-shake to improve the stability of the image.

[0049] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the application will be further described in detail below with reference to the drawings and specific embodiments.

[0050] As shown in Figure 1 and Figure 2 The application provides an image anti-shake method for mobile phone camera combined with telescope shooting, which comprises the following steps:

[0051] S1: acquiring the multiple of the telescope, the field of view angle of the camera and the shaking angular velocity data of the mobile phone. The telescope is arranged in the front of the camera.

[0052] Specifically, the multiple of the telescope is used as the sensitivity of the mobile phone anti-shake processing, that is, the rated sensitivity of the mobile phone anti-shake is improved by N times, and N is the multiple of the telescope.

[0053] In the embodiment, the shaking angular velocity data of the movement of the camera in each axis (including the pitch direction and the left-right direction) is obtained by the gyroscope built in the mobile phone. The shaking angular velocity data includes the angular velocity data of the camera and the shaking direction of the mobile phone.

[0054] S2: performing integral operation on the shaking angular velocity data to obtain the shaking angle of the camera. In the embodiment, the integral operation is performed on the angular velocity data of each axis respectively to obtain the shaking angle of the corresponding axis.

[0055] S3: filtering the shaking angle to obtain a first shaking component and a second shaking component. The frequency of the first shaking component is greater than the frequency of the second shaking component. The shaking amplitude of the first shaking component is smaller than the shaking amplitude of the second shaking component.

[0056] In order to improve the anti-shake effect, the two frequency points of the high frequency and the medium frequency need to be set reasonably in advance. The two frequency points can be set in advance according to experience, adjusted through the software interface, or automatically adjusted by the software during the shooting process. The frequency range of the first shaking component and the frequency range of the second shaking component can be automatically adjusted according to the hand shaking characteristics of the operator, and the adjustment principle is to fully utilize the maximum stroke of the optical anti-shake execution component. As a specific embodiment, the frequency of the first shaking component is about 3Hz to 5Hz, and the frequency of the second shaking component is about 1Hz to 3Hz.

[0057] Further, S3 specifically comprises:

[0058] The low-pass filtering algorithm is used to filter the shaking angle to obtain a preliminary shaking amount, i.e. to filter the low-frequency component of the shaking angle.

[0059] The high-pass filtering algorithm is used to filter the preliminary shaking amount to obtain a first shaking component, i.e. to filter the medium-frequency component of the preliminary shaking amount. The first shaking component is used as a high-frequency small-angle anti-shake target of the optical anti-shake execution component.

[0060] According to the preliminary shaking amount and the first shaking component, a second shaking component is determined: DM=D-DH; wherein DM is the second shaking component, D is the preliminary shaking amount, and DH is the first shaking component.

[0061] S4: According to the first shaking component and the magnification of the telescope, the action amount of the optical anti-shake execution component of the camera is controlled to perform optical anti-shake on the camera.

[0062] Specifically, according to the first shaking component and the magnification of the telescope, an optical anti-shake target amount is determined: DH*N. According to the optical anti-shake target amount, the camera is optically anti-shaken by the optical anti-shake execution component in the mobile phone. That is, the product of the first shaking component and the magnification of the telescope is used as the movement target amount of the optical anti-shake execution component (this target amount cannot exceed the maximum stroke of the optical anti-shake mechanism, if the optical anti-shake target amount is greater than the maximum stroke of the optical anti-shake execution component, the amplitude of the first shaking component is reduced, and the reduced part is combined into the second shaking component), the movement of the optical anti-shake execution component is driven to eliminate the high-frequency small-angle shaking.

[0063] S5: An image is captured by the camera after optical anti-shake.

[0064] S6: According to the second shaking component, the field of view angle of the camera, and the magnification of the telescope, an equivalent offset ratio of the residual shaking is determined: EOR=N*DM / M; wherein EOR is the equivalent offset ratio of the residual shaking, N is the magnification of the telescope, DM is the second shaking component, and M is the field of view angle of the camera.

[0065] The second shaking component is the shaking amount that cannot be eliminated by optical anti-shake, and needs to be further eliminated by optical anti-shake.

[0066] S7: According to the equivalent offset ratio of the residual shaking, the image is moved in the opposite direction of the shaking to obtain an anti-shaken image. The basic stability of the image is maintained to realize digital anti-shake.

[0067] Further, when the number of cameras is multiple, the image anti-shake method of the mobile phone camera combined with the telescope shooting further comprises:

[0068] S0: select a camera that is combined with a telescope to collect images from multiple cameras. Specifically, through the mobile phone software interface or communication mode, select the camera that is combined with the telescope for magnification shooting.

[0069] After using the telescope for magnification shooting, all image jitter caused by hand movement is magnified, and pure optical image stabilization can only eliminate a very small jitter angle, and digital image stabilization cannot eliminate high-frequency jitter, therefore, the present application combines optical image stabilization and digital image stabilization, first uses optical image stabilization to eliminate high-frequency jitter, which is a prerequisite for obtaining a clear image, but optical image stabilization has a small effective range, therefore, the present application eliminates the remaining jitter through digital image stabilization, thereby increasing the effective range and obtaining a stable image.

[0070] As shown in Figure 3 , the present application provides an image stabilization system for mobile phone camera combined with telescope shooting, which comprises a mobile phone 1 and a telescope 2. Figure 4 As shown in , the mobile phone 1 comprises a camera 4, a gyroscope 5, an optical image stabilization execution component 6 and a processor 7. The camera 4, the gyroscope 5 and the optical image stabilization execution component 6 are connected with the processor 7. The telescope 2 is arranged at the front of the camera 4.

[0071] The present application needs to call the hardware resources of the mobile phone, including the gyroscope, the camera, the optical image stabilization execution component, the mobile phone screen and the processor.

[0072] The gyroscope 5 is used to obtain the jitter angular velocity data of the camera 4.

[0073] The processor 7 is used to obtain the magnification of the telescope 2 and the field of view angle of the camera 4, perform integral operation on the jitter angular velocity data to obtain the jitter angle of the camera 4, filter the jitter angle to obtain a first jitter component and a second jitter component. The frequency of the first jitter component is greater than the frequency of the second jitter component. The jitter amplitude of the first jitter component is smaller than the jitter amplitude of the second jitter component.

[0074] The optical image stabilization execution component 6 is used to perform optical image stabilization on the camera 4 according to the first jitter component and the magnification of the telescope 2.

[0075] The camera 4 is used to collect images.

[0076] The processor 7 is further used to determine the equivalent offset proportion of the remaining jitter according to the second jitter component, the field of view angle of the camera 4 and the magnification of the telescope 2, move the image to the opposite direction of the jitter according to the equivalent offset proportion of the remaining jitter to obtain the image after image stabilization.

[0077] Further, the image anti-shake system of the mobile phone camera combined with the telescope shooting further comprises an adapter clip 3 or an adapter shell. The telescope 2 is fixed on the camera 4 through the adapter clip 3 or the adapter shell.

[0078] The application adds a column supporting telescope multiplication shooting in the camera software of the mobile phone itself. Since the software is developed by a mobile phone manufacturer, the underlying hardware such as a gyroscope and an optical anti-shake execution component can be flexibly called, and better anti-shake effect can be obtained.

[0079] In addition, an application supporting telescope multiplication shooting can be developed, and the underlying support of the mobile phone manufacturer, especially the data and control interface of the underlying hardware such as the gyroscope and the optical anti-shake execution component, is needed to realize the combination of optical anti-shake and digital anti-shake and improve the anti-shake effect.

[0080] As an optional embodiment, when the mobile phone camera is combined with the telescope shooting, it can be erected on a firm tripod to avoid touching and wind blowing, and a relatively stable picture can be shot through the isolation of shaking, but the degree of freedom is very low, and it cannot be implemented in many application scenarios.

[0081] The application uses the optical anti-shake execution component and the digital anti-shake processing of the mobile phone, uses a specified processing flow, can eliminate the high-multiplication image shaking caused by the telescope, obtains a stable image, has high anti-shake precision, large effective range, low comprehensive cost, and wide application range.

[0082] In order to better understand the scheme of the application, the anti-shake effect of the mobile phone camera combined with the telescope will be described below in combination with specific embodiments.

[0083] When the camera of a certain mobile phone is not multiplied, the field of view of the shot image is 60°, the rated travel of the optical anti-shake execution component is ±1mm, and the shaking amount that can be eliminated is 1 / 10 (about 6°) of the field of view of the camera. That is, the elimination range of the body shaking is ±3°, which basically meets the needs of regular shooting.

[0084] After the 10x telescope is installed for multiplication, the field of view of the image becomes about 6°, the anti-shake travel is not changed due to hardware limitation, and is still ±1mm. The anti-shake efficiency under the maximum travel is still 1 / 10 (about 0.6°) of the entire picture, that is, ±0.3°. The handheld shaking is about three to four degrees, which exceeds the effective range of the optical anti-shake, and cannot completely eliminate the influence of the hand shaking in regular shooting. Therefore, the optical anti-shake and the digital anti-shake need to be combined together to obtain better anti-shake effect.

[0085] When the external shake (preliminary shake amount) is 1°, the high-frequency shake component (first shake component) eliminated by the optical shake elimination assembly is 0.3° at most, and the remaining shake component (second shake component) is 0.7°, the equivalent offset ratio is 0.7 / 6, which is approximately equal to 11.67%, so moving the image in the opposite direction of the shake by an amplitude of 11.67% can achieve the basic stability of the image.

[0086] The present application is based on the existing hardware of the mobile phone, and is completely realized by software, only needs to install a cheap telescope, and does not need to purchase an expensive gyroscopic anti-shake telescope, so that the shake can be eliminated, the stability of the image is improved, and the comprehensive cost is reduced. Moreover, the telescope does not need to be assumed on a tripod, and long-focus shooting can be realized by easily holding the telescope, so that the present application is suitable for shooting distant scenery, near details, the moon, concerts or ball games and the like, and has a wide application range.

[0087] The principles and implementation manners of the present application are described by using specific examples in the present application, and the above description of the examples is only used for helping to understand the method of the present application and the core idea thereof; meanwhile, according to the idea of the present application, the specific implementation manners and application range can be changed by those skilled in the art. In conclusion, the content of the present description should not be understood as the limitation of the present application.

Claims

1. A method for image stabilization using a mobile phone camera combined with a telescope, characterized in that, The image stabilization method for images captured by a mobile phone camera in conjunction with a telescope includes: The telescope's magnification, the camera's field of view, and the phone's angular velocity data are acquired; the telescope is positioned in front of the phone's camera. The camera's sway angle is obtained by integrating the sway angular velocity data. The sway angle is filtered to obtain a first jitter component and a second jitter component. Specifically, this includes: filtering the sway angle using a low-pass filter algorithm to obtain an initial jitter amount; filtering the initial jitter amount using a high-pass filter algorithm to obtain a first jitter component; and determining a second jitter component based on the initial jitter amount and the first jitter component: DM = D - DH; where DM is the second jitter component, D is the initial jitter amount, and DH is the first jitter component; the frequency of the first jitter component is greater than the frequency of the second jitter component; and the jitter amplitude of the first jitter component is less than the jitter amplitude of the second jitter component. The action amount of the optical image stabilization (OIS) actuator of the camera is controlled according to the first jitter component and the magnification of the telescope to perform optical image stabilization on the camera. Specifically, this includes: determining the target amount of optical image stabilization based on the first jitter component and the magnification of the telescope; the target amount of optical image stabilization is less than or equal to the maximum travel of the OIS actuator; if the target amount of optical image stabilization is greater than the maximum travel of the OIS actuator, the amplitude of the first jitter component is reduced, and the reduced portion is merged into the second jitter component; and optical image stabilization is performed on the camera through the OIS actuator according to the target amount of optical image stabilization. Images are captured using a camera with optical image stabilization. Based on the second jitter component, the field of view of the camera, and the magnification of the telescope, determine the equivalent offset ratio of the remaining jitter: EOR = N DM / M; where EOR is the equivalent offset ratio of the remaining jitter, N is the magnification of the telescope, DM is the second jitter component, and M is the field of view of the camera. Based on the equivalent offset ratio of the remaining jitter, the image is moved in the opposite direction of the jitter to obtain the image after image stabilization.

2. The image stabilization method for mobile phone cameras combined with telescopes according to claim 1, characterized in that, The frequency range of the first jitter component and the frequency range of the second jitter component are automatically adjusted according to the hand jitter characteristics of the operator. The adjustment principle is to make full use of the maximum stroke of the optical image stabilization actuator.

3. The image stabilization method for mobile phone cameras combined with telescopes according to claim 1, characterized in that, The frequency of the first jitter component is 3Hz to 5Hz, and the frequency of the second jitter component is 1Hz to 3Hz.

4. The image stabilization method for mobile phone cameras combined with telescopes according to claim 1, characterized in that, When the number of cameras is multiple, the image stabilization method for mobile phone cameras combined with telescopes further includes: Choose from multiple cameras to combine with a telescope to acquire images.

5. An image stabilization system for mobile phone cameras combined with telescopes, characterized in that, The image stabilization method for images captured by a mobile phone camera combined with a telescope includes: a mobile phone and a telescope; the mobile phone includes a camera, a gyroscope, an optical image stabilization actuator, and a processor; the camera, the gyroscope, and the optical image stabilization actuator are all connected to the processor; the telescope is located in front of the camera; The gyroscope is used to acquire the camera's angular velocity data. The processor is used to acquire the magnification of the telescope and the field of view of the camera, integrate the angular velocity data to obtain the sway angle of the camera, and filter the sway angle to obtain a first jitter component and a second jitter component. Specifically, this includes: filtering the sway angle using a low-pass filtering algorithm to obtain an initial jitter amount; filtering the initial jitter amount using a high-pass filtering algorithm to obtain a first jitter component; and determining a second jitter component based on the initial jitter amount and the first jitter component: DM = D - DH; where DM is the second jitter component, D is the initial jitter amount, and DH is the first jitter component; the frequency of the first jitter component is greater than the frequency of the second jitter component; and the jitter amplitude of the first jitter component is less than the jitter amplitude of the second jitter component. The optical image stabilization actuator is used to perform optical image stabilization on the camera based on the first jitter component and the magnification of the telescope. Specifically, it includes: determining an optical image stabilization target amount based on the first jitter component and the magnification of the telescope; the optical image stabilization target amount is less than or equal to the maximum travel of the optical image stabilization actuator; if the optical image stabilization target amount is greater than the maximum travel of the optical image stabilization actuator, then reducing the amplitude of the first jitter component and merging the reduced portion into the second jitter component; and performing optical image stabilization on the camera through the optical image stabilization actuator based on the optical image stabilization target amount. The camera is used to capture images; The processor is further configured to determine the equivalent offset ratio of the remaining jitter based on the second jitter component, the field of view of the camera, and the magnification of the telescope; and to shift the image in the opposite direction of the jitter according to the equivalent offset ratio of the remaining jitter to obtain the image after image stabilization; the equivalent offset ratio of the remaining jitter is determined using the following formula: EOR=N DM / M; where EOR is the equivalent offset ratio of the remaining jitter, N is the telescope magnification, DM is the second jitter component, and M is the field of view of the camera.

6. The image stabilization system for mobile phone cameras combined with telescopes according to claim 5, characterized in that, The image stabilization system for mobile phone cameras combined with telescopes also includes an adapter clip or adapter shell; the telescope is fixed to the front of the camera via the adapter clip or adapter shell.

Images