Shooting method and device, storage medium and chip

Abstract

The invention relates to a shooting method and device, a storage medium and a chip. The shooting method comprises the following steps: determining a cutting coefficient according to a zoom ratio of the electronic equipment in response to a zoom operation; the image collected by the first camera is cut according to the cutting coefficient to obtain a preview image, the preview image is displayed in a screen of the electronic equipment, the image displayed in the screen is a gradually-changed preview image, the gradually-changed preview image is gradually aligned with an image collected by a second camera, and the image collected by the second camera is displayed in the screen of the electronic equipment. The focal lengths of the first camera and the second camera are different. According to the technical scheme, the optimal cutting coefficient can be determined in the zoom scene, the image quality of the preview image is improved, and then the zoom operation effect is guaranteed.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a shooting method, device, storage medium and chip. Background Art

[0002] In the related art, in a multi-camera zoom system, the zoom operation involves cameras with different focal lengths; it is necessary to perform cropping on the image captured by one of the cameras so that the processed image can be aligned with the image captured by another camera as much as possible to ensure the multi-camera zoom effect. During the cropping process, the cropping is performed according to the cropping factor. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a shooting method, device, storage medium and chip, which can determine the optimal cropping factor in the zoom scene, improve the quality of the preview image, and thus ensure the effect of the zoom operation.

[0004] According to a first aspect of an embodiment of the present disclosure, a shooting method is provided, comprising: in response to a zoom operation, determining a cropping factor according to a zoom ratio of an electronic device; cropping an image captured by a first camera according to the cropping factor to obtain a preview image, and displaying the preview image on a screen, wherein the image displayed on the screen is a gradient preview image, and the gradient preview image gradually aligns with an image captured by a second camera, and the focal lengths of the first camera and the second camera are different.

[0005] In an embodiment of the present disclosure, determining a cropping factor based on a zoom ratio of an electronic device includes: determining a first target disparity based on the zoom ratio; determining a target cropping factor based on the zoom ratio; and determining the cropping factor based on the first target disparity, the target cropping factor, and image parameters of an image displayed on the screen.

[0006] In an embodiment of the present disclosure, determining the cropping coefficient according to the first target disparity, the target cropping coefficient, and image parameters of the image displayed on the screen includes: determining the first cropping coefficient according to the first target disparity, the target cropping coefficient, and image parameters of the image displayed on the screen; determining the second cropping coefficient according to the first cropping coefficient, the target cropping coefficient, and a preset cropping coefficient relationship, wherein the preset cropping coefficient relationship is: the target cropping coefficient is equal to the product of the first cropping coefficient and the second cropping coefficient; cropping the image captured by the first camera according to the cropping coefficient to obtain a preview image, including: cropping the image captured by the first camera according to the first cropping coefficient to obtain a cropped image; cropping the cropped image according to the second cropping coefficient to obtain the preview image.

[0007] In an embodiment of the present disclosure, determining the first target parallax according to the zoom ratio includes: determining the parallax between the image captured by the first camera and the image captured by the second camera, a first minimum zoom ratio, and a second minimum zoom ratio, the first minimum zoom ratio being the minimum zoom ratio at which the image captured by the first camera can be displayed as a screen image, and the second minimum zoom ratio being the minimum zoom ratio at which the image captured by the second camera can be displayed as a screen image; determining the first target parallax according to the zoom ratio, the parallax between the image captured by the first camera and the image captured by the second camera, the first minimum zoom ratio, and the second minimum zoom ratio.

[0008] In an embodiment of the present disclosure, determining the parallax between an image captured by the first camera and an image captured by the second camera includes: determining an internal parameter of the first camera, a distance between the first camera and the second camera, and a minimum focusing distance of the first camera; and determining the parallax between the image captured by the first camera and the image captured by the second camera based on the internal parameter, the distance between the first camera and the second camera, and the minimum focusing distance.

[0009] In the embodiment of the present disclosure, determining the intrinsic parameter of the first camera includes: determining the intrinsic parameter of the first camera according to the effective focal length of the first camera and the pixel size corresponding to the first camera.

[0010] In an embodiment of the present disclosure, the product of the first target disparity and the target cropping coefficient is equal to the product of the second target disparity and the second cropping coefficient, and the second target disparity is the image disparity when the cropped image is cropped according to the second cropping coefficient.

[0011] In an embodiment of the present disclosure, a cropping frame parameter corresponding to the second cropping coefficient is equal to a ratio of the image parameter to the second cropping coefficient.

[0012] In an embodiment of the present disclosure, a preset relationship is satisfied between the second target disparity, the cropping frame parameters corresponding to the second cropping coefficient, and the image parameters, and the preset relationship is used to ensure that the preview image has no black edges.

[0013] In the embodiment of the present disclosure, determining the target cropping factor based on the zoom ratio includes: determining a first minimum zoom ratio, where the first minimum zoom ratio is the minimum zoom ratio at which the image captured by the first camera can be displayed as a screen image; determining the target cropping factor based on the zoom ratio and the first minimum zoom ratio.

[0014] In the embodiment of the present disclosure, if the parallax direction is a horizontal direction, the image parameter is the image height; if the parallax direction is a vertical direction, the image parameter is the image width.

[0015] According to a second aspect of an embodiment of the present disclosure, a shooting device is provided, including: a first module, configured to determine a cropping factor according to a zoom ratio of the electronic device in response to a zoom operation; a second module, configured to crop an image captured by the first camera according to the cropping factor to obtain a preview image, and display the preview image on a screen, wherein a field of view center of the preview image is aligned with a field of view center of an image captured by the second camera; a focal length of the first camera is different from a focal length of the second camera.

[0016] According to a third aspect of an embodiment of the present disclosure, a shooting device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to: execute the steps of the shooting method provided in the first aspect of the present disclosure.

[0017] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the shooting method provided in the first aspect of the present disclosure are implemented.

[0018] According to a fifth aspect of an embodiment of the present disclosure, a chip is provided, including a processor and an interface; the processor is used to read instructions to execute the steps of the shooting method provided in the first aspect of the present disclosure.

[0019] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: after the zoom operation is triggered, the first camera and the second camera with different focal lengths will both capture images, and the screen of the electronic device will display a gradual preview image obtained by cropping the image captured by the first camera according to the cropping factor, and the gradual preview image gradually aligns with the image captured by the second camera. The cropping factor is determined based on the zoom ratio of the electronic device. Thus, the cropping factor is optimized compared to the empirical value; and the cropping factor is the optimal cropping factor under the zoom ratio of the electronic device, which can improve the image quality of the preview image and thus ensure the effect of the zoom operation.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0022] Figure 1is a schematic diagram of a non-center cutting process provided according to an exemplary embodiment.

[0023] Figure 2 The figure is a flowchart of a shooting method according to an exemplary embodiment.

[0024] Figure 3 is a schematic diagram showing image relationship according to an exemplary embodiment.

[0025] Figure 4 The figure is a schematic diagram showing a front-end cropped image and a rear-end cropping frame according to an exemplary embodiment.

[0026] Figure 5A It is a front-end cropped image of the prior art according to an exemplary embodiment.

[0027] Figure 5B is a front-end cropped image of the present disclosure shown according to an exemplary embodiment.

[0028] Fig. 6A It is a preview image of the prior art shown according to an exemplary embodiment.

[0029] Figure 6B is a preview image of the present disclosure shown according to an exemplary embodiment.

[0030] Figure 7 is a block diagram of a shooting device according to an exemplary embodiment.

[0031] Figure 8 is a block diagram of a photographing device according to an exemplary embodiment.

[0032] Fig. 9 is a block diagram of a photographing device according to an exemplary embodiment. DETAILED DESCRIPTION

[0033] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0034] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0035] The technical solution provided by the present disclosure can be applied to scenarios involving multi-camera zoom. In these application scenarios, two or more cameras are involved, for example: a multi-camera zoom system of a mobile phone, a multi-camera zoom system of a camera.

[0036] In these multi-camera zoom scenarios, cameras with different focal lengths are involved. Take two cameras as an example, where one camera can be a wide-angle camera and the other camera can be a telephoto camera, then the wide-angle camera is smaller than the telephoto camera. For another example, one camera can be a wide-angle camera and the other camera can be an ultra-wide-angle camera. Or, there are more cases of two cameras with different focal lengths, which are not given one by one here.

[0037] In these multi-camera zoom scenarios, when zoom operations are involved, the corresponding camera switching will be involved. Before the camera is switched, the screen display image of the electronic device is the image captured by the camera before zooming. After the camera is switched, the screen display image of the electronic device will be switched to the image captured by the camera after zooming. Since the focal lengths of the two cameras are different, the effects of the images captured by the two cameras are also different. Therefore, in order to ensure that the image displayed on the screen does not jump when switching cameras, the field of view centers of the images captured by the two cameras need to be aligned, and this alignment operation depends on image cropping.

[0038] Taking two cameras as an example, assuming that one of the two cameras is a wide-angle camera and the other is a telephoto camera. The corresponding zoom process may include: when the user slides the camera from the wide-angle camera to the telephoto camera to zoom, the camera system first performs a center crop (a cropping method based on the center of the image) on the wide-angle image before a certain zoom ratio (preset value). After reaching a certain zoom ratio (preset value, different from the previous zoom ratio), the telephoto camera is turned on in the background. At this time, the camera system performs a non-center crop (a cropping method not based on the center of the image) on the wide-angle image, so that the wide-angle image slowly aligns with the telephoto image. After reaching the perspective switching zoom ratio, the wide-angle image is completely aligned with the telephoto image, so that when the perspective switches, the displayed image will not have a sense of jump.

[0039] Before non-center cropping, the camera system will transform the wide-angle image, and the position of the image after the transformation will change compared to the image before the transformation. Therefore, the subsequent cropping process is not based on the center of the image, so it is called non-center cropping.

[0040] It can be understood that if the electronic device includes more than two cameras, the corresponding zoom process is also performed based on the two cameras. That is, the technical solution provided by the present disclosure can also be applied to electronic devices including more than two cameras.

[0041] The technical solution provided by the present disclosure can be applied to the non-center cropping process in the above-mentioned zoom process. The non-center cropping process includes two cropping stages, which can be called front-end cropping and rear-end cropping. Both the front-end cropping and the rear-end cropping correspond to corresponding cropping coefficients, and the front-end cropping coefficient and the rear-end cropping coefficient are inversely proportional. That is, the larger the front-end cropping coefficient, the smaller the rear-end cropping coefficient.

[0042] Figure 1 is a schematic diagram of a non-center cutting process provided according to an exemplary embodiment. Figure 1 As shown, the non-center cropping process includes front-end cropping and back-end cropping. The cropping frame corresponding to the front-end cropping (the dotted frame in the figure) can represent the front-end cropping coefficient. After the original image is cropped based on the cropping frame, the front-end cropped image is obtained; the cropping frame corresponding to the back-end cropping can represent the back-end cropping coefficient. After the front-end cropped image is cropped based on the cropping frame, the back-end cropped image is obtained. After two croppings, the final image is different from the original image.

[0043] It can be understood that the front-end cutting and the back-end cutting here do not refer to cutting at different processing ends, but refer to two cutting stages.

[0044] Among them, based on the cropping factor, the cropping frame size can be determined. The cropping factor can represent the ratio of the original image size to the cropped image size. For example, if the cropping factor is 1, it means that there is no need to crop the original image; if the cropping factor is 2, the cropped image size is only half of the original image.

[0045] In relevant application scenarios, if the backend cropping factor is large, the image quality will be seriously lost, resulting in poor image application effects. For example, the image after backend cropping will be switched to the screen resolution through upsampling, and then displayed as a preview image; because upsampling will lose image quality, if the image quality before upsampling is low, the final preview image quality will be poor.

[0046] At present, both the front-end cropping factor and the back-end cropping factor are set based on experience. Usually, the front-end cropping factor is set smaller. When the front-end cropping factor is smaller, the back-end cropping factor will be larger, resulting in poor image quality and poor image application effect.

[0047] Based on this, the present disclosure provides a technical solution, in which the cropping factor is determined according to the zoom magnification of the electronic device. Thus, the cropping factor is optimized compared to the empirical value; and the cropping factor is the optimal cropping factor under the zoom magnification of the electronic device, which can improve the image quality of the preview image and thus ensure the effect of the zoom operation.

[0048] Figure 2 is a flowchart of a shooting method according to an exemplary embodiment. Figure 2 As shown, the shooting method is used in an electronic device, and the electronic device may include: a first camera, a second camera and a screen. The shooting method includes the following steps.

[0049] Step S21, in response to a zoom operation, determining a crop factor according to a zoom magnification of the electronic device.

[0050] Step S22, cropping the image captured by the first camera according to the cropping coefficient to obtain a preview image, and displaying the preview image on the screen. The image displayed on the screen is a gradual preview image, which gradually aligns with the image captured by the second camera, and the focal lengths of the first camera and the second camera are different.

[0051] In an exemplary embodiment, the zoom operation may be: a user's touch input operation on the corresponding shooting interface on the screen (such as a two-finger pinch or two-finger push operation). Or, a user's touch input operation on the zoom control displayed on the shooting interface (such as a press or slide operation). It may also be that the user moves the electronic device to make the camera closer to or farther away from the shooting object (i.e., automatically zooming based on the distance to the shooting object), etc.

[0052] In an exemplary embodiment, the focal length of the first camera is smaller than the focal length of the second camera. For example, the first camera is a wide-angle camera, and the second camera is a telephoto camera.

[0053] In another exemplary embodiment, the focal length of the first camera is greater than the focal length of the second camera. For example, the first camera is a telephoto camera, and the second camera is a wide-angle camera.

[0054] The zoom ratio of an electronic device can be changed according to the user's shooting needs. For example, if the user is currently shooting a wide-angle image and wants to switch to shooting a telephoto image, the zoom ratio will be switched to turn on the telephoto camera. During the zoom process, the wide-angle image needs to be aligned with the telephoto image so that there is no obvious jump in the image displayed on the screen when switching.

[0055] In some embodiments, if the current zoom ratio of the multi-camera zoom system reaches a preset zoom ratio, the second camera is turned on. The preset zoom ratio can be configured in advance according to the actual situation of the multi-camera zoom system, and the specific value is not limited here.

[0056] It can be understood that the zoom ratio of the electronic device will continue to change. When the zoom ratio has not reached the switching zoom ratio, the images displayed on the screen of the electronic device are all preview images corresponding to the images captured by the first camera. Therefore, the execution premise of step S22 may be that the zoom ratio is less than the switching zoom ratio.

[0057] In the disclosed embodiment, for the electronic device, during the zoom ratio change process, the zoom ratio corresponding to the first camera gradually approaches the zoom ratio corresponding to the second camera, and the preview image displayed is an image that gradually changes with the zoom ratio, and the gradual preview image gradually aligns with the image captured by the second camera.

[0058] Furthermore, when the zoom ratio of the electronic device reaches the switching zoom ratio, the screen of the electronic device displays a preview image corresponding to the image captured by the second camera.

[0059] In the embodiment of the present disclosure, the gradual preview image is gradually aligned with the image captured by the second camera. It can be that the center of the field of view of the gradual preview image is gradually aligned with the center of the field of view of the second camera; it can also be that the area of ​​interest in the gradual preview image is gradually aligned with the area of ​​interest in the image captured by the second camera.

[0060] As an optional implementation, step S21 includes: determining a first target disparity according to the zoom ratio; determining a target cropping factor according to the zoom ratio; and determining a cropping factor according to the first target disparity, the target cropping factor and image parameters of the image displayed on the screen.

[0061] In an exemplary embodiment, the cropping of an image involves two cropping stages, and thus the cropping coefficient also involves two cropping coefficients. Therefore, determining the cropping coefficient according to the first target disparity, the target cropping coefficient, and the image parameters of the image displayed on the screen may include: determining the first cropping coefficient according to the first target disparity, the target cropping coefficient, and the image parameters of the image displayed on the screen; determining the second cropping coefficient according to the first cropping coefficient, the target cropping coefficient, and a preset cropping coefficient relationship, wherein the preset cropping coefficient relationship is: the target cropping coefficient is equal to the product of the first cropping coefficient and the second cropping coefficient.

[0062] In this implementation, step S22 may include: cropping the image captured by the first camera according to the first cropping coefficient to obtain a cropped image; and cropping the cropped image according to the second cropping coefficient to obtain a preview image.

[0063] It can be understood that the cutting process in step S22 corresponds to the aforementioned non-center cutting process. Furthermore, the first cutting coefficient can be used as the front-end cutting coefficient, and the second cutting coefficient can be used as the back-end cutting coefficient.

[0064] In some embodiments, the parallax between the image center point of the image captured by the first camera and the image center point of the image captured by the second camera can be understood as the total parallax, and the first target parallax can be determined by the parallax between the image captured by the first camera and the image captured by the second camera.

[0065] In some embodiments, the target cropping factor is equal to the product of the first cropping factor and the second cropping factor, so that the first cropping factor and the second cropping factor are in inverse proportion, that is, the larger the first cropping factor is, the smaller the second cropping factor is; the smaller the first cropping factor is, the larger the second cropping factor is. Therefore, in the present disclosure, it is necessary to ensure that the cropping result corresponding to the first cropping factor retains the edge on the basis of ensuring that the first cropping factor is as large as possible, so that the cropping result corresponding to the second cropping factor does not have a black edge.

[0066] The image parameters of the image displayed on the screen of the electronic device can be understood as the parameters of the image displayed on the display screen of the electronic device, which is known information. Since the image aligned with the image captured by the second camera will be displayed as a preview image, the cropping factor needs to be determined in combination with the image parameters.

[0067] In some embodiments, the image parameter is related to the parallax direction. For example, if the parallax direction is horizontal, the image parameter is the image height; if the parallax direction is vertical, the image parameter is the image width.

[0068] As an optional implementation, determining the first target parallax according to the zoom ratio includes: determining the parallax between the image captured by the first camera and the image captured by the second camera, the first minimum zoom ratio and the second minimum zoom ratio, the first minimum zoom ratio being the minimum zoom ratio at which the image captured by the first camera can be displayed as a screen image, and the second minimum zoom ratio being the minimum zoom ratio at which the image captured by the second camera can be displayed as a screen image; determining the first target parallax according to the zoom ratio, the parallax between the image captured by the first camera and the image captured by the second camera, the first minimum zoom ratio and the second minimum zoom ratio.

[0069] The parallax between the image captured by the first camera and the image captured by the second camera is the parallax between the center of the image captured by the first camera and the center of the image captured by the second camera. In the embodiment of the present disclosure, a theoretical parallax is taken.

[0070] The zoom magnification of the electronic device is the zoom magnification currently used by the electronic device, which may depend on the shooting requirements of the user, and the zoom magnification value may be directly acquired.

[0071] The first minimum zoom ratio can be understood as the minimum zoom ratio at which the image captured by the first camera can be displayed as (supported by) a screen image. It can be a known parameter of the first camera or a parameter determined after multiple applications.

[0072] The second minimum zoom ratio can be understood as the minimum zoom ratio at which the image captured by the second camera can be displayed as (supported by) a screen image. It can be a known parameter of the second camera or a parameter determined after multiple applications.

[0073] As an optional implementation, determining the parallax between the image captured by the first camera and the image captured by the second camera includes: determining the internal parameters of the first camera, the distance between the first camera and the second camera, and the minimum focusing distance of the first camera; determining the parallax between the image captured by the first camera and the image captured by the second camera based on the internal parameters, the distance between the first camera and the second camera, and the minimum focusing distance of the first camera.

[0074] In this implementation, determining the internal parameter of the first camera may include: determining the internal parameter of the first camera according to the effective focal length of the first camera and the pixel size corresponding to the first camera.

[0075] For example, the intrinsic parameters of the first camera can be determined by the following formula:

[0076]

[0077] Among them, f is the intrinsic parameter of the first camera, EFL is the effective focal length, and pixel size is the pixel size.

[0078] It is understandable that when determining the parallax between the images, the distance between the first camera and the object needs to be used. In the disclosed embodiment, the minimum focus distance of the first camera is a known parameter of the camera, and it is used as the distance between the first camera and the object to calculate the maximum parallax of the first camera within the screen display range.

[0079] It can be understood that the distance between the first camera and the object is a variable during the shooting process. The minimum focus distance is taken here. As long as the final processed image does not have black edges at this distance, the final processed image will not have black edges when the distance is greater than this distance.

[0080] The distance between the first camera and the second camera may be the actual distance between the two cameras, and the actual distance is known information.

[0081] For example, the parallax between the image captured by the first camera and the image captured by the second camera can be determined by the following formula:

[0082]

[0083] Wherein, shift is the disparity between the image captured by the first camera and the image captured by the second camera, baseline is the distance between the first camera and the second camera, z is the minimum focus distance of the first camera, and f is the internal parameter of the first camera.

[0084] In some embodiments, the parallax between the image captured by the first camera and the image captured by the second camera, and the distance between the first camera and the second camera are theoretical values. Therefore, more accurate values ​​can be obtained by calibration, for example, by improving the accuracy by camera calibration. The camera calibration method can refer to mature technologies in the art and will not be described in detail here.

[0085] Furthermore, based on the parallax between the image captured by the first camera and the image captured by the second camera, combined with the zoom ratio of the electronic device, the first minimum zoom ratio, and the second minimum zoom ratio, parallax splitting is performed to determine the first target parallax.

[0086] For example, the first target disparity can be determined by the following formula:

[0087]

[0088] Among them, shift_sub is the first target disparity, user_ratio is the zoom ratio, shift is the disparity between the image captured by the first camera and the image captured by the second camera, min_ratio is the first minimum zoom ratio, and max_ratio is the second minimum zoom ratio.

[0089] Taking a wide-angle camera as an example, min_ratio can be 0.6x and max_ratio can be 1.0x.

[0090] As an optional implementation, determining a target cropping factor based on a zoom ratio includes: determining a first minimum zoom ratio, the first minimum zoom ratio being the minimum zoom ratio at which an image captured by a first camera can be displayed as a screen image; determining a target cropping factor based on the zoom ratio and the first minimum zoom ratio.

[0091] For example, the target crop factor can be determined by the following formula:

[0092]

[0093] Among them, crop_all is the target cropping factor, user_ratio is the zoom ratio, and min_ratio is the first minimum zoom ratio.

[0094] In the embodiment of the present disclosure, the first target disparity, the target cropping coefficient, the image parameter and the first cropping coefficient satisfy a preset relationship, and the preset relationship can be derived according to the conditions that the first cropping coefficient needs to satisfy. The derivation process is described below. It should be noted that in the following derivation process, the same parameters involved in different formulas are only explained when the parameters appear for the first time. The same parameters that appear subsequently should be understood as the same definition and no repeated explanation is given.

[0095] The product of the first crop factor and the second crop factor is equal to the target crop factor, which can be expressed as:

[0096] crop_all = crop_f*crop_p

[0097] Among them, crop_all is the target cropping coefficient, crop_f is the first cropping coefficient, and crop_p is the second cropping coefficient.

[0098] In some embodiments, the image captured by the first camera and the second target disparity; the second target disparity is the image disparity when the cropped image is cropped according to the second cropping coefficient. And the product of the first target disparity and the target cropping coefficient is equal to the product of the second target disparity and the second cropping coefficient. This relationship can be expressed as:

[0099] shift_p*crop_p=shift_sub*crop_all

[0100] Among them, shift_sub is the first target disparity, and shift_p is the second target disparity.

[0101] Furthermore, the cropping frame parameter corresponding to the second cropping factor is equal to the ratio of the image parameter to the second cropping factor, and this relationship can be understood as a known and fixed relationship. Taking the image parameter as the image height as an example, this relationship can be expressed as:

[0102]

[0103] Among them, crop_height is the cropping box height corresponding to the second cropping factor, and preview_height is the image height.

[0104] It can be understood that if the image parameter is the image width, the image height in the above formula is replaced by the image width, and the crop box height is replaced by the crop box width. For example:

[0105]

[0106] Among them, crop_weight is the cropping box width corresponding to the second cropping factor, and preview_weight is the image width.

[0107] Furthermore, the second target disparity, the cropping frame parameters corresponding to the second cropping coefficient, and the image parameters satisfy a preset relationship, and the preset relationship is used to ensure that the preview image has no black edges.

[0108] Please refer to Figure 3 , is a schematic diagram of image relationship according to an exemplary embodiment. Figure 3 As shown, the image after front-end cropping is translated, and its position changes compared with the image before the translation. In order to ensure that there is no black edge on the image after back-end cropping, the sum of half the cropping frame height and the parallax to the second target needs to be equal to half the image height, and this relationship is the preset relationship.

[0109] For example, the preset relationship can be expressed by the following formula:

[0110]

[0111] Further, based on the above-listed formulas, the relationship between the first cropping coefficient, the first target disparity, the image parameter and the target cropping coefficient can be derived. For example, the relationship can be expressed by the following formula:

[0112]

[0113] It can be understood that if the image parameter is the image width, the image height in the above formula is replaced by the image width, and the crop box height is replaced by the crop box width. For example:

[0114]

[0115] Furthermore, by utilizing the relationship represented by the formula and combining the first target disparity, the image parameter and the target cropping coefficient, the first cropping coefficient can be determined.

[0116] Furthermore, since the relationship between the first cropping factor and the second cropping factor is also fixed, the second cropping factor can also be determined based on the first cropping factor.

[0117] Further, Figure 4 is a schematic diagram showing a front-end cropped image and a rear-end cropping frame according to an exemplary embodiment. Figure 4 As shown, by using the first cropping coefficient and the second cropping coefficient of the present disclosure, the image after the front end is cropped can be cropped just enough to avoid black edges in the back end.

[0118] In some embodiments, after cropping the image captured by the first camera, the cropped image can be aligned with the image captured by the second camera using a preset image alignment algorithm. In this way, during the zoom ratio change, the gradually changing preview image is gradually aligned with the image captured by the second camera.

[0119] In some embodiments, the image cropped by the first cropping factor is input into other algorithm modules for relevant calculations before further cropping. For example: 3A module. 3A refers to: Autofocus: Control the displacement of the focus motor to make the lens focus at the appropriate position. Autoexposure: Adjust the shutter, aperture, and sensitivity to make the image reach the appropriate brightness. Autowhite balance: Adjust the color temperature of the image by adjusting the value of the R / B channel of the image.

[0120] When applied to the 3A algorithm module, since the invalid areas (irrelevant image parts) in the image cropped by the first cropping coefficient are smaller, the influence of the invalid areas can be reduced and the accuracy of the 3A module can be improved. Figure 5A is a front-end cropped image of the prior art according to an exemplary embodiment, such as Figure 5A As shown, the left side of the image contains an invalid area. Figure 5B is a front-end cropped image of the present disclosure according to an exemplary embodiment, such as Figure 5B As shown, the image has less invalid area compared with the image after front-end cropping in the prior art.

[0121] Furthermore, the idea of ​​determining the front-end cropping coefficient in the technical solution of the disclosed embodiment is to make the front-end cropping coefficient as large as possible while ensuring that no black edges appear in the back-end cropping. The benefit brought about is less back-end cropping, which can significantly improve the preview image quality.

[0122] Fig. 6A is a preview image of the prior art according to an exemplary embodiment. Figure 5A The local area of ​​the image shown, Figure 6B is a preview image of the present disclosure shown according to an exemplary embodiment, and the preview image is Figure 5B The local area of ​​the image shown. Fig. 6A and Figure 6B By comparison, we can see Fig. 6A The image quality is significantly lower than Figure 6B image quality.

[0123] Therefore, by using the optimal cropping coefficient at each user magnification, the front end can be cropped as much as possible while retaining sufficient image edges, which can significantly improve the quality of the cropped image; and further improve the quality of the preview image obtained based on the cropped image.

[0124] Figure 7 is a block diagram of a photographing device according to an exemplary embodiment. Figure 7 , the device includes a first module 701 and a second module 702.

[0125] The first module 701 is configured to determine a crop factor according to a zoom magnification of the electronic device in response to a zoom operation;

[0126] The second module 702 is configured to crop the image captured by the first camera according to the cropping factor to obtain a preview image, and display the preview image on the screen of the electronic device, wherein the image displayed on the screen is a gradient preview image, and the gradient preview image gradually aligns with the image captured by the second camera, and the focal lengths of the first camera and the second camera are different.

[0127] In an exemplary embodiment, the first module 701 is further configured to: determine a first target disparity according to the zoom ratio; determine a target cropping factor according to the zoom ratio; determine the cropping factor according to the first target disparity, the target cropping factor and image parameters of the image displayed on the screen.

[0128] The first module 701 includes a first determination module, which is configured to: determine a first cropping coefficient according to the first target disparity, the target cropping coefficient and the image parameters of the image displayed on the screen; the second determination module is configured to determine a second cropping coefficient according to the first cropping coefficient, the target cropping coefficient and a preset cropping coefficient relationship, and the preset cropping coefficient relationship is: the target cropping coefficient is equal to the product of the first cropping coefficient and the second cropping coefficient; the second module 702 is further configured to: crop the image captured by the first camera according to the first cropping coefficient to obtain a cropped image; crop the cropped image according to the second cropping coefficient to obtain the preview image.

[0129] In an exemplary embodiment, the first module 701 includes: a parallax determination module, configured to: determine the parallax between the image captured by the first camera and the image captured by the second camera, a first minimum zoom ratio, and a second minimum zoom ratio, the first minimum zoom ratio being the minimum zoom ratio at which the image captured by the first camera can be displayed as a screen image, and the second minimum zoom ratio being the minimum zoom ratio at which the image captured by the second camera can be displayed as a screen image; determine the first target parallax according to the zoom ratio, the parallax between the image captured by the first camera and the image captured by the second camera, the first minimum zoom ratio, and the second minimum zoom ratio.

[0130] In an exemplary embodiment, the disparity determination module is further configured to: determine the internal parameters of the first camera, the distance between the first camera and the second camera, and the minimum focusing distance of the first camera; and determine the disparity between the image captured by the first camera and the image captured by the second camera based on the internal parameters, the distance between the first camera and the second camera, and the minimum focusing distance.

[0131] In an exemplary embodiment, the photographing device further includes: an internal parameter determination module configured to determine the internal parameter of the first camera according to the effective focal length of the first camera and the pixel size corresponding to the first camera.

[0132] In an exemplary embodiment, the first module 701 is further configured to: determine a first minimum zoom ratio, where the first minimum zoom ratio is the minimum zoom ratio at which the image captured by the first camera can be displayed as a screen image; and determine the target cropping factor based on the zoom ratio and the first minimum zoom ratio.

[0133] In an exemplary embodiment, the product of the first target disparity and the target cropping factor is equal to the product of the second target disparity and the second cropping factor, and the second target disparity is the image disparity when the cropped image is cropped according to the second cropping factor.

[0134] In an exemplary embodiment, a cropping frame parameter corresponding to the second cropping factor is equal to a ratio of the image parameter to the second cropping factor.

[0135] In an exemplary embodiment, a preset relationship is satisfied between the second target disparity, the cropping frame parameter corresponding to the second cropping coefficient, and the image parameter, and the preset relationship is used to ensure that the preview image has no black edges.

[0136] In an exemplary embodiment, if the parallax direction is a horizontal direction, the image parameter is an image height; if the parallax direction is a vertical direction, the image parameter is an image width.

[0137] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0138] The present disclosure also provides a computer-readable storage medium on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the shooting method provided by the present disclosure are implemented.

[0139] Figure 88 is a block diagram of a photographing device 800 according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0140] Reference Figure 8 , the device 800 may include one or more of the following components: a first processing component 802 , a first memory 804 , a first power supply component 806 , a multimedia component 808 , an audio component 810 , a first input / output interface 812 , a sensor component 814 , and a communication component 816 .

[0141] The first processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The first processing component 802 may include one or more first processors 820 to execute instructions to complete all or part of the steps of the above-described method. In addition, the first processing component 802 may include one or more modules to facilitate interaction between the first processing component 802 and other components. For example, the first processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the first processing component 802.

[0142] The first memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The first memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0143] The first power source component 806 provides power to various components of the device 800. The first power source component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 800.

[0144] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0145] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the first memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0146] The first input / output interface 812 provides an interface between the first processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0147] The sensor assembly 814 includes one or more sensors for providing various aspects of the status assessment of the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0148] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0149] In an exemplary embodiment, the device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above-mentioned shooting method.

[0150] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a first memory 804 including instructions, and the instructions can be executed by the first processor 820 of the device 800 to complete the above-mentioned shooting method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access first memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0151] In addition to being an independent electronic device, the above-mentioned device can also be a part of an independent electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be an IC or a collection of multiple ICs; the chip can include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC), etc. The above-mentioned integrated circuit or chip can be used to execute executable instructions (or codes) to implement the above-mentioned shooting method. The executable instructions can be stored in the integrated circuit or chip, or can be obtained from other devices or equipment, for example, the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instruction may be stored in the memory, and when the executable instruction is executed by the processor, the above-mentioned shooting method is implemented; alternatively, the integrated circuit or chip may receive the executable instruction through the interface and transmit it to the processor for execution, so as to implement the above-mentioned shooting method.

[0152] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device. The computer program has a code portion for executing the above-mentioned photographing method when executed by the programmable device.

[0153] Fig. 9 900 is a block diagram of a photographing device 900 according to an exemplary embodiment. For example, the device 900 may be provided as a server. Fig. 9 The device 900 includes a second processing component 922, which further includes one or more second processors, and a second memory resource represented by a second memory 932, for storing instructions executable by the second processing component 922, such as an application. The application stored in the second memory 932 may include one or more modules, each corresponding to a set of instructions. In addition, the second processing component 922 is configured to execute the instructions to perform the above-mentioned shooting method.

[0154] The device 900 may further include a second power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and a second input / output interface 958. The device 900 may operate based on an operating system stored in the second memory 932, such as Windows Server 2000. TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0155] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0156] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A shooting method, characterized in that: include: In response to a zoom operation, determining a crop factor according to a zoom magnification of the electronic device; The image captured by the first camera is cropped according to the cropping factor to obtain a preview image, and the preview image is displayed on the screen of the electronic device, wherein the image displayed on the screen is a gradient preview image, and the gradient preview image gradually aligns with the image captured by the second camera, and the focal lengths of the first camera and the second camera are different.

2. The shooting method according to claim 1, characterized in that: The step of determining the cropping factor according to the zoom ratio of the electronic device includes: determining a first target parallax according to the zoom ratio; Determining a target crop factor according to the zoom ratio; The cropping coefficient is determined according to the first target disparity, the target cropping coefficient, and image parameters of the image displayed on the screen.

3. The shooting method according to claim 2, characterized in that: The determining the cropping coefficient according to the first target disparity, the target cropping coefficient and the image parameter of the image displayed on the screen comprises: determining a first cropping coefficient according to the first target disparity, the target cropping coefficient, and image parameters of the image displayed on the screen; Determining a second cropping factor according to the relationship between the first cropping factor, the target cropping factor and a preset cropping factor, wherein the preset cropping factor relationship is: the target cropping factor is equal to the product of the first cropping factor and the second cropping factor; Cropping the image captured by the first camera according to the cropping factor to obtain a preview image, including: Cropping the image captured by the first camera according to the first cropping coefficient to obtain a cropped image; The cropped image is cropped according to the second cropping coefficient to obtain the preview image.

4. The shooting method according to claim 2, characterized in that: The determining the first target disparity according to the zoom ratio includes: Determine a parallax between an image captured by the first camera and an image captured by the second camera, a first minimum zoom ratio, and a second minimum zoom ratio, wherein the first minimum zoom ratio is a minimum zoom ratio at which the image captured by the first camera can be displayed as a screen image, and the second minimum zoom ratio is a minimum zoom ratio at which the image captured by the second camera can be displayed as a screen image; The first target disparity is determined according to the zoom ratio, the disparity between the image captured by the first camera and the image captured by the second camera, the first minimum zoom ratio, and the second minimum zoom ratio.

5. The shooting method according to claim 4, characterized in that: Determining a disparity between an image captured by the first camera and an image captured by the second camera includes: Determine an internal parameter of the first camera, a distance between the first camera and the second camera, and a minimum focusing distance of the first camera; The parallax between the image captured by the first camera and the image captured by the second camera is determined according to the internal parameter, the distance between the first camera and the second camera, and the minimum focus distance.

6. The shooting method according to claim 5, characterized in that: Determining the internal parameters of the first camera includes: An intrinsic parameter of the first camera is determined according to an effective focal length of the first camera and a pixel size corresponding to the first camera.

7. The shooting method according to claim 3, characterized in that: The product of the first target disparity and the target cropping coefficient is equal to the product of the second target disparity and the second cropping coefficient, and the second target disparity is the image disparity when the cropped image is cropped according to the second cropping coefficient.

8. The shooting method according to claim 3, characterized in that: The cropping frame parameter corresponding to the second cropping factor is equal to the ratio of the image parameter to the second cropping factor.

9. The shooting method according to claim 7, characterized in that: The second target disparity, the cropping frame parameters corresponding to the second cropping coefficient, and the image parameters satisfy a preset relationship, and the preset relationship is used to ensure that the preview image has no black edges.

10. The shooting method according to claim 2, characterized in that: The determining a target cropping factor according to the zoom ratio includes: Determine a first minimum zoom ratio, where the first minimum zoom ratio is a minimum zoom ratio at which an image captured by the first camera can be displayed as a screen image; The target cropping factor is determined according to the zoom magnification and the first minimum zoom magnification.

11. The shooting method according to any one of claims 2 to 10, characterized in that: If the parallax direction is a horizontal direction, the image parameter is the image height; if the parallax direction is a vertical direction, the image parameter is the image width.

12. A photographing device, characterized in that: include: A first module is configured to determine a crop factor according to a zoom magnification of the electronic device in response to a zoom operation; The second module is configured to crop the image captured by the first camera according to the cropping factor to obtain a preview image, and display the preview image on the screen of the electronic device, wherein the image displayed on the screen is a gradient preview image, and the gradient preview image gradually aligns with the image captured by the second camera, and the focal lengths of the first camera and the second camera are different.

13. A photographing device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to: execute the steps of the method described in any one of claims 1-11.

14. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 11 are implemented.

15. A chip, characterized in that: The method comprises a processor and an interface; the processor is used to read instructions to execute the steps of the method according to any one of claims 1 to 11.

Images