Macro photography method, electronic device and computer-readable storage medium

Abstract

The embodiment of the present application discloses a macro shooting method, an electronic device and a computer-readable storage medium, which are used to switch and relay through at least two macro cameras with different focal lengths during the zoom process of the macro mode to achieve macro shooting with a wider focal length. The method includes: in the macro shooting mode, when the zoom ratio is in a first ratio range, the picture captured by the first macro camera is displayed in the viewfinder; when the zoom ratio is the camera switching ratio, the picture displayed in the viewfinder is switched from the picture captured by the first macro camera to the picture captured by the second macro camera, so as to display the picture captured by the second macro camera in the viewfinder; when the zoom ratio is greater than the camera switching ratio and is in a second ratio range, the picture captured by the second macro camera is displayed in the viewfinder, and the second ratio range includes the camera switching ratio; wherein the focal length of the first macro camera is different from the focal length of the second macro camera.

Description

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on April 2, 2022, with application number 202210351277.3 and application name “Macro photography method, electronic device and computer-readable storage medium”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of imaging technology, and in particular to a macro shooting method, an electronic device, and a computer-readable storage medium. Background Art

[0003] Macro photography can capture tiny objects at a very close distance, giving users a different photography experience. With the maturity of the miniaturization technology of macro camera modules, it has become possible to realize macro photography on miniaturized and portable electronic devices.

[0004] Currently, macro photography is usually achieved using a single fixed-focus macro camera. For example, a mobile phone uses the ultra-close focusing capability and digital zoom of an ultra-wide-angle macro camera to achieve ultra-wide-angle macro photography.

[0005] However, a single fixed-focus macro camera has a single focal length, covers a small focal length, and has a poor macro shooting effect. Summary of the invention

[0006] The embodiments of the present application provide a macro shooting method, an electronic device, and a computer-readable storage medium, which can solve the problem that the existing macro shooting can only cover a single focal length and the macro shooting effect is poor.

[0007] In a first aspect, an embodiment of the present application provides a macro shooting method, which is applied to an electronic device, the electronic device including at least two macro cameras, the method comprising:

[0008] In the macro shooting mode, when the zoom ratio is in the first ratio range, the image captured by the first macro camera is displayed in the viewfinder;

[0009] When the zoom ratio is the camera switching ratio, the picture displayed in the viewfinder is switched from the picture captured by the first macro camera to the picture captured by the second macro camera;

[0010] When the zoom ratio is greater than the camera switching ratio and is within a second ratio range, the image captured by the second macro camera is displayed in the viewfinder, and the second ratio range includes the camera switching ratio;

[0011] Among them, the focal length of the first macro camera is different from the focal length of the second macro camera; the at least two macro cameras include the first macro camera and the second macro camera.

[0012] It can be seen from the above technical solution that in the zoom process of the macro shooting mode, the embodiment of the present application switches and relays macro cameras based on different focal lengths to achieve macro shooting with a wider focal length, and the macro shooting effect is better.

[0013] Exemplarily, the first macro camera is an ultra-wide-angle macro camera, the second macro camera is a telephoto macro camera, the first magnification range is 0.5x to 3.5x, the second magnification range is 3.5x to 15x, and the camera switching magnification is 3.5x. When the zoom ratio is in the first magnification range, i.e., the ultra-wide-angle focal length, the ultra-wide-angle macro camera is used for macro photography; when the zoom ratio reaches the camera switching ratio, the ultra-wide-angle macro camera is switched to the telephoto macro camera; when the zoom ratio is in the second magnification range, i.e., the telephoto focal length, the telephoto macro camera is used for macro photography. In this way, macro photography from the ultra-wide-angle focal length to the telephoto focal length is achieved. The picture displayed in the viewfinder can be a full FOV picture or a partial FOV picture captured by the macro camera.

[0014] In some possible implementations of the first aspect, the method further includes:

[0015] A navigation window is superimposed on the preview picture displayed in the viewfinder, and the navigation window is used to display the navigation picture; wherein the field of view of the navigation picture is greater than the field of view of the preview picture.

[0016] In this implementation, a navigation screen with a larger field of view (FOV) is superimposed on the preview screen, which facilitates the user to find the subject and composition during macro shooting, and improves the user's macro shooting experience. The navigation screen and the preview screen can be a full FOV screen or a partial FOV screen captured by the macro camera.

[0017] In some possible implementations of the first aspect, when the zoom ratio is in the second ratio range, the navigation screen is a screen captured by the first macro camera, and the preview screen is a screen captured by the second macro camera. Exemplarily, the screen captured by the telephoto macro camera is used as the preview screen, and the screen captured by the ultra-wide-angle macro camera is used as the navigation screen.

[0018] In some possible implementations of the first aspect, when the zoom ratio is in a first ratio range, the navigation screen and the preview screen are both pictures captured by the first macro camera; and / or, when the zoom ratio is in a second ratio range, the navigation screen and the preview screen are both pictures captured by the second macro camera.

[0019] In some possible implementations of the first aspect, when the zoom ratio is in a first ratio range, the navigation screen and the preview screen are both pictures captured by the first macro camera; when the zoom ratio is in a second ratio range, the navigation screen is the picture captured by the first macro camera, and the preview screen is the picture captured by the second macro camera.

[0020] In some possible implementations of the first aspect, when the zoom magnification is in a first magnification range, the navigation screen and the preview screen are both pictures captured by the first macro camera; when the zoom magnification is in a third magnification range, the navigation screen is the picture captured by the first macro camera, and the preview screen is the picture captured by the second macro camera; when the zoom magnification is in a fourth magnification range, the navigation screen and the preview screen are both pictures captured by the second macro camera; wherein the second magnification range includes the third magnification range and the fourth magnification range, and the third magnification range includes the camera switching magnification.

[0021] Exemplarily, the first magnification range is 0.5x~3.5x, at this time it is ultra-wide-angle preview and ultra-wide-angle navigation; the second magnification range is 3.5x~15x, and the third magnification range is 3.5x~10x, at this time it is telephoto preview and ultra-wide-angle navigation; the second magnification range is greater than 10x, at this time it is telephoto preview and telephoto navigation.

[0022] In some possible implementations of the first aspect, the zoom ratio is within a first ratio range;

[0023] The image captured by the first macro camera is displayed in the viewfinder, and a navigation window is superimposed on the preview image displayed in the viewfinder, including:

[0024] During the zooming process in the first magnification range, the image captured by the first macro camera is subjected to image offset cropping according to the initially calibrated image offset value or the last updated image offset value to obtain a first cropped image; according to the image offset value, the position of the first cropped image in the image captured by the first macro camera is identified by an identification pattern to obtain a first identification image; the first cropped image is displayed in a viewfinder, and the first identification image is displayed in a navigation window of the viewfinder; wherein the identification pattern is used to identify the relative position relationship between the preview screen and the navigation screen.

[0025] Furthermore, the relative position relationship between the preview screen and the navigation screen is obtained by image matching, and is marked in the navigation window by an identification pattern, so that the user can know the position of the preview screen FOV in the navigation screen FOV based on the identification pattern, which is convenient for macro composition and finding the shooting subject. Exemplarily, the identification pattern is a rectangular frame.

[0026] In addition, the image captured by the first macro camera is cropped according to the image offset value, so that the picture transition during zooming is smoother and more natural, thereby improving the user's macro shooting experience.

[0027] Exemplarily, the first magnification range is 0.5x to 3.5x. During the zoom process of 0.5x to 3.5x, the image offset value between the center of the telephoto picture and the center of the ultra-wide-angle picture is determined by image matching, and the ultra-wide-angle picture is cropped according to the image offset value, and the cropped image is used as the display image, and the position of the preview picture FOV in the navigation picture FOV is marked according to the image offset value to obtain an ultra-wide-angle image including a rectangular frame (i.e., the first identification image), and the ultra-wide-angle image including the rectangular frame and the cropped image are superimposed and displayed to achieve ultra-wide-angle preview and ultra-wide-angle navigation. In a specific application, the ultra-wide-angle image can be first reduced, and then the position of the preview picture can be marked in the reduced image according to the offset value to obtain a navigation picture including the mark, and then the navigation picture including the mark can be displayed at the corresponding position of the viewfinder, and the navigation window is smaller than the viewfinder, that is, the viewfinder includes the navigation window.

[0028] In some possible implementations of the first aspect, the method further includes:

[0029] Image feature matching is performed based on the first cropped image and the image captured by the second macro camera to obtain a first image feature matching result; an image offset value of the image captured by the first macro camera is determined based on the first image feature matching result and the zoom ratio of the image captured by the first macro camera, and the offset value is updated based on the image offset value of the image captured by the first macro camera. When matching image features, the first cropped image and the image captured by the second macro camera may be downsampled first, and then the two downsampled reduced images may be matched for image features to reduce the amount of calculation.

[0030] In some possible implementations of the first aspect, determining, according to the first image feature matching result and the zoom ratio of the image captured by the first macro camera, an image offset value of the image captured by the first macro camera includes:

[0031] By formula Obtain an image offset value of an image captured by the first macro camera; wherein offset represents an offset value from the center of the field of view of the first macro camera to the center of the field of view of the second macro camera, which is a pre-calibrated offset value or an image feature matching result; offset pre Indicates the image offset value of the previous frame, offset cur Indicates the image offset value of the current frame image; zoom pre Indicates the zoom ratio of the previous frame, zoom curIndicates the zoom ratio of the current frame image; zoom end Indicates the zoom ratio at which the image shift ends, which is less than or equal to the camera switching ratio; zoom start Indicates the zoom ratio at which the image shift starts, which is greater than or equal to the minimum zoom ratio of the first macro camera.

[0032] In some possible implementations of the first aspect, the zoom ratio is within a second ratio range or a third ratio range; a picture captured by the second macro camera is displayed in a viewfinder, and a navigation window is superimposed on a preview picture displayed in the viewfinder, including:

[0033] During the zooming process in the second magnification range or the third magnification range, the image captured by the second macro camera is digitally zoomed and cropped to obtain a second cropped image; according to the initial calibrated image offset value or the last updated image offset value, the position of the second cropped image in the target image is identified by the identification pattern to obtain a second identification image; the second cropped image is displayed in the viewfinder, and the second identification image is displayed in the navigation window of the viewfinder; wherein the target image is the third cropped image obtained by performing image offset cropping on the image captured by the first macro camera according to the initial calibrated image offset value or the updated image offset value; or, the full field angle image captured by the first macro camera. In this implementation, by superimposing the navigation screen on the preview screen, it is convenient for users to compose macro shots and find the subject.

[0034] For example, a telephoto image captured by a telephoto macro camera is used as a preview image, and an image captured by an ultra-wide-angle macro camera is used as a navigation image. The second identification image may be a telephoto cropped image including a rectangular frame; in a specific application, the position of the preview FOV may be marked in the image cropped from the ultra-wide-angle image according to the offset value, and the marked image may be obtained, and then a reduced image of the marked image may be obtained to obtain a navigation image.

[0035] In some possible implementations of the first aspect, the method further includes: performing image feature matching based on the second cropped image and the target image to obtain a second image feature matching result; determining an image offset value of the target image based on the second image feature matching result and a zoom ratio of the target image, and updating the offset value based on the image offset value of the target image.

[0036] In some possible implementations of the first aspect, the zoom ratio is within a fourth ratio range; the image captured by the second macro camera is displayed in the viewfinder, and a navigation window is superimposed on the preview image displayed in the viewfinder, including: digitally zooming and cropping the image captured by the second macro camera to obtain a fourth cropped image; identifying the position of the fourth cropped image in the image captured by the second macro camera by an identification pattern to obtain a third identification image; displaying the fourth cropped image in the viewfinder, and displaying the third identification image in the navigation window of the viewfinder. In this implementation, by superimposing the navigation screen on the preview screen, it is convenient for users to compose macro shots and find shooting subjects.

[0037] For example, the full FOV image of the telephoto macro camera is used as the navigation image, and the partial FOV image is used as the preview image. In a specific application, the full FOV image of the telephoto camera can be first reduced, and then the position of the preview FOV can be marked in the reduced image.

[0038] In some possible implementations of the first aspect, the method also includes: if the zoom ratio is in the fifth magnification range and a photo operation is detected, then in response to the photo operation, the image captured by the first macro camera is output; if the zoom ratio is in the sixth magnification range and a photo operation is detected, then in response to the photo operation, the image captured by the first macro camera and the image captured by the second macro camera are fused to obtain a fused image, and the fused image is output; if the zoom ratio is in the second magnification range and a photo operation is detected, then in response to the photo operation, the image captured by the second macro camera is output; wherein the first magnification range includes the fifth magnification range and the sixth magnification range.

[0039] For example, the first magnification range is 0.5x to 3.5x, the fifth magnification range is 0.5x to 2x, at this time, the ultra-wide-angle macro camera is used to take pictures; the sixth magnification range is 2x to 3.5x, at this time, the ultra-wide-angle image and the telephoto image are fused; when it is greater than or equal to 3.5x, the telephoto macro camera is used to take pictures. In this implementation, the quality of the pictures is improved by image fusion, and the user's macro shooting experience is further improved.

[0040] In some possible implementations of the first aspect, the fused image includes a target area and a non-target area, and the image quality of the target area is higher than that of the non-target area; the target area is an area in the image captured by the first macro camera corresponding to the image captured by the second macro camera. Exemplarily, the target area is an area in the ultra-wide-angle image corresponding to the telephoto image FOV. In this case, the ultra-wide-angle image with a larger FOV covers the telephoto FOV image.

[0041] In some possible implementations of the first aspect, the method further includes: detecting a first shooting distance of an image captured by a target camera; if the first shooting distances of N consecutive frames of images are all less than a first threshold, switching from the target camera to a default macro camera, and displaying the image captured by the default macro camera in the viewfinder; the default macro camera is the first macro camera or the second macro camera. In this implementation, when it is determined that the conditions for automatically entering the macro shooting mode are met, the macro shooting mode is automatically entered, further improving the macro shooting experience. Exemplarily, the target camera is the main camera.

[0042] In some possible implementations of the first aspect, the method further includes: if the first shooting distances of the N consecutive frames of images are all less than the first threshold, displaying a prompt message in the viewfinder, the prompt message being used to prompt that the macro shooting mode has been entered. In this implementation, the prompt message prompts the user that the macro shooting mode has been entered, further improving the macro shooting experience.

[0043] In some possible implementations of the first aspect, the method further includes: detecting a second shooting distance of an image captured by the current macro camera; if the second shooting distances of the consecutive M frames of images are greater than a second threshold, switching from the current macro camera to the default camera, and displaying the image captured by the default camera in the viewfinder; the current macro camera is the first macro camera or the second macro camera. In this implementation, when it is determined that the conditions for automatically exiting the macro shooting mode are met, the macro shooting mode is automatically exited, further improving the macro shooting experience. Exemplarily, the default camera can be a main camera or another camera.

[0044] In some possible implementations of the first aspect, the viewfinder further includes a first shortcut control and a second shortcut control; the first shortcut control is used to switch the picture displayed in the viewfinder from the picture captured by the first macro camera to the picture captured by the second macro camera; the second shortcut control is used to switch the picture displayed in the viewfinder from the picture captured by the second macro camera to the picture captured by the first macro camera. In this implementation, the user can quickly switch the macro camera through the shortcut control, further improving the user experience. Exemplarily, the first shortcut control is a telephoto macro shortcut control, and the second shortcut control is an ultra-wide-angle macro shortcut control.

[0045] In a second aspect, an embodiment of the present application provides a macro shooting method, which is applied to an electronic device, and the method includes:

[0046] In the macro shooting mode, the image captured by the first macro camera is displayed in the viewfinder, and a navigation window is superimposed on the preview image displayed in the viewfinder. The navigation window is used to display the navigation image, and the field of view of the navigation image is greater than the field of view of the preview image.

[0047] It can be seen from the above technical solution that the embodiment of the present application facilitates users to compose macro images and find shooting subjects by superimposing a navigation screen with a larger FOV on the preview screen, thereby improving the user's macro shooting experience. Exemplarily, the first macro camera is a telephoto macro camera or an ultra-wide-angle macro camera.

[0048] In some possible implementations of the second aspect, the navigation picture is a picture captured by a first macro camera; or a picture captured by a second macro camera, and the field of view of the second macro camera is greater than the field of view of the first macro camera.

[0049] In some possible implementations of the second aspect, the navigation picture is a picture captured by the first macro camera; the picture captured by the first macro camera is displayed in a viewfinder, and the navigation window is superimposed on the preview picture displayed in the viewfinder, including:

[0050] According to the initially calibrated image offset value, or the last updated image offset value, the image captured by the first macro camera is subjected to image offset cropping to obtain a first cropped image; according to the image offset value, the position of the first cropped image in the image captured by the first macro camera is identified by an identification pattern to obtain a first identification image; the first cropped image is displayed in a viewfinder, and the first identification image is displayed in a navigation window of the viewfinder; wherein the identification pattern is used to identify the relative position relationship between the preview screen and the navigation screen.

[0051] Exemplarily, the first macro camera is an ultra-wide-angle macro camera, and the full FOV image captured by the ultra-wide-angle macro camera is used as the navigation image, and the partial FOV image captured by the ultra-wide-angle macro camera is used as the preview image.

[0052] In some possible implementations of the second aspect, the method further includes:

[0053] Image feature matching is performed based on the first cropped image and the image captured by the second macro camera to obtain a first image feature matching result; an image offset value of the image captured by the first macro camera is determined based on the first image feature matching result and the zoom ratio of the image captured by the first macro camera, and the offset value is updated based on the image offset value of the image captured by the first macro camera.

[0054] In some possible implementations of the second aspect, determining, according to the first image feature matching result and the zoom ratio of the image captured by the first macro camera, an image offset value of the image captured by the first macro camera includes:

[0055] By formula Obtain an image offset value of an image captured by the first macro camera; wherein offset represents an offset value from the center of the field of view of the first macro camera to the center of the field of view of the second macro camera, which is a pre-calibrated offset value or an image feature matching result; offset pre Indicates the image offset value of the previous frame, offset cur Indicates the image offset value of the current frame image; zoom pre Indicates the zoom ratio of the previous frame, zoom cur Indicates the zoom ratio of the current frame image; zoom end Indicates the zoom ratio at which the image shift ends, which is less than or equal to the camera switching ratio; zoom start Indicates the zoom ratio at which the image shift starts, which is greater than or equal to the minimum zoom ratio of the first macro camera.

[0056] In some possible implementations of the second aspect, the navigation picture is a picture captured by the second macro camera;

[0057] The image captured by the first macro camera is displayed in the viewfinder, and a navigation window is superimposed on the preview image displayed in the viewfinder, including:

[0058] Digitally zoom and crop the image captured by the first macro camera to obtain a second cropped image; identify the position of the second cropped image in the target image by an identification pattern according to the initial calibrated image offset value or the last updated image offset value to obtain a second identification image; display the second cropped image in a viewfinder, and display the second identification image in a navigation window of the viewfinder; wherein the target image is a third cropped image obtained by performing image offset cropping on the image captured by the second macro camera according to the initial calibrated image offset value or the updated image offset value: or, a full field of view angle image captured by the second macro camera.

[0059] Exemplarily, the second macro camera is an ultra-wide-angle macro camera, and the first macro camera is a telephoto macro camera. The telephoto picture is used as the preview picture, and the full FOV picture or partial FOV picture captured by the ultra-wide-angle macro camera is used as the navigation picture, as long as the FOV of the partial FOV picture of the ultra-wide-angle macro camera is larger than that of the telephoto picture.

[0060] In some possible implementations of the second aspect, the method further includes:

[0061] Image feature matching is performed on the second cropped image and the target image to obtain a second image feature matching result; an image offset value of the target image is determined according to the second image feature matching result and the zoom ratio of the target image, and the offset value is updated according to the image offset value of the target image.

[0062] In some possible implementations of the second aspect, the navigation picture is a picture captured by the first macro camera;

[0063] The image captured by the first macro camera is displayed in the viewfinder, and a navigation window is superimposed on the preview image displayed in the viewfinder, including:

[0064] The image captured by the first macro camera is digitally zoomed and cropped to obtain a fourth cropped image; the position of the fourth cropped image in the image captured by the first macro camera is marked by the marking pattern to obtain a third marking image; the fourth cropped image is displayed in a viewfinder, and the third marking image is displayed in a navigation window of the viewfinder. Exemplarily, the first macro camera is a telephoto macro camera, in which case the full FOV image captured by the telephoto macro camera is used as the navigation image, and the partial FOV image of the telephoto macro camera is used as the preview image.

[0065] In some possible implementations of the second aspect, the method further includes:

[0066] When a photographing operation is detected, in response to the photographing operation, the image captured by the first macro camera and the image captured by the second macro camera are fused to obtain a fused image, and the fused image is output; the fused image includes a target area and a non-target area, and the image quality of the target area is higher than that of the non-target area; the target area is an area in the image captured by the first macro camera that corresponds to the image captured by the second macro camera.

[0067] In some possible implementations of the second aspect, the method further includes:

[0068] Detect the first shooting distance of the image captured by the target camera; if the first shooting distances of N consecutive frames of images are all less than the first threshold, switch from the target camera to the first macro camera, and display the image captured by the default macro camera in the viewfinder.

[0069] In some possible implementations of the second aspect, the method further includes: if the first shooting distances of N consecutive frames of images are all less than a first threshold, displaying a prompt message in the viewfinder, wherein the prompt message is used to indicate that the macro shooting mode has been entered.

[0070] In some possible implementations of the second aspect, the method also includes: detecting a second shooting distance of the image captured by the first macro camera; if the second shooting distances of M consecutive frames of images are greater than a second threshold, switching from the first macro camera to the default camera, and displaying the picture captured by the default camera in the viewfinder.

[0071] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method described in any one of the first aspect or the second aspect is implemented.

[0072] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method of any one of the first aspect or the second aspect mentioned above.

[0073] In a fifth aspect, an embodiment of the present application provides a chip system, the chip system includes a processor, the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement the method as described in any one of the first aspect or the second aspect. The chip system can be a single chip, or a chip module composed of multiple chips.

[0074] In a sixth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device executes the method described in the first aspect or the second aspect above.

[0075] It can be understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 A schematic block diagram of the hardware structure of the electronic device 100 provided in an embodiment of the present application;

[0077] Figure 2 A software structure block diagram of the electronic device 100 provided in an embodiment of the present application;

[0078] Figure 3A A schematic diagram of manually entering a macro shooting mode provided in an embodiment of the present application;

[0079] Figure 3B A schematic diagram of automatically entering a macro shooting mode provided in an embodiment of the present application;

[0080] Figure 3C A schematic diagram of a prompt to exit the macro shooting mode provided in an embodiment of the present application;

[0081] Figure 3D A schematic diagram of a zoom process in a macro shooting mode provided in an embodiment of the present application;

[0082] Figure 3E A schematic diagram of a shortcut control provided in an embodiment of the present application;

[0083] Figure 4A schematic diagram of the ultra-wide-angle preview process provided in an embodiment of the present application;

[0084] Figure 5 A schematic diagram of the image navigation process of a single macro camera provided in an embodiment of the present application;

[0085] Figure 6 A schematic diagram of image navigation for macro photography provided in an embodiment of the present application;

[0086] Figure 7 A schematic diagram of image navigation provided in an embodiment of the present application;

[0087] Figure 8 Another schematic diagram of image navigation provided by an embodiment of the present application;

[0088] Fig. 9 A schematic diagram of a long-focus low-medium magnification image navigation solution provided in an embodiment of the present application;

[0089] Fig.10 A schematic diagram of a process flow of a long-focus low-medium magnification image navigation solution provided in an embodiment of the present application;

[0090] Fig.11 A schematic diagram of an image navigation provided by an embodiment of the present application;

[0091] Fig.12 A schematic diagram of the process flow of the ultra-wide-angle and high-magnification navigation solution provided in an embodiment of the present application;

[0092] Fig.13 A schematic diagram of the long-focus high-magnification navigation solution flow chart provided in an embodiment of the present application;

[0093] Fig.14 A schematic diagram of a fused image provided in an embodiment of the present application;

[0094] Fig.15 A schematic diagram of a macro shooting process provided by an embodiment of the present application;

[0095] Fig.16A A schematic diagram of macro photography provided in an embodiment of the present application;

[0096] Fig. 16B Another schematic diagram of macro photography provided in an embodiment of the present application;

[0097] Fig. 16C Another schematic diagram of macro photography provided in an embodiment of the present application;

[0098] Fig.17A A schematic diagram of a macro video preview scene provided in an embodiment of the present application;

[0099] Fig. 17BA schematic diagram of macro video recording provided in an embodiment of the present application. DETAILED DESCRIPTION

[0100] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures and technologies are provided to facilitate a thorough understanding of the embodiments of the present application.

[0101] At present, when an electronic device realizes a macro shooting function based on a single fixed-focus macro camera, when the zoom ratio is not the zoom ratio of the macro camera, the macro shooting image is often obtained through digital zoom technology, that is, by cropping, enlarging, and interpolating the image captured by the macro camera at the zoom ratio, to obtain a macro shooting image at an equivalent focal length. However, a single macro camera covers a single focal length, and the cropped and enlarged image often has problems such as image distortion, poor image quality, and poor macro shooting effect.

[0102] Taking a mobile phone as an example, the mobile phone is integrated with an ultra-wide-angle macro camera. The ultra-wide-angle macro camera is a fixed-focus camera with a zoom ratio of 0.5x, and the zoom ratio range that can be covered is 0.5x to 15x.

[0103] In the process of macro shooting by the mobile phone based on the ultra-wide-angle macro camera, when the zoom ratio is 0.5x, the mobile phone obtains an optical image through the ultra-wide-angle macro camera, and no digital zoom is required at this time; when the zoom ratio is greater than 0.5x, after obtaining an optical image with a zoom ratio of 0.5x, the mobile phone performs digital zoom on the optical image to obtain a macro image at an equivalent focal length. In this case, at a high zoom ratio, the macro image obtained by digitally zooming the ultra-wide-angle optical image has poor image quality.

[0104] For another example, a mobile phone is integrated with a telephoto macro camera, which is a fixed-focus camera with a zoom ratio of 3x and can cover a zoom ratio range of 3x to 15x.

[0105] In the process of macro shooting based on the telephoto macro camera, when the zoom ratio is 3x, the mobile phone obtains the optical image through the telephoto macro camera, and no digital zoom is required at this time; when the zoom ratio is greater than 3x, after the mobile phone obtains the optical image with a zoom ratio of 3x, it performs digital zoom on the optical image to obtain a macro image at an equivalent focal length. Similarly, when the zoom ratio is high, the macro image obtained by digitally zooming the telephoto optical image has poor image quality. In addition, macro shooting with a focal length below 3x cannot be achieved based on a single telephoto macro camera, resulting in a single focal length covered by macro shooting.

[0106] In order to address the problem that a single fixed-focus macro camera has a single focal length and poor macro shooting effect, an embodiment of the present application provides a macro shooting solution based on at least two macro cameras, so that during the zooming process of macro shooting, macro shooting with a wider focal length can be achieved by switching relays of at least two macro cameras, thereby improving the macro shooting effect.

[0107] Specifically, in the macro shooting solution provided in the embodiment of the present application, the electronic device has at least two macro cameras, and the focal lengths of at least two macro cameras are inconsistent. In the macro shooting mode, after the electronic device obtains the current zoom ratio, it can determine which focal length of the macro camera the current zoom ratio falls on, and whether the camera switching ratio has been reached. If the current zoom ratio falls into the focal length of a certain macro camera, the electronic device uses the macro camera for macro shooting. If the current zoom ratio reaches the camera switching ratio, the electronic device switches from the current macro camera to another macro camera, and uses the other macro camera for macro shooting; if the current zoom ratio does not reach the camera switching ratio, the current macro camera continues to be used for macro shooting. In this way, through the switching relay of multiple macro cameras, macro shooting with a wider focal length can be achieved, and the macro shooting effect is better.

[0108] Among them, the macro camera refers to a camera with macro focus and macro photography capabilities, which may include but is not limited to: an ultra-wide-angle macro camera, a wide-angle macro camera, and a telephoto macro camera.

[0109] In the embodiment of the present application, the focal lengths of the at least two macro cameras are inconsistent, thereby achieving macro shooting at different focal lengths and different zoom ratios.

[0110] The at least two macro cameras may be fixed-focus lenses. In this case, during the zooming process of the macro shooting mode, the electronic device performs discontinuous optical zoom based on the at least two fixed-focus macro cameras. Furthermore, during the discontinuous zooming process, an image at a corresponding zoom ratio is obtained by digital zooming. Of course, the at least two macro cameras may be continuous zoom lenses, which is not limited here.

[0111] For example, the electronic device includes two fixed-focus cameras. One is an ultra-wide-angle macro camera with a zoom ratio of 0.5x, which can cover a zoom ratio of 0.5x to 3.5x; the other is a telephoto macro camera with a zoom ratio of 3.5x, which can cover a zoom ratio of 3.5x to 15x. The camera switching ratio is 3.5x.

[0112] At this time, the electronic device obtains the current zoom ratio during the zoom process of macro shooting. If the current zoom ratio is between 0.5x and 3.5x, the ultra-wide-angle macro camera is used for macro shooting. If the current zoom ratio is 3.5x, it is determined that the zoom ratio reaches the camera switching ratio, and the camera switching is performed to switch from the ultra-wide-angle macro camera to the telephoto macro camera. After switching, the telephoto macro camera is used for macro shooting. If the current zoom ratio is greater than 3.5x, the telephoto macro camera is used for macro shooting.

[0113] It can be understood that if the current zoom ratio is between 0.5x and 3.5x, the electronic device performs digital zoom and cropping on the image captured by the ultra-wide-angle macro camera to obtain an image at the corresponding zoom ratio; if the current zoom ratio is greater than 3.5x, the electronic device performs digital zoom and cropping on the image captured by the telephoto macro camera to obtain an image at the corresponding zoom ratio.

[0114] Compared with the existing macro shooting solution based on a single fixed-focus macro camera, the macro shooting solution based on at least two fixed-focus macro cameras provided in the embodiment of the present application has a wider focal length covered by macro shooting and a better macro shooting effect.

[0115] Specifically, the above-mentioned macro shooting solution based on a fixed-focus telephoto macro camera can only achieve 3x to 15x macro shooting, and cannot achieve macro shooting at a focal length below 3x. However, the embodiment of the present application is based on a 0.5x ultra-wide-angle macro camera and a 3.5x telephoto macro camera, which can achieve 0.5x to 15x macro shooting, and the macro shooting covers a wider focal length.

[0116] In addition, the above-mentioned macro shooting solution based on the fixed-focus ultra-wide-angle macro camera can achieve 0.5x-15x macro shooting, but when the zoom ratio is high (for example, 10x), the 0.5x wide-angle optical image is digitally zoomed and cropped to obtain a macro image, and the image quality is poor, and the macro shooting effect is poor. In the macro shooting solution based on two fixed-focus cameras in the embodiment of the present application, when the zoom ratio is greater than 3.5x, the 0.5x wide-angle optical image is no longer digitally zoomed and cropped, but the 3.5x telephoto optical image is digitally zoomed and cropped.

[0117] When the zoom ratio is greater than 3.5x, the quality of the image obtained by digitally zooming and cropping the 3.5x telephoto optical image is better than that obtained by digitally zooming and cropping the 0.5x wide-angle optical image, thereby improving the macro shooting effect.

[0118] It is understandable that the focal length refers to the segmentation of the focal length of the lens. The zoom ratio and focal length can be converted to each other. For the convenience of expression, the focal length is expressed by the zoom ratio above. For example, the focal length of the ultra-wide-angle macro camera is 0.5x~3.5x, which refers to the focal length range corresponding to the zoom ratio of 0.5x~3.5x, where the focal length corresponding to the zoom ratio of 0.5x is the focal length of the ultra-wide-angle macro camera, and the focal lengths corresponding to other zoom ratios are equivalent focal lengths obtained by digital zoom.

[0119] The electronic device of the embodiment of the present application may be, but is not limited to, a mobile phone, a tablet computer, and a laptop computer, etc. The embodiment of the present application limits the specific type and specific structure of the electronic device.

[0120] For example, see Figure 1 , is a schematic diagram of the hardware structure of the electronic device 100 provided in the embodiment of the present application. Figure 1 As shown, the electronic device 100 may include a processor 101, a memory 102, a focus motor 103 and a macro camera 104. The electronic device 100 may include 1 or N macro cameras 104, where N is a positive integer greater than 1. Of course, in addition to the macro camera, the electronic device 100 may also include other types of cameras. For example, the electronic device 100 is a mobile phone and includes an ultra-wide-angle macro camera, a telephoto macro camera, and a main camera.

[0121] Optionally, the electronic device 100 may further include a sensor module 105. The sensor module 105 may include but is not limited to a distance sensor 105A and the like.

[0122] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. For example, when the electronic device 100 is a mobile phone or a tablet computer, the electronic device 100 may also include a touch screen composed of a display screen and a touch sensor, etc.

[0123] The processor 101 may include one or more processing units. For example, the processor 101 may include an application processor (AP), a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, and / or a digital signal processor (DSP). Among them, different processing units may be independent devices or integrated into one or more processors. Among them, the controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0124] In some embodiments, the processor 101 may include one or more interfaces, such as a mobile industry processor interface (MIPI) and / or a general-purpose input / output (GPIO) interface.

[0125] The MIPI interface can be used to connect the processor 101 with peripheral devices such as a display screen and a camera 104. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 101 and the macro camera 104 communicate via the CSI interface to implement the macro shooting function of the electronic device 100. The processor 101 and the display screen communicate via the DSI interface to implement the display function of the electronic device 100.

[0126] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 101 with the macro camera 104, the display screen, the sensor module 105, etc. The GPIO interface can also be configured as a MIPI interface, etc.

[0127] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0128] The electronic device 100 can realize the shooting function through the ISP, the macro camera 104, the video codec, the GPU, the display screen and the application processor.

[0129] The ISP is used to process the data fed back by the macro camera 104. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the inner eye. The ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the macro camera 104.

[0130] The macro camera 104 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format.

[0131] The optical zoom in the embodiment of the present application can also be implemented based on multiple fixed-focus lenses. In this case, one or more camera modules in the macro camera 104 can include multiple fixed-focus lenses. The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals.

[0132] The memory 102 can be used to store computer executable program codes, and the executable program codes include instructions. The processor 101 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory. The memory 102 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0133] The distance sensor 105A is used to measure the distance between the lens and the subject. The electronic device 100 can measure the distance through an infrared distance sensor, a laser distance sensor or a time of flight (TOF) camera. In some embodiments, in the shooting scene, the electronic device 100 can use the distance sensor 105A to measure the distance to achieve fast focusing. In the embodiment of the present application, in the macro shooting scene, the electronic device can also use the distance sensor 105A to measure the distance to achieve the function of automatically entering the macro shooting mode and automatically exiting the macro shooting mode. The focus motor 103 is used to control the focus module to move within the focus range to achieve focusing. Normally, the electronic device 100 zooms first and then focuses. For example, in the macro shooting scene, after the electronic device 100 enters the macro shooting mode, it zooms according to the zoom ratio selected by the user, and then focuses after zooming.

[0134] After introducing the hardware architecture of the electronic device 100 , the software architecture of the electronic device 100 is exemplarily introduced below.

[0135] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0136] Figure 2 1 is a software structure block diagram of the electronic device 100 according to an embodiment of the present application.

[0137] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0138] The application layer can include a series of application packages. Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0139] The application framework layer provides application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions. Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0140] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make the data accessible to applications. This data can include video, images, audio, etc.

[0141] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0142] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, etc. The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify the completion of downloads, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is prompted in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0143] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system. The core library contains two parts: one is the function that the Java language needs to call, and the other is the Android core library.

[0144] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0145] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0146] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0147] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0148] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing. The 2D graphics engine is a drawing engine for 2D drawing. The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, sensor driver, and focus motor driver.

[0149] The following is an illustrative description of the workflow of the software and hardware of the electronic device 100 in conjunction with a shooting scenario.

[0150] For example, when the touch sensor receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into an original input event (including touch coordinates, timestamp of the touch operation, and other information). The original input event is stored in the kernel layer. The application framework layer obtains the original input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation, and the control corresponding to the single-click operation is a control of a camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer, and captures a static image or video through the macro camera 104.

[0151] After introducing the hardware architecture and software architecture that may be involved in the electronic device 100, the technical solution provided in the embodiment of the present application is described in detail below using the electronic device 100 as an example.

[0152] In the macro shooting solution based on at least two macro cameras provided in the embodiment of the present application, the electronic device 100 can first enter the macro shooting mode, and then obtain the current zoom ratio during the zooming process in the macro shooting mode, and determine the macro camera to be used and whether to switch the macro camera according to the current zoom ratio.

[0153] In some embodiments of the present application, the electronic device 100 may enter the macro shooting mode in response to a user instruction, where the user instruction is used to instruct the electronic device 100 to enter the macro shooting mode. That is, the user may manually enter the macro shooting mode.

[0154] For example, see Figure 3A 1 is a schematic diagram of manually entering the macro shooting mode, the electronic device 100 is specifically a mobile phone 31, and the mobile phone 31 includes a main camera, an ultra-wide-angle macro camera and a telephoto macro camera. Figure 3AAs shown, the main interface 311 of the mobile phone 31 includes applications such as smart life, settings, calendar, gallery, and camera 312. When the mobile phone 31 detects a user's click operation on the camera 312, the mobile phone 31 responds to the click operation and displays a preview interface 313. At this time, the preview interface 313 displays the picture captured by the main camera, which includes the shooting subject 319. The preview interface 313 includes a zoom bar 314 and a zoom control 315, and also includes controls for various shooting modes, namely: aperture mode control, night scene mode control, video mode control 316, photo mode control 317, and macro mode control 318. The mobile phone 31 is currently in photo mode.

[0155] The current zoom ratio is 1x as shown on the zoom bar 314. The user can change the zoom ratio by dragging the zoom control 315 on the zoom bar 314. Dragging the zoom control 315 to the left can increase the zoom ratio, and dragging the zoom control 315 to the right can decrease the zoom ratio.

[0156] The user can enter the corresponding shooting mode by clicking the control of the shooting mode. For example, if the user needs to enter the video recording mode, the user can click the video recording mode control 316 to enter the video recording mode.

[0157] In order to enter the macro shooting mode, the user can manually click the macro mode control 318. After the mobile phone 31 detects the click operation on the macro mode control 318, it enters the macro shooting mode in response to the click operation and displays the preview interface 320. At this time, the user instruction for instructing the electronic device 100 to enter the macro shooting mode is a click instruction on the macro mode control 318.

[0158] Specifically, after detecting a click operation on the macro mode control 318, the mobile phone 31 switches from the current photo shooting mode to the macro shooting mode, starts the ultra-wide-angle macro camera, obtains image data collected by the ultra-wide-angle macro camera, and displays the picture captured by the ultra-wide-angle macro camera on the preview interface 320. That is, after detecting a click operation on the macro mode control 318, the mobile phone 31 switches from the main camera to the ultra-wide-angle macro camera. At this time, when the macro shooting mode is turned on, the default is to switch from the main camera to the ultra-wide-angle macro camera. In other embodiments, it is also possible to switch from the main camera to the telephoto macro camera by default, that is, after entering the macro shooting mode, the picture captured by the telephoto macro camera is displayed on the preview interface 320.

[0159] When switching from the main camera to the ultra-wide-angle macro camera, the FOV of the picture displayed on the preview interface 320 is kept unchanged. Of course, the FOV of the picture displayed on the preview interface 320 does not need to be kept unchanged when the camera is switched, and this is not limited here.

[0160] In some embodiments of the present application, the electronic device 100 may also exit the macro shooting mode in response to a user instruction for instructing to exit the macro shooting mode, that is, the user may manually exit the macro shooting mode.

[0161] by Figure 3A For example, if the user needs to exit the macro shooting mode, the user can click on the control of other shooting modes to manually exit the macro shooting mode. For example, after entering the macro shooting mode, the mobile phone 31 detects that the user clicks on the video mode control 316, and in response to the click operation on the video mode control 316, exits the macro shooting mode and enters the video mode. Of course, the user can manually exit the macro shooting mode by manually exiting the camera program. Alternatively, the mobile phone 31 may also provide a control for exiting the macro shooting mode. When a click operation of the user on the control is detected, the macro shooting mode is exited in response to the click operation. The method of manually exiting the macro shooting mode is not limited here.

[0162] In addition to being able to manually enter and exit the macro shooting mode, the electronic device 100 can also automatically enter and exit the macro shooting mode in some other embodiments of the present application. In this case, when the electronic device 100 determines that the conditions for automatically entering the macro shooting mode are met, it automatically enters the macro shooting mode; when the conditions for automatically exiting the macro shooting mode are met, it automatically exits the macro shooting mode.

[0163] For example, the condition for automatically entering the macro shooting mode is: detecting that the shooting distance of N consecutive frames of images is less than the first threshold. The condition for automatically exiting the macro shooting mode is: detecting that the shooting distance of M consecutive frames of images is greater than the second threshold. For example, N is 30 frames, the first threshold is 10 cm; M is 60 frames, and the second threshold is 50 cm.

[0164] In a specific application, the electronic device 100 can measure the distance through the distance sensor 105A or TOF to detect the above shooting distance.

[0165] It is worth pointing out that the conditions for automatically entering the macro shooting mode and the conditions for automatically exiting the macro shooting mode may also be other conditions, which are not limited here.

[0166] For example, in order to reduce the amount of calculation, the electronic device 100 may detect the shooting distance of the image every preset number of frames (for example, 2 frames), and when the shooting distances detected in N frames are all less than the first threshold, it is determined that the conditions for automatically entering the macro shooting mode are met, and the macro shooting mode is automatically entered. Similarly, the electronic device 100 may also detect the shooting distance of the image every preset number of frames (for example, 3 frames), and when the shooting distances detected in M ​​frames are all greater than the second threshold, it is determined that the conditions for automatically exiting the macro shooting mode are met, and the macro shooting mode is automatically exited.

[0167] For another example, the condition for automatically entering the macro shooting mode is: the shooting distance detected in the first time period is less than the third threshold. The condition for automatically exiting the macro shooting mode is: the shooting distance detected in the second time period is greater than the fourth threshold. The first time period, the second time period, the third threshold and the fourth threshold can all be set according to actual needs.

[0168] For example, see Figure 3B 1 is a schematic diagram of automatically entering the macro shooting mode, when the mobile phone 31 detects a user's click operation on the camera 312, in response to the click operation, a preview interface 321 is displayed. The preview interface 321 displays the picture captured by the main camera, and the picture includes the shooting subject 319. The mobile phone 31 is currently in the shooting mode, that is, the preview interface 321 is a shooting preview interface.

[0169] After entering the photo preview state, the mobile phone 31 detects the distance between the mobile phone 31 and the shooting subject 319, that is, the shooting distance, and determines whether the shooting distance is less than the first threshold. If it is detected that the shooting distances of the consecutive N frames of images captured by the main camera are less than the first threshold, the mobile phone 31 determines that the conditions for automatically entering the macro shooting mode are met, and automatically enters the macro shooting mode. After the mobile phone 31 enters the macro shooting mode, it switches from the main camera to the ultra-wide-angle macro camera, and displays the picture captured by the ultra-wide-angle macro camera on the preview interface 322. In addition, in order to prompt the user that the macro shooting mode has been entered, the mobile phone 31 also displays a prompt message 323 on the preview interface 322. The prompt message 323 is specifically "Macro mode is turned on", which is used to prompt the user that the macro shooting mode has been automatically entered.

[0170] It is understandable that in addition to prompting the user that the macro shooting mode has been automatically entered through the prompt information 323, the user can also be prompted to enter the macro shooting mode through other methods. For example, the mobile phone 31 plays a prompt voice "Macro mode is turned on" through the speaker. In addition, in addition to prompting the user that the macro shooting mode has been automatically entered through the prompt operation, the user can also be prompted how to operate in the macro shooting mode through the prompt operation. For example, the mobile phone 31 can also display the macro shooting mode operation prompt information on the preview interface 322. Of course, the mobile phone 31 may not perform the prompt operation.

[0171] In addition, in order to reduce misoperation, after determining that the conditions for the automatic macro shooting mode are met, the electronic device 100 may also first perform a prompt operation to prompt the user whether to enter the macro shooting mode; after receiving the user's instruction to confirm entering the macro shooting mode, the electronic device 100 enters the macro shooting mode. Figure 3B In the scenario, after the mobile phone 31 determines that the conditions for automatically entering the macro shooting mode are met, a prompt window pops up on the preview interface 321, and the prompt window includes two option buttons of "Yes" and "No", and a prompt message "It has been detected that the conditions are met, do you want to enter the macro shooting mode?"; when the mobile phone 31 detects a click operation on the "Yes" option button, it enters the macro shooting mode in response to the click operation.

[0172] It should be noted that Figure 3A and Figure 3B Please refer to the above for the same parts, which will not be repeated here.

[0173] It should also be noted that Figure 3B The macro shooting mode is not set as a shooting mode in parallel with the photo and video modes, but the macro shooting mode is set in the photo mode. Figure 3A A macro shooting mode is set separately in the system, and the macro shooting mode is parallel to the photo shooting mode, the video recording mode and the like. In the specific implementation, Figure 3A and Figure 3B Both methods are acceptable, that is, the macro shooting function can be set as a shooting mode alone, or the macro shooting function can be set in other shooting modes, which is not limited here.

[0174] exist Figure 3B In the scenario, after the mobile phone 31 automatically enters the macro shooting mode, it continues to detect the shooting distance between it and the shooting subject 319 through the distance sensor or TOF, and determines whether the shooting distance is greater than the second threshold. If it is detected that the shooting distances of the continuous M frames of images collected by the macro camera are all greater than the second threshold, the mobile phone 31 determines that the conditions for automatically exiting the macro shooting mode are met, and automatically exits the macro shooting mode.

[0175] In order to improve the user experience, the electronic device 100 may display a corresponding prompt message on the preview interface when automatically exiting the macro shooting mode, to prompt the user that the macro shooting mode has been exited. Figure 3C A schematic diagram showing a prompt to exit the macro shooting mode, based on Figure 3B In the scenario, the mobile phone 31 is in the macro shooting mode. After determining that the conditions for automatically exiting the macro shooting mode are met, the macro shooting mode is automatically exited, and a prompt message 325 is displayed on the preview interface 324 to prompt the user that the macro shooting mode is currently turned off. The prompt message 325 is specifically "Macro mode is turned off". Among them, Figure 3C and Figure 3A , Figure 3B The same parts as above can be found and will not be repeated here.

[0176] In an embodiment of the present application, when the electronic device 100 exits the macro shooting mode, it can determine the focal length of the camera used after exiting the macro shooting mode according to the current zoom ratio. For example, the electronic device 100 includes a main camera, a telephoto macro camera, and an ultra-wide-angle macro camera. When the electronic device 100 exits the macro shooting mode, if the current zoom ratio is in the telephoto focal length, it returns to the normal telephoto preview and focuses, that is, the focused telephoto image is displayed on the preview interface; if the current zoom ratio is in the wide-angle focal length above 1x, it returns to the main camera preview, that is, the picture captured by the main camera is displayed on the preview interface; if the current zoom ratio is in the wide-angle focal length below 1x, it returns to the wide-angle focal length, that is, the picture captured by the ultra-wide-angle camera is displayed on the preview interface. It can be understood that the telephoto macro camera refers to a telephoto camera with macro photography capabilities, and the ultra-wide-angle macro camera refers to an ultra-wide-angle camera with macro photography capabilities.

[0177] Of course, the electronic device 100 can also return to the default focal length and default camera when exiting the macro shooting mode. For example, it is preset that when exiting the macro shooting mode, the default focal length is the focal length of the main camera, and the default camera is the main camera, that is, no matter which focal length the current zoom ratio is at when exiting the macro shooting mode, when exiting the macro shooting mode, it will automatically switch to the main camera and the focal length of the main camera.

[0178] As can be seen from the above, the electronic device 100 can enter and exit the macro shooting mode manually, or automatically. In comparison, the electronic device 100 determines whether to automatically enter and exit the macro shooting mode according to the shooting distance, which is more convenient to operate and provides a better macro photography experience for the user. Furthermore, when entering and / or exiting the macro shooting mode, a prompt message is used to remind the user that the macro shooting mode has been entered and exited, and the user's macro photography experience is also higher.

[0179] It can be understood that in some embodiments of the present application, the electronic device 100 can be in the macro shooting mode by default, that is, the electronic device 100 does not need to enter the macro shooting mode from another shooting mode.

[0180] In an embodiment of the present application, after entering the macro shooting mode, the electronic device 100 obtains the current zoom ratio, and determines whether the current zoom ratio reaches the camera switching ratio based on the focal length of the macro camera; if the current zoom ratio reaches the camera switching ratio, the macro camera is switched to achieve macro photography with a wider focal length.

[0181] During the zooming process in the macro shooting mode, the zoom ratio may be obtained through human-computer interaction, that is, the zoom ratio is input by the user; or it may be determined autonomously by the electronic device 100 .

[0182] For example, see Figure 3D A schematic diagram of the zoom process in macro shooting mode is shown, based on Figure 3B In the scenario, after determining that the conditions for automatically entering the macro shooting mode are met, the mobile phone 31 automatically enters the macro shooting mode, displays the picture captured by the ultra-wide-angle macro camera on the preview interface 322, and also displays a prompt message 323 on the preview interface 322 to prompt the user that the macro mode is turned on.

[0183] After entering the macro shooting mode, the user can change the zoom ratio by dragging the zoom control 315 on the zoom bar 314. During the zooming process, if the current zoom ratio reaches the camera ratio switching ratio, the photo preview screen of the mobile phone 31 switches from the ultra-wide-angle macro camera to the telephoto macro camera, or from the telephoto macro camera to the ultra-wide-angle macro camera.

[0184] Among them, for the convenience of description, unless otherwise mentioned, the ultra-wide-angle macro camera and telephoto macro camera mentioned below are fixed-focus cameras, and the zoom ratio range covered by the ultra-wide-angle macro camera is 0.5x~3.5x, which includes 0.5x but excludes 3.5x; the zoom ratio range covered by the telephoto macro camera is 3.5x~15x, which includes 3.5x. The camera switching ratio is 3.5x. Of course, in specific applications, the zoom ratio range supported by the ultra-wide-angle macro camera and the telephoto macro camera is not limited to the range mentioned above.

[0185] like Figure 3DAs shown, the current zoom ratio displayed on the preview interface 322 is 1x. The user drags the zoom control 315 to the right to increase the zoom ratio to 1.5x. In response to the user's dragging operation on the zoom control 315, the mobile phone 31 obtains that the current zoom ratio is 1.5x, and determines that the current zoom ratio 1.5x does not reach the camera switching ratio 3.5x. In addition, the current zoom ratio 1.5x falls within the focal length of the ultra-wide-angle macro camera, so in the process of increasing the zoom ratio from 1x to 1.5x, the mobile phone 31 does not need to switch the macro camera, and still uses the ultra-wide-angle macro camera for macro shooting. Specifically, the mobile phone 31 performs digital zoom cropping on the image captured by the ultra-wide-angle macro camera to obtain a display image, and displays the display image on the preview interface 326 to display the image captured by the ultra-wide-angle macro camera on the preview screen 326.

[0186] After the user increases the zoom ratio to 1.5x, he continues to drag the zoom control 315 to the right to increase the zoom ratio from 1.5x to 3.5x. In response to the user's dragging operation on the zoom control 315, the mobile phone 31 obtains that the current zoom ratio is 3.5x. The mobile phone 31 determines that the current zoom ratio of 3.5x reaches the camera switching ratio, and it is necessary to switch the macro camera, so it switches from the ultra-wide-angle macro camera to the telephoto macro camera, and uses the telephoto macro camera for macro shooting. Specifically, the mobile phone 31 displays the picture captured by the telephoto macro camera on the preview interface 327 according to the current zoom ratio. In this way, when the zoom ratio reaches the camera switching ratio, the picture displayed on the preview interface of the mobile phone 31 is switched from the ultra-wide-angle macro camera to the telephoto macro camera.

[0187] After the user increases the zoom ratio to 3.5x, he continues to drag the zoom control 315 to the right to increase the zoom ratio from 3.5x to 4.0x. In response to the user's dragging operation on the zoom control 315, the mobile phone 31 obtains that the current zoom ratio is 4.0x. The mobile phone 31 determines that the current zoom ratio of 4.0x does not reach the camera switching ratio, and there is no need to switch the macro camera, so the telephoto macro camera continues to be used for macro shooting. Specifically, the mobile phone 31 performs digital zoom cropping on the image captured by the telephoto macro camera according to the current zoom ratio, obtains a display image, and displays the display image on the preview interface 328 to display the picture captured by the telephoto macro camera on the preview interface 328. The current zoom ratio is also displayed on the preview interface 328 and the preview interface 327.

[0188] By comparing preview interface 326 , preview interface 327 , preview interface 328 and preview interface 327 , it can be seen that the magnification of the photographed subject 319 is getting larger and larger.

[0189] exist Figure 3DIn the scene, in the macro shooting mode, the mobile phone 31 realizes mobile phone macro photography from the ultra-wide-angle focal length to the telephoto focal length through the switching relay between the ultra-wide-angle macro camera and the telephoto macro camera.

[0190] In the embodiment of the present application, in addition to switching the camera by dragging the zoom control 315, the camera can also be switched through the shortcut control.

[0191] For example, see Figure 3E A schematic diagram of the shortcut control shown is based on Figure 3B In the scenario, after the mobile phone 31 automatically enters the macro shooting mode, a prompt message 323 is displayed on the preview interface 329, and the ultra-wide-angle macro camera is switched from the main camera to the ultra-wide-angle macro camera by default, and the ultra-wide-angle macro camera is used for macro shooting, that is, the picture captured by the ultra-wide-angle macro camera is displayed on the preview interface 329.

[0192] The preview interface 329 also includes an ultra-wide-angle macro shortcut control 330 and a telephoto macro shortcut control 331. If the user needs to enter the ultra-wide-angle macro mode, the user can click the ultra-wide-angle macro shortcut control 330; if the user needs to enter the telephoto macro mode, the user can click the telephoto macro shortcut control 331. In the ultra-wide-angle macro mode, the ultra-wide-angle macro camera is used for macro shooting; in the telephoto macro mode, the telephoto macro camera is used for macro shooting.

[0193] like Figure 3E As shown, after the mobile phone 31 determines that the conditions for automatically entering the macro shooting mode are met, it automatically enters the macro shooting mode, and when automatically entering the macro shooting mode, it enters the ultra-wide-angle macro mode by default. At this time, the ultra-wide-angle macro shortcut control 330 in the preview interface 329 is in the on state, and the telephoto macro shortcut control 331 is in the off state.

[0194] It should be noted that in order to distinguish the open and closed states of the shortcut controls, Figure 3E In the example, a shadow is added to the shortcut control to indicate the closed state of the shortcut control, and no shadow is added to the shortcut control to indicate the open state of the shortcut control. In specific applications, the open state and the closed state can also be distinguished by other methods, for example, different colors can be used to indicate the open state and the closed state respectively.

[0195] In addition, the specific form of the shortcut control is not limited to Figure 3D Also, the display location of the shortcut control is not limited to Figure 3D As shown, for example, the telephoto macro shortcut control may be displayed on the zoom bar 314 at a position where the zoom ratio is 3.5x, which is not limited here.

[0196] After the mobile phone 31 enters the ultra-wide-angle macro mode by default, the user can switch from the ultra-wide-angle macro mode to the telephoto macro mode by clicking the telephoto macro shortcut control 331. Figure 3E As shown, after detecting a click operation on the telephoto macro shortcut control 331, the mobile phone 31 increases the zoom ratio from 1x to 3.5x in response to the click operation, switches from the ultra-wide-angle macro camera to the telephoto macro camera, and uses the telephoto macro camera for macro shooting, that is, displays the picture captured by the telephoto macro camera on the preview interface 332. The ultra-wide-angle macro shortcut control 330 on the preview interface 332 is in a closed state, and the telephoto macro shortcut control 331 is in an open state.

[0197] After the mobile phone 31 switches to the telephoto macro mode, if the user needs to switch to the ultra-wide-angle macro mode, the user can also click the ultra-wide-angle macro shortcut control 330 to quickly switch from the telephoto macro mode to the ultra-wide-angle macro mode.

[0198] exist Figure 3E In the scenario, the mobile phone 31 can not only automatically exit the macro shooting mode according to the shooting distance, but also exit the macro shooting mode through the ultra-wide-angle macro shortcut control 330. For example, the mobile phone 31 is in the ultra-wide-angle macro mode, that is, the ultra-wide-angle macro shortcut control 330 is turned on and the telephoto macro shortcut control 331 is turned off. The user can exit the macro shooting mode by clicking the ultra-wide-angle macro shortcut control 330.

[0199] It is understandable that in Figure 3E In the scene, after the mobile phone 31 enters the macro shooting mode, the user can also change the zoom ratio by dragging the zoom control 315. When the user increases the zoom ratio to 3.5x by dragging the zoom control 315, the mobile phone 31 determines that the current zoom ratio of 3.5x reaches the camera switching ratio, and switches from the ultra-wide-angle macro camera to the telephoto macro camera, uses the telephoto macro camera for macro shooting, and displays the telephoto macro shortcut control 331 as on, and displays the ultra-wide-angle macro shortcut control 330 as off.

[0200] It is worth pointing out that compared with the method of switching the macro camera by dragging the zoom control on the zoom bar, the method of switching the macro camera through the shortcut control is more convenient to operate and the user's macro photography experience is better.

[0201] It should be noted that when the electronic device 100 includes at least three macro cameras, the zoom process in the macro shooting mode is similar to the zoom process of the two macro cameras in the above example, that is, when the zoom magnification reaches the camera switching magnification, the macro camera switching relay is performed; when the zoom magnification does not reach the camera switching magnification, the macro camera with the focal length corresponding to the zoom magnification is used for macro shooting.

[0202] As can be seen from the above, the macro shooting solution based on at least two macro cameras provided in the embodiment of the present application can achieve macro photography with a wider focal length and a better macro photography effect through the switching relay of at least two macro cameras during the zooming process of the macro shooting mode.

[0203] Based on the above embodiments, it can be seen that in the macro shooting solution based on at least two macro cameras, the electronic device 100 needs to perform macro camera switching relay. However, the field of view centers of the macro cameras are inconsistent, resulting in an offset between the image centers of the macro cameras, which in turn results in an unsmooth and unnatural picture transition when the macro cameras are switched during the zoom process, reducing the user's macro photography experience.

[0204] In order to improve the image transition effect when the macro camera is switched, so that the image transition is smoother and more natural when the macro camera is switched, so as to improve the user's macro shooting experience, the embodiment of the present application provides an image center alignment scheme, which performs image offset cropping on the image captured by the first target macro camera according to the image offset value during the zoom process of the macro shooting mode, obtains a cropped image, and uses the cropped image as the image to be displayed; and, based on the cropped image, calculates the image offset value between the image center of the first target macro camera and the image center of the second target macro camera, and updates the offset value according to the image offset value. In this way, image offset cropping is performed through the image offset value, so that the center of the cropped image and the center of the image captured by the second target macro camera gradually approach each other during the zoom process, and when switching from the first target macro camera to the second target macro camera, the field of view center of the displayed image remains unchanged, and the image transition is smoother and more natural.

[0205] Among them, for the convenience of description, the two macro cameras involved in the macro camera switching process are defined as the first target macro camera and the second target macro camera. The first target macro camera refers to the macro camera with the focal length corresponding to the current zoom ratio, that is, the macro camera currently in use. The second target macro camera refers to the macro camera that needs to be switched, that is, it needs to be switched from the first target macro camera to the second target macro camera. Usually, the focal length of the second target macro camera is adjacent to the focal length of the first target macro camera. The focal length of the first target macro camera is smaller than the focal length of the second target macro camera.

[0206] After entering the macro shooting mode, the electronic device 100 turns on the first target macro camera and the second target macro camera, and acquires images captured by the first target macro camera and the second target macro camera.

[0207] In the initial stage, the image offset value is the initial image offset value, and the initial image offset value is pre-calibrated. The electronic device 100 performs image offset cropping on the image captured by the first target macro camera according to the initial image offset value to obtain a cropped image; then the cropped image is used as the display image, that is, the cropped image is displayed on the preview interface, and image feature matching is performed based on the cropped image and the image captured by the second target macro camera to obtain an image feature matching result; then, based on the image feature matching result, the image offset value of the current frame image is determined, and the image offset value is updated based on the image offset value of the current frame image. Among them, the current frame image is the image captured by the first target macro camera. Offset value update refers to using the calculated image offset value as the new image offset value.

[0208] After the image offset value is updated, the image offset value is the image offset value last updated. When the zoom ratio changes, the electronic device 100 performs image offset cropping on the image captured by the first target macro camera according to the updated image offset value to obtain a cropped image; the cropped image is used as the display image, that is, the cropped image is displayed on the preview interface, and image feature matching is performed based on the cropped image and the image captured by the second target macro camera, and then the image offset value of the current frame image is determined according to the image feature matching result; then the image offset value is updated according to the image offset value of the current frame image. This cycle is repeated until the zoom ratio reaches the camera switching ratio, and the display image is replaced from the cropped image to the image captured by the second target macro camera.

[0209] For example, during the zoom process of macro photography, for the Nth frame image of the first target macro camera, image offset cropping is performed on the Nth frame image according to the image offset value updated last time to obtain the display image corresponding to the Nth frame image; and, based on the display image corresponding to the Nth frame image and the image captured by the second target macro camera, image feature matching is performed to calculate the image offset value corresponding to the Nth frame image, and the image offset value is updated to the image offset value corresponding to the Nth frame image.

[0210] If image offset cropping is performed on each frame, for the Nth frame, the image offset value last updated is the image offset value corresponding to the N-1th frame. At the N+1th frame, image offset cropping is performed on the N+1th frame according to the image offset value corresponding to the Nth frame to obtain the display image corresponding to the N+1th frame; and image feature matching is performed based on the display image corresponding to the N+1th frame and the image captured by the second target macro camera to calculate the image offset value corresponding to the N+1th frame, and the image offset value is updated to the image offset value corresponding to the N+1th frame. Of course, image offset cropping does not need to be performed on each frame.

[0211] Among them, the image feature matching method can be but is not limited to scale-invariant feature transform (SIFT), fast nearest neighbor search package (FLANN), or ORB (Oriented Fast and Rotated Brief) and other methods, and the image feature matching method is not limited here.

[0212] In the process of image feature matching, in order to reduce the amount of calculation, the cropped image and the image captured by the second macro camera can be downsampled, and then the downsampled image can be used for image feature matching.

[0213] Exemplarily, the image offset value of the current frame image is calculated by the following formula 1.

[0214]

[0215] Wherein, offset represents the offset value from the field of view center of the first target macro camera to the field of view center of the second target macro camera, and its initial value is obtained from pre-calibrated data, and then it will be updated in real time to the value calculated by the image feature matching algorithm. pre Indicates the image offset value of the previous frame, offset cur Indicates the image offset value of the current frame image. zoompr.e indicates the zoom ratio of the previous frame image. cur Indicates the zoom ratio of the current frame image. end The zoom factor indicating the end of the image shift needs to be less than or equal to the switching factor of the first target macro camera and the second target macro camera.

[0216] When zoom cur =zoom pre =zoom start When pre =0, offset is the pre-calibrated image offset value. zoom start Indicates the zoom factor at which the image shift starts. start and zoom end These are all pre-set values. zoom start Greater than or equal to the minimum zoom ratio of the first target macro camera, and less than zoom end When the zoom ratio is greater than zoom nd When the image offset value is o ffset.

[0217] From the above formula, we can see that when the zoom ratio remains unchanged, that is, zoom cur =zoom pre When cur =offset pre , that is, offset cur This can avoid the error in the offset obtained by image feature matching, which will cause the image to be offset.

[0218] When the zoom ratio increases, the offset increases linearly. cur ≥zoom end When cur =offset, at this time, when switching from the first target macro camera to the second target macro camera, the field of view center of the displayed image can be kept unchanged, and the picture transition when the camera is switched is smoother and more natural.

[0219] In order to better introduce the image center alignment solution, the following Figure 3D and Figure 4 The ultra-wide-angle preview process diagram shown is introduced and explained.

[0220] like Figure 3D and Figure 4 As shown, the electronic device 100 is specifically a mobile phone 31, and the mobile phone 31 includes a main camera, a telephoto macro camera and an ultra-wide-angle macro camera. At this time, the first target macro camera is an ultra-wide-angle macro camera with a focal length of 0.5x to 3.5x; the second target macro camera is a telephoto macro camera with a focal length of 3.5x to 15x. The camera switching magnification is 3.5x. zoom start 1x, zoom end is 3.5x.

[0221] When the mobile phone 31 is in the photo preview state, if it detects that the shooting distances of N consecutive frames of images are less than the first threshold, it determines that the conditions for automatically entering the macro shooting mode are met, and automatically enters the macro shooting mode, turns on the telephoto macro camera and the ultra-wide-angle macro camera, captures telephoto images through the telephoto macro camera, and captures wide-angle images through the ultra-wide-angle macro camera.

[0222] When entering macro shooting mode, the zoom ratio is 1x, that is, zoom cur =zoom pre =zoom start =1, offset pre =0, offsef is the pre-calibrated image offset value.

[0223] When the zoom ratio is 1x, the image offset value is the initial offset value. After the telephoto image is processed by the ISP, a downsampled telephoto reduction image is output. When the wide-angle image passes through the ISP module, the ISP module performs image offset cropping according to the zoom ratio and the image offset value to obtain a cropped image. At this time, according to the initial offset value, the wide-angle image is subjected to image offset cropping to obtain a cropped image, and the cropped image is used as a wide-angle preview image with a zoom ratio of 1x, that is, the cropped image is displayed on the preview interface 322. In addition, the mobile phone 31 also downsamples the cropped image to obtain a wide-angle reduction image, and performs image feature matching on the telephoto reduction image and the wide-angle reduction image, calculates the image offset value between the center of the telephoto image and the center of the wide-angle image based on the image feature matching, and then updates the offset value based on the image offset value.

[0224] The user drags the zoom control 315 to change the zoom from 1x to 3.5x. The mobile phone 31 performs the following operations for each frame of the image: Figure 4 process.

[0225] When the zoom ratio of the current frame image is 1.5x, the mobile phone 31 downsamples the telephoto image through the ISP to obtain a telephoto zoomed image; performs image offset cropping on the wide-angle image according to the updated image offset value to obtain a cropped image, and displays the cropped image on the preview interface 326; the cropped image is also downsampled to obtain a wide-angle zoomed image; the wide-angle zoomed image and the telephoto zoomed image are then matched with each other for image features, and the image offset value of the current frame image is calculated according to the image feature matching result and the current zoom ratio through the above formula 1, and the offset value is updated according to the image offset value of the current frame image.

[0226] When the zoom ratio of the current frame image is 3.5x, the mobile phone 31 determines that the current zoom ratio reaches the camera switching ratio, and uses the telephoto image as the display image, that is, displays the telephoto image on the preview interface 327 to switch the preview interface from the ultra-wide-angle macro camera to the telephoto macro camera.

[0227] In the above scenario, during the zoom process of the macro shooting mode, the wide-angle image is offset and cropped through the image offset value during ultra-wide-angle preview, so as to make a registration offset to the telephoto FOV through FOV offset cropping, so that the telephoto FOV is as close as possible to the center of the FOV after ultra-wide-angle cropping. As a result, when switching from the wide-angle focal length to the telephoto focal length, the image transition is smoother and more natural, and the user's macro shooting experience is better.

[0228] It should be noted that in some other embodiments of the present application, the electronic device 100 does not need to calculate the image offset value of the current frame image during the zoom process in the macro shooting mode, and does not need to perform image offset cropping according to the image offset value. In this case, the electronic device 100 can directly use the full FOV image or partial FOV image of the current macro camera as the image to be displayed.

[0229] When the electronic device 100 is in macro shooting mode, the shooting distance between the camera and the subject is relatively close. When the zoom ratio is large or at a high focal length, the preview FOV is small, so that the user's observation field of view is easily blocked by the screen of the electronic device 100, and the subject is easily lost during shaking, which makes the interaction and composition of macro shooting difficult, affecting the user experience.

[0230] In order to solve the problem of interaction and composition difficulties in macro shooting, the embodiment of the present application overlays the image navigation screen on the preview screen of the macro shooting mode, and the FOV of the image navigation screen is larger than the FOV of the preview screen. In this way, through the image navigation screen with a larger FOV, it is convenient for users to find macro shooting subjects and adjust the composition, which makes user interaction and composition more convenient and provides a better macro shooting experience.

[0231] In some embodiments of the present application, the image navigation screen and the preview screen may be images captured by the same macro camera. In this case, the electronic device 100 may include at least one macro camera. The electronic device 100 may use the full FOV screen captured by the macro camera as the image navigation screen, and the partial FOV screen cropped from the full FOV screen as the preview screen, thereby making the FOV of the image navigation screen larger than the FOV of the preview screen. Of course, the electronic device 100 may also use the partial FOV screen captured by the macro camera as the image navigation screen, as long as the field of view of the partial FOV screen corresponding to the navigation screen is larger than the field of view of the partial FOV screen corresponding to the preview screen.

[0232] For example, see Figure 5 The image navigation process diagram of a single macro camera is shown. After entering the macro shooting mode, the electronic device 100 turns on the macro camera and obtains the image captured by the macro camera; the full FOV image captured by the macro camera is cropped by the ISP to crop a partial FOV image from the full FOV image, and the cropped partial FOV image is used as a preview screen; the full FOV image captured by the macro camera is also downsampled by the ISP to obtain a reduced image output by the ISP, and the position of the partial FOV image is marked in the reduced image according to the position of the partial FOV image in the full FOV image. The position of the partial FOV image in the full FOV image can be obtained by the image cropping process; the marked reduced image is used as the navigation screen; finally, the preview screen and the navigation screen are superimposed and displayed.

[0233] For example, see Figure 6 The schematic diagram of image navigation for macro photography is shown, in which a mobile phone 61 is in macro photography mode, and a photo preview screen 611 is displayed, and a navigation window 612 is superimposed on the photo preview screen 611. The navigation window 612 is used to display the navigation screen, and the position of the preview screen 611 in the full FOV image is marked by a marking box 613 on the navigation screen. In addition, Figure 6 The zoom ratio in is Ax, and the value of A is arbitrary.

[0234] By observing the navigation picture in the navigation window 612 , the user can know the position of the preview picture 611 in the full FOV image, which is convenient for the user to adjust the macro composition and find the macro shooting subject.

[0235] It should be noted that the electronic device 100 may overlay the navigation screen on the preview screen only when the zoom ratio is large or the focal length is high. Figure 6 In the example, the mobile phone 61 includes a telephoto macro camera, and the value of A is greater than 10, that is, the current zoom ratio is greater than 10x, and is in a telephoto high ratio, that is, when the mobile phone 61 is in a high focal length, the navigation screen is superimposed on the preview screen. For another example, Figure 6 The mobile phone 61 includes an ultra-wide-angle macro camera, and the value of A is between 1.5 and 3.5, that is, the current zoom ratio is between 1.5x and 3.5x, which belongs to the high-magnification focal length of the ultra-wide-angle macro camera. Of course, the electronic device 100 can also ignore the values ​​of the zoom ratio and focal length, and after entering the macro shooting mode, the navigation screen is superimposed on the preview screen.

[0236] It should be pointed out that when the image navigation screen and the preview screen are captured by the same macro camera, if the electronic device includes at least two macro cameras, the electronic device can not only perform camera macro switching relay based on the at least two macro cameras to achieve macro shooting with a wider focal length, but also perform image navigation based on a single macro camera during the zoom process of macro shooting.

[0237] At this time, the electronic device 100 uses the partial FOV image of the currently used macro camera as the preview screen and the full FOV image of the macro camera as the preview screen.

[0238] For example, the electronic device includes a telephoto macro camera and an ultra-wide-angle macro camera. When the zoom ratio is between 0.5x and 3.5x, the electronic device uses the ultra-wide-angle macro camera for macro photography, uses the partial FOV image captured by the ultra-wide-angle macro camera as a preview image, and uses the full FOV image captured by the ultra-wide-angle macro camera as a navigation image. When the zoom ratio is greater than 3.5x, after the electronic device switches from the ultra-wide-angle macro camera to the telephoto macro camera, it uses the telephoto macro camera for macro photography, uses the partial FOV image captured by the telephoto macro camera as a preview image, and uses the full FOV image captured by the telephoto macro camera as a navigation image.

[0239] It should be pointed out that when the image navigation screen and the preview screen are captured by the same macro camera, in addition to using the full FOV screen captured by the macro camera as the navigation screen, you can also use the partial FOV screen captured by the macro camera as the navigation screen, as long as the FOV of the navigation screen is larger than the FOV of the preview screen.

[0240] In some other embodiments of the present application, the image navigation screen and the preview screen may not be images captured by the same macro camera, and the FOV of the macro camera of the image navigation screen may be larger than the FOV of the macro camera of the preview screen. In this case, the electronic device 100 may include at least two macro cameras. For example, a mobile phone includes an ultra-wide-angle macro camera and a telephoto macro camera, the FOV of the ultra-wide-angle macro camera is larger than the FOV of the telephoto macro camera, the picture captured by the ultra-wide-angle macro camera is used as the image navigation screen, and the picture captured by the telephoto macro camera is used as the preview screen.

[0241] Of course, the FOV of the macro camera of the navigation screen can also be smaller than the FOV of the macro camera of the preview screen. In this case, the image can be cropped to ensure that the FOV of the image navigation screen is larger than the FOV of the preview screen. That is, the full FOV image captured by the macro camera corresponding to the navigation screen is cropped to obtain a cropped image corresponding to the navigation screen; the full FOV image of the macro camera corresponding to the preview screen is cropped to obtain a cropped image corresponding to the preview screen; the FOV of the cropped image corresponding to the navigation screen is larger than the FOV of the cropped image corresponding to the preview screen.

[0242] It should be pointed out that in addition to using the full FOV image captured by the macro camera as the navigation image, the partial FOV image captured by the macro camera can also be used as the navigation image, as long as the FOV of the navigation image is larger than the FOV of the preview image. Furthermore, when the partial FOV image captured by the macro camera is used as the navigation image, the field of view of the FOV image of the navigation image is different due to different zoom ratios, that is, different FOV images can be cropped from the full FOV image as the navigation image according to the zoom ratio.

[0243] In the image navigation scheme based on at least two macro cameras, the electronic device 100 can switch relays based on at least two macro cameras during the zooming process in the macro shooting mode. In this way, not only a wider focal length macro photography is achieved, but also the convenience of macro composition adjustment and finding macro shooting subjects is improved through the image navigation screen with a larger FOV.

[0244] For example, see Figure 7 A schematic diagram of image navigation is shown, in which the electronic device 100 is a mobile phone 71, and the mobile phone 71 includes a main camera, a telephoto macro camera and an ultra-wide-angle macro camera.

[0245] like Figure 7 As shown, after the mobile phone 71 detects a click operation on the camera 711, in response to the click operation, a photo preview interface 712 is displayed. The photo preview interface 712 includes a photo subject 713, a zoom bar 714, and a zoom control 715. At this time, the photo preview interface 712 displays the picture captured by the main camera, which is a preview picture, and the current zoom ratio is 1x.

[0246] In the shooting preview state, the mobile phone 71 detects the distance between the mobile phone 71 and the photographing subject 713. When it is detected that the shooting distance of N consecutive frames of images is less than the first threshold, the mobile phone 71 confirms that the conditions for automatically entering the macro shooting mode are met, automatically enters the macro shooting mode, and displays the shooting preview interface 716. The shooting preview interface 716 displays the picture captured by the ultra-wide-angle macro camera. The shooting preview interface 716 includes an ultra-wide-angle macro shortcut control 717, a telephoto macro shortcut control 718, and a prompt message 719. At this time, when the mobile phone 71 enters the macro shooting mode, the ultra-wide-angle macro camera is used by default, and the ultra-wide-angle macro shortcut control 717 is in the on state, and the telephoto macro shortcut control 718 is in the off state. The mobile phone 71 can continuously display the prompt message 719 in the macro shooting mode, or cancel the display of the prompt message 719 after continuously displaying the prompt message 719 for a period of time.

[0247] After entering the macro shooting mode, the user drags the zoom control 715 to the right to increase the zoom ratio from 1x to 3.5x. Of course, the user can also quickly switch to the telephoto macro by clicking the telephoto macro shortcut control 718. In response to the user's dragging operation on the zoom control 715, the mobile phone 71 determines that the current zoom ratio is 3.5x, and when the camera switching ratio is reached, the photo preview interface 720 is displayed. The photo preview interface 720 displays the picture captured by the telephoto macro camera. That is, when the camera switching ratio is reached, the photo preview screen of the mobile phone 71 switches from the picture captured by the ultra-wide-angle macro camera to the picture captured by the telephoto macro camera. In this way, through the switching relay of the ultra-wide-angle macro camera and the telephoto macro camera, macro photography from the ultra-wide-angle focal length to the telephoto focal length is realized.

[0248] After the mobile phone 71 switches from the ultra-wide-angle macro camera to the telephoto macro camera, it switches from the ultra-wide-angle macro mode to the telephoto macro mode. Therefore, when the zoom ratio reaches 3.5x, the ultra-wide-angle macro shortcut control 717 on the photo preview interface 720 is in a closed state, and the telephoto macro shortcut control 718 is in an open state.

[0249] In addition, after switching to the telephoto macro camera, the mobile phone 71 also superimposes a navigation window 721 on the photo preview interface 720, and the navigation window 721 is used to display the navigation screen. In addition, a rectangular frame 722 is used to mark the position of the preview screen in the navigation screen. The navigation screen displayed in the navigation window 721 is an interface captured by the ultra-wide-angle macro camera. At this time, the full FOV screen captured by the ultra-wide-angle macro camera can be used as a navigation screen, or the full FOV screen captured by the ultra-wide-angle macro camera can be cropped to a suitable field of view (for example, 78°), and then the cropped image can be used as a navigation screen. During the zoom process of the macro shooting mode, the mobile phone 71 can obtain the position of the preview screen in the ultra-wide-angle image by calculating the image offset value between the center of the telephoto image and the center of the ultra-wide-angle image, and the relative position relationship between the preview screen and the ultra-wide-angle image is identified by a rectangular frame 722. In this way, on the basis of switching between the ultra-wide-angle macro camera and the telephoto macro camera to achieve macro photography with a wider focal length, when the preview screen is switched from the ultra-wide-angle macro camera to the telephoto macro camera, the screen captured by the macro camera with a larger FOV (i.e., the ultra-wide-angle macro camera) is used as the navigation screen, which facilitates users to perform macro composition and find macro shooting targets through the navigation screen with a larger FOV.

[0250] Of course, the electronic device 100 may not use at least two macro cameras for switching relay. In this case, after entering the macro shooting mode, the electronic device 100 uses only one macro camera for macro photography, and does not use multiple macro cameras for switching relay. Instead, a macro camera with a larger FOV is used for image navigation.

[0251] For example, see Figure 8 Another schematic diagram of image navigation is shown, in which the electronic device 100 is a mobile phone 81, and the mobile phone 81 includes a main camera, a telephoto macro camera and an ultra-wide-angle macro camera. The telephoto macro camera is only used for macro photography, and the ultra-wide-angle macro camera is not used for macro photography, but only for image navigation.

[0252] like Figure 8 As shown, after the mobile phone 81 detects a click operation on the camera 811, in response to the click operation, a photo preview interface 812 is displayed, in which a preview image is displayed. The photo preview interface 812 includes a photo subject 813, a zoom bar 814, and a zoom control 815. At this time, the photo preview interface 812 displays the image captured by the main camera, and the current zoom ratio is 1x.

[0253] In the shooting preview state, the mobile phone 81 detects the distance between the mobile phone 81 and the subject 813. When it is detected that the shooting distance of N consecutive frames of images is less than the first threshold, the mobile phone 81 determines that the conditions for automatically entering the macro shooting mode are met, automatically enters the macro shooting mode, and displays the shooting preview interface 816. The shooting preview interface 816 displays the picture captured by the telephoto macro camera. The shooting preview interface 816 includes prompt information 817, a navigation window 818 and a rectangular frame 819.

[0254] The navigation window 818 is used to display the navigation screen, which is the screen captured by the ultra-wide-angle macro camera. The mobile phone 81 obtains the position of the preview screen in the image navigation screen by calculating the image offset value between the center of the ultra-wide-angle image and the center of the telephoto image, and identifies the relative position relationship between the preview screen and the navigation screen through the rectangular frame 819.

[0255] It should be pointed out that the navigation scheme based on at least two cameras with different FOVs provided in the embodiment of the present application can be applied to the macro shooting mode, and when applied to the macro shooting mode, at least two cameras may include some macro cameras and some non-macro cameras. For example, the electronic device includes a telephoto macro camera and a wide-angle camera. At this time, the telephoto macro camera is used for macro photography, and the wide-angle camera is used for image navigation, that is, the picture captured by the telephoto macro camera is used as the preview picture, and the picture captured by the wide-angle camera is used as the navigation picture; it can also be applied to the non-macro shooting mode. In addition, at least two cameras with different FOVs may not be macro cameras. For example, in the normal photo mode, when the mobile phone can reach a zoom ratio of 15x, the picture captured by the telephoto camera is used as the preview picture, and the navigation window is superimposed on the photo preview interface. The navigation picture displayed in the navigation window is the picture captured by the ultra-wide-angle camera.

[0256] It should also be pointed out that the identification pattern used to identify the preview screen and the navigation screen is not limited to the rectangular frame mentioned above, and the display position of the identification pattern is not limited to the position mentioned above. The identification pattern and the display position and display method of the identification pattern are not limited here.

[0257] It should be noted that, based on the image navigation scheme of at least two macro cameras mentioned above, when switching relay based on at least two macro cameras, it is not necessary to perform image offset cropping through the image center alignment scheme mentioned above, but the picture transition is not smooth and natural when the camera is switched.

[0258] Furthermore, based on the image navigation scheme of at least two macro cameras mentioned above, when switching relays based on at least two macro cameras, in order to ensure that the picture transition is smoother and more natural when the camera is switched, the electronic device 100 can also use the above-mentioned image center alignment scheme to perform image offset cropping according to the image offset value. At this time, after the electronic device 100 enters the macro shooting mode, it uses the first macro camera for macro shooting by default, and displays the picture captured by the first macro camera in the preview interface as a preview picture. It is understandable that the electronic device 100 can superimpose the navigation picture on the preview picture after entering the macro shooting mode; it can also superimpose the navigation picture on the preview picture when the zoom ratio reaches the target ratio. The latter is used as an example for introduction and explanation below.

[0259] During the zooming process of the macro shooting mode, the electronic device 100 obtains the current zoom ratio, determines whether the current zoom ratio reaches the camera switching ratio, and whether it reaches the target ratio; if the current zoom ratio reaches the camera switching ratio and reaches the target ratio, the first macro camera is switched to the second macro camera, that is, the preview screen is switched from the first macro camera to the second macro camera, and the screen captured by the first macro camera is superimposed and displayed on the preview screen as an image navigation screen. The FOV of the first macro camera is greater than the FOV of the second macro camera. In addition, during the zooming process to the camera switching ratio, the image offset value between the image center of the first macro camera and the image center of the second macro camera at the current zoom ratio can be calculated by the above-mentioned image center alignment scheme, and the relative position relationship between the navigation screen and the preview screen is identified according to the image offset value, and the image captured by the first macro camera is cropped according to the image offset value, and the cropped image is used as the preview image to be displayed, so as to ensure that the screen transition is more natural and smooth when switching from the first macro camera to the second macro camera.

[0260] For example, the following is combined with Fig. 9 A schematic diagram of a long-focus low-medium magnification image navigation scheme is shown, and Fig.10A flowchart of a long-focus low-medium magnification image navigation solution is shown for introduction and explanation.

[0261] At this time, the electronic device 100 is a mobile phone 91, and the mobile phone 91 includes a main camera, a telephoto macro camera and an ultra-wide-angle macro camera. The target magnification is 3.5x.

[0262] like Fig. 9 As shown, after the mobile phone 91 detects a click operation on the camera 911, in response to the click operation, a photo preview interface 912 is displayed. The photo preview interface 912 includes a zoom bar 914 and a zoom control 915, and also includes a photographing subject 913. At this time, the photo preview interface 912 displays the picture captured by the main camera, and the current zoom ratio is 1x.

[0263] In the shooting preview state, the mobile phone 91 detects the distance between the mobile phone 91 and the subject 913. When it is detected that the shooting distance of N consecutive frames of images is less than the first threshold, the mobile phone 91 confirms that the conditions for automatically entering the macro shooting mode are met, automatically enters the macro shooting mode, and displays the shooting preview interface 916. The shooting preview interface 916 includes an ultra-wide-angle macro shortcut control 917, a telephoto macro shortcut control 918, and a prompt message 919. At this time, when the mobile phone 91 enters the macro shooting mode, the ultra-wide-angle macro camera is used by default, and the ultra-wide-angle macro shortcut control 917 is in the on state, and the telephoto macro shortcut control 918 is in the off state.

[0264] After entering the macro shooting mode, the mobile phone 91 turns on the ultra-wide-angle macro camera and the telephoto macro camera to obtain the ultra-wide-angle image captured by the ultra-wide-angle micro camera and the telephoto image captured by the telephoto macro camera.

[0265] When the macro shooting mode is just entered, the mobile phone 91 performs image offset cropping on the image captured by the ultra-wide-angle macro camera according to the initial offset value to obtain a cropped image, and uses the cropped image as the display image, that is, displays the cropped image on the photo preview interface 916. That is, after the mobile phone 91 enters the macro shooting mode, the preview screen switches from the image captured by the main camera to the image captured by the ultra-wide-angle macro camera.

[0266] In the macro shooting mode, the user can drag the zoom control 915 on the zoom bar 914 to change the zoom ratio. During the zooming process, the mobile phone 91 obtains the current zoom ratio in real time, and determines whether the current zoom ratio reaches the camera switching ratio and whether it reaches the target ratio.

[0267] like Fig. 9As shown, after the mobile phone 91 enters the macro mode for shooting, a photo preview interface 916 is displayed, and the photo preview interface 916 shows that the current zoom ratio is 1x. The user drags the zoom control 915 to the right to change the zoom ratio from 1x to 1.5x, and a photo preview interface 920 is displayed. At this time, the zoom ratio has not reached the camera switching ratio and the target ratio, and the mobile phone 91 continues to use the ultra-wide-angle macro camera for macro photography. In response to the user's dragging operation on the zoom control 915, the mobile phone 91 executes the image center alignment solution during the zooming process to crop the image for display. Specifically, for the telephoto image, the mobile phone 91 performs downsampling processing through the ISP to obtain a telephoto reduced image output by the ISP; for the ultra-wide-angle image, the mobile phone 91 performs image offset cropping through the ISP according to the updated image offset value to obtain a cropped image, and uses the cropped image as the display image, that is, uses the cropped image as the ultra-wide-angle preview image; in addition, the cropped image is also downsampled through the ISP to obtain an ultra-wide-angle reduced image, and image feature matching is performed between the ultra-wide-angle reduced image and the telephoto reduced image, and the image offset value between the center of the telephoto image and the center of the ultra-wide-angle image is calculated according to the image feature matching result and the current zoom ratio, and the offset value is updated according to the image offset value.

[0268] The user continues to drag the zoom control 915 to the right, increasing the zoom ratio from 1.5x to 3.5x. In response to the user's dragging operation on the zoom control 915, the mobile phone 91 determines that the current zoom ratio is 3.5x, reaching the camera switching ratio and the target ratio, and needs to perform camera switching and image navigation screen overlay display.

[0269] Specifically, Fig.10 As shown, for the telephoto image, the mobile phone 91 performs digital zoom cropping on the telephoto image through the ISP to obtain a telephoto cropped image, and uses the telephoto cropped image as a telephoto preview interface, that is, displays the telephoto cropped image as a preview screen on the photo preview interface 921. In addition, the mobile phone 91 also performs downsampling processing on the telephoto cropped image through the ISP to obtain a telephoto reduction image. For the ultra-wide-angle image, the mobile phone 91 performs image offset cropping on the ultra-wide-angle image through the ISP according to the updated image offset value to obtain an ultra-wide-angle cropped image, and performs downsampling processing on the ultra-wide-angle cropped image to obtain an ultra-wide-angle reduction image.

[0270] Then, the mobile phone 91 matches the image features of the ultra-wide-angle reduced image and the telephoto reduced image, calculates the image offset value between the center of the ultra-wide-angle image and the center of the telephoto image according to the image feature matching result and the current zoom ratio, and updates the offset value according to the image offset value. In addition, the mobile phone 91 also marks the position of the telephoto cropped image in the ultra-wide-angle cropped image according to the image offset value, downsamples the marked ultra-wide-angle cropped image, and obtains a marked reduced image. The marked reduced image is displayed as an ultra-wide-angle navigation screen and superimposed on the telephoto preview interface.

[0271] like Fig. 9 As shown, the mobile phone 91 displays the telephoto cropped image on the photo preview interface 921, so that the preview screen switches from the ultra-wide-angle macro camera to the telephoto macro camera, realizing the switching relay of the macro camera. In addition, the mobile phone 91 also overlays the marked reduced image on the preview interface, as shown in FIG. Fig. 9 As shown in the image navigation window 922 in the image navigation window, the image navigation window 922 displays the marked thumbnail image as the navigation screen, and the rectangular frame 923 is used to mark the position of the preview screen in the navigation screen. In the photo preview interface 921, the mobile phone 91 has switched from the ultra-wide-angle macro to the telephoto macro, so the ultra-wide-angle macro shortcut control 917 is in the off state, and the telephoto macro shortcut control 918 is in the on state.

[0272] After switching from the ultra-wide-angle macro to the telephoto macro, the user continues to drag the zoom control 915 to the right to increase the zoom ratio from 3.5x to 4x. In response to the user's dragging operation on the zoom control 915, the mobile phone 91 determines that the current zoom ratio is 4x, which is in the focal length of the telephoto macro camera. At this time, the mobile phone 91 performs the following operations: Fig.10 The process shown is to obtain a telephoto cropped image and display the telephoto cropped image on the photo preview interface 924; and based on the image offset value, mark the position of the telephoto cropped image in the ultra-wide-angle cropped image to identify the position of the telephoto preview screen in the ultra-wide-angle navigation screen, and display the marked zoomed image in the image navigation window 922.

[0273] The user can also continue to drag the zoom control 915 to increase the zoom ratio. When the zoom ratio is greater than 10x, the mobile phone 91 can continue to use the picture captured by the ultra-wide-angle macro camera as the image navigation picture, or when the zoom ratio exceeds the telephoto focal length, the navigation picture is not displayed. However, if the ultra-wide-angle macro camera is continued to be used for navigation at a high magnification, the rectangular frame used for identification on the navigation picture is small, which is not conducive to the accurate navigation of the high-magnification telephoto picture. Therefore, in order to achieve accurate navigation of the high-magnification telephoto picture, the mobile phone 91 can also use the telephoto full FOV picture captured by the telephoto macro camera for the navigation picture when the zoom ratio is greater than 10x, and use the telephoto partial FOV picture captured by the telephoto macro camera as the preview picture.

[0274] In addition, from Fig. 9 It can be seen that when the zoom ratio is in the focal length of the ultra-wide-angle macro camera, i.e., 0.5x to 3.5x, the mobile phone 91 does not superimpose the navigation screen on the preview screen during the zoom process of macro shooting. In other embodiments, the mobile phone 91 can also superimpose the image navigation screen on the focal length of the ultra-wide-angle macro camera. However, when the ultra-wide-angle macro camera has a low-to-medium magnification focal length (e.g., 0.5x to 1.5x), the FOV of the preview screen is still large, and the user can also adjust the macro composition and find the macro shooting target more conveniently. Therefore, the navigation screen may not be displayed in the low-to-medium magnification focal length of the ultra-wide-angle macro camera. When the zoom ratio is in the high-to-medium magnification focal length of the ultra-wide-angle macro camera (e.g., 1.5x to 3.5x), the FOV of the preview screen is small, and the convenience of adjusting the macro composition and finding the macro shooting target is poor. The mobile phone 91 can display the navigation screen by superimposing it to improve the user's interactive experience of macro shooting.

[0275] For example, the following is combined with Fig.11 A schematic diagram showing the fork of the image navigation, Fig.12 The schematic diagram of the ultra-wide-angle high-magnification navigation solution flow chart is shown, and Fig.13 The flowchart of the long-focus and high-magnification navigation scheme is shown for introduction and explanation.

[0276] At this time, the electronic device 100 is a mobile phone 111, and the mobile phone 111 includes a main camera, a telephoto macro camera, and an ultra-wide-angle macro camera. The camera switching magnification is 3.5x, and the target magnification is 1.5x.

[0277] like Fig.11As shown, after the mobile phone 111 detects a click operation on the camera 1111, in response to the click operation, a photo preview interface 1112 is displayed. The photo preview interface 1112 includes a shooting subject 1113, a zoom bar 1114, and a zoom control 1115. At this time, the photo preview interface 1112 displays the picture captured by the main camera as a preview picture, and the current zoom ratio is 1x.

[0278] In the shooting preview state, the mobile phone 111 detects the distance between the mobile phone 111 and the photographing subject 1113. When it is detected that the shooting distance of N consecutive frames of images is less than the first threshold, the mobile phone 111 confirms that the conditions for automatically entering the macro shooting mode are met, automatically enters the macro shooting mode, and displays the photo preview interface 1116. The photo preview interface 1116 includes ultra-wide-angle macro prompt information 1117. At this time, when the mobile phone 111 enters the macro shooting mode, the ultra-wide-angle macro camera and the telephoto macro camera are turned on to obtain the ultra-wide-angle image captured by the ultra-wide-angle micro camera and the telephoto image captured by the telephoto macro camera. In addition, the mobile phone 111 uses the ultra-wide-angle macro camera for macro shooting by default, and the photo preview interface 1116 displays the picture captured by the ultra-wide-angle macro camera.

[0279] When the macro shooting mode is just entered, the mobile phone 111 performs image offset cropping on the image captured by the ultra-wide-angle macro camera according to the initial offset value to obtain a cropped image, and uses the cropped image as the display image, i.e., displays the cropped image on the photo preview interface 1116. That is, after the mobile phone 111 enters the macro shooting mode, the preview screen switches from the image captured by the main camera to the image captured by the ultra-wide-angle macro camera.

[0280] When the zoom ratio is 1x, the zoom ratio falls within the focal length of the ultra-wide-angle macro camera and belongs to the ultra-wide-angle low-medium ratio. The ultra-wide-angle low-medium ratio range is 0.5x to 1.5x, and the ultra-wide-angle high ratio range is 1.5x to 3.5x. Therefore, the mobile phone 111 does not superimpose the navigation screen on the preview screen.

[0281] The user drags the zoom control 1115 to increase the zoom ratio from 1x to 1.5x. In response to the drag operation on the zoom control 1115, the mobile phone 111 determines that the current zoom ratio is 1.5x, reaching the target ratio, not reaching the camera switching ratio, and belonging to the ultra-wide-angle high ratio range, and uses the ultra-wide-angle full FOV screen as the navigation screen and the ultra-wide-angle partial FOV as the preview screen. At this time, the mobile phone 111 executes the following Fig.12 The process shown is used to achieve ultra-wide-angle and high-magnification navigation.

[0282] Specifically, for telephoto images, the mobile phone 111 downsamples the telephoto images through the ISP to obtain a telephoto zoom-out image. For ultra-wide-angle images, the mobile phone 111 performs image offset cropping on the ultra-wide-angle images through the ISP according to the updated image offset value to obtain a cropped image, and uses the cropped image as the ultra-wide-angle preview screen, that is, displays the cropped image on the photo preview interface 1118; the cropped image is also downsampled through the ISP to obtain an ultra-wide-angle zoom-out image, and the ultra-wide-angle zoom-out image and the wide-angle zoom-out image are image feature matched, and the image offset value between the center of the ultra-wide-angle image and the center of the telephoto image is calculated according to the image feature matching and the current zoom ratio, and the offset value is updated according to the image offset value. In addition, the uncropped ultra-wide-angle image is also downsampled to obtain a zoom-out image, and then the position of the ultra-wide-angle partial FOV (i.e., the cropped image) in the ultra-wide-angle full FOV image is marked in the zoom-out image according to the updated image offset value to obtain a marked zoom-out image, and the marked zoom-out image is used as the ultra-wide-angle navigation screen, which is superimposed and displayed on the ultra-wide-angle preview screen. like Fig.11 As shown, an image navigation window 1119 and a rectangular frame 1120 for displaying an image navigation screen are superimposed on the photo preview interface 1118 , and the rectangular frame 1120 is used to identify the position of the ultra-wide-angle preview screen in the ultra-wide-angle navigation screen.

[0283] The user drags the zoom control 1115 to increase the zoom ratio from 1.5x to 3.5x. In response to the dragging operation on the zoom control 1115, the mobile phone 111 determines that the current zoom ratio is 3.5x, reaching the camera switching ratio, and it belongs to the low, medium and high ratios of the telephoto. Then, the camera needs to be switched, and the ultra-wide-angle picture is used as the navigation picture, and the telephoto picture is used as the preview picture. The low and medium ratio range of the telephoto is 3.5x to 10x, and the high ratio range of the telephoto is greater than 10x. At this time, the mobile phone 111 executes the following Fig.10 According to the process shown in the figure, a telephoto preview image and an ultra-wide-angle navigation image are obtained, and the ultra-wide-angle navigation image is superimposed and displayed on the telephoto preview image. Fig.11 As shown, when the zoom is 3.5x, the mobile phone 111 displays a photo preview interface 1121, on which a navigation window 1119 and a rectangular frame 1120 are superimposed. At this time, the navigation screen displayed in the navigation window 1119 is based on Fig.10 The ultra-wide-angle navigation screen is determined by the process shown in the figure, and the preview screen is based on Fig.10 The telephoto preview screen is determined by the process shown.

[0284] The user drags the zoom control 1115 to increase the zoom ratio from 3.5x to 12x. In response to the drag operation on the zoom control 1115, the mobile phone 111 determines that the current zoom ratio is 12x, which is a telephoto high ratio, and needs to use the telephoto full FOV screen as the navigation screen and the telephoto partial FOV screen as the preview screen. At this time, the mobile phone 111 can perform the following operations: Fig.13 The process shown is to achieve long-focus and high-magnification navigation.

[0285] Specifically, for the telephoto full FOV image, the mobile phone 111 performs downsampling processing through the ISP to obtain a reduced image output by the ISP; in addition, the telephoto full FOV image is digitally zoomed and cropped to crop a telephoto partial FOV image from the telephoto full FOV image, and the telephoto partial FOV image is used as a telephoto preview image and displayed on the photo preview interface 1122. In addition, the mobile phone 111 also marks the position of the cropped partial FOV image in the full FOV image according to the cropping position to obtain a telephoto navigation image, and displays the telephoto navigation image on the image navigation window 1119 on the photo preview interface 1122.

[0286] It should be noted that, in the image navigation solution, the macro camera of the electronic device 100 can be a fixed-focus camera or a continuous zoom camera, which is not limited here.

[0287] In the above-mentioned macro shooting scenario, the electronic device 100 can perform switching relay based on at least two macro cameras to achieve macro photography with a wider focal length; in macro photography, it can also superimpose a navigation screen with a larger FOV on the preview screen to improve the convenience of users in adjusting macro composition and finding macro shooting targets.

[0288] When the user finds a suitable macro shooting angle and macro shooting target through the above-mentioned macro camera switching and image navigation operations, the user can trigger the electronic device 100 to perform a shooting operation to obtain a macro shooting picture or a macro shooting video.

[0289] When the electronic device 100 takes a macro picture, the current zoom ratio is different, and the way of taking pictures and outputting pictures may also be different. Exemplarily, the electronic device 100 includes an ultra-wide-angle macro camera and a telephoto macro camera. When the zoom ratio is at an ultra-wide-angle low-medium ratio, the electronic device 100 uses the ultra-wide-angle macro camera to take pictures and output pictures; when the zoom ratio is at an ultra-wide-angle high ratio, the electronic device 100 uses the ultra-wide-angle macro camera to obtain an ultra-wide-angle image, and uses the telephoto macro camera to obtain a telephoto image, and fuses the ultra-wide-angle image and the telephoto image to obtain a fused image, and uses the fused image as the output image; when the zoom ratio is at a telephoto ratio, the electronic device 100 uses the telephoto macro camera to take pictures and output pictures.

[0290] During image fusion, the electronic device 100 can determine the position of the telephoto image in the ultra-wide-angle image by matching image features, that is, determine the area in the ultra-wide-angle image corresponding to the telephoto image, which is the telephoto FOV area; then, the telephoto image is used to improve the image quality of the telephoto FOV image in the ultra-wide-angle image, thereby improving the image quality of macro photography. In specific applications, the image fusion method can be arbitrary. For example, the telephoto image is directly used to replace the telephoto FOV area in the ultra-wide-angle image.

[0291] For example, see Fig.14 The schematic diagram of the fused image is shown, and the fused image 141 is a fused image corresponding to a zoom ratio of 2x, and the fused image includes a telephoto FOV area. When the zoom ratio is 2x, the electronic device 100 shoots a 2x ultra-wide-angle image through the ultra-wide-angle macro camera, and shoots a telephoto image through the telephoto macro camera; the telephoto image and the ultra-wide-angle image are matched for image features to determine the telephoto FOV area in the ultra-wide-angle image, and the telephoto image and the ultra-wide-angle image are fused to obtain a fused image 141, so as to improve the image quality of the telephoto FOV area in the ultra-wide-angle image through the telephoto image.

[0292] For example, the following is combined with Fig.15 A schematic diagram of a macro shooting process is shown for introduction and explanation.

[0293] At this time, the electronic device 100 includes an ultra-wide-angle macro camera and a telephoto macro camera.

[0294] After entering the macro shooting mode, the electronic device 100 turns on the telephoto macro camera and the ultra-wide-angle macro camera, and obtains images captured by the two macro cameras. In the preview stage, when the zoom ratio is less than 3.5x, the ultra-wide-angle display is used, that is, the picture captured by the ultra-wide-angle macro camera is used as the preview picture; when the zoom ratio is greater than or equal to 3.5x, the telephoto display and wide-angle navigation are used, that is, the picture captured by the telephoto camera is used as the preview picture, and the picture captured by the ultra-wide-angle macro camera is used as the navigation picture.

[0295] It should be noted that, regarding the preview navigation part and the image center alignment solution, please refer to the introduction of the corresponding content above, which will not be repeated here.

[0296] In addition, the electronic device 100 also calculates the image offset value between the center of the ultra-wide-angle image and the center of the telephoto image by image feature matching.

[0297] When taking photos and outputting images, when the zoom ratio is less than 2x, the electronic device 100 uses an ultra-wide-angle image output to take photos.

[0298] For example, see Fig.16AA schematic diagram of macro photography is shown, where the mobile phone 161 includes a main camera, a telephoto macro camera and an ultra-wide-angle macro camera.

[0299] After the mobile phone 161 detects a click operation on the camera 1611, in response to the click operation, a photo preview interface 1612 is displayed. The photo preview interface 1612 includes a shooting subject 1613, a zoom bar 1614, and a zoom control 1615. At this time, the zoom bar 1614 is at the 1x position. At this time, the photo preview interface displays the picture captured by the main camera as a preview picture.

[0300] When the mobile phone 161 is in the photo preview state and determines that the conditions for automatically entering the macro shooting mode are met, the photo preview interface 1616 is displayed. The photo preview interface 1616 includes prompt information 1617. When entering the macro shooting mode, the mobile phone 161 uses the ultra-wide-angle macro camera for macro shooting by default, that is, switches from the main camera to the ultra-wide-angle macro camera, and displays the picture captured by the ultra-wide-angle macro camera on the photo preview interface 1616.

[0301] After entering the macro shooting mode, the mobile phone 161 detects a drag operation on the zoom control 1615, and in response to the drag operation, zooms and focuses by pushing the zoom motor and the focus motor, and outputs the image on the preview interface.

[0302] When the mobile phone 161 detects that the zoom control 1615 is at 1.5x, it displays a photo preview interface 1618. The photo preview interface 1618 includes a navigation window 1619 for displaying a navigation screen, and a rectangular frame 1620 for identifying the relative position relationship between the preview screen and the navigation screen. At this time, the zoom ratio of 1.5x falls within the focal length of the ultra-wide-angle macro camera, and the mobile phone 161 uses the ultra-wide-angle partial FOV image as the preview screen and the ultra-wide-angle full FOV image as the navigation screen.

[0303] When the zoom ratio is 1.5x, the mobile phone 161 detects a click operation on the photo control 1621, and in response to the click operation, performs a photo operation. At this time, since the zoom ratio of 1.5x is less than 2x, the ultra-wide-angle output image is used to take a photo, that is, the image captured by the ultra-wide-angle macro camera is used as a photo, and the photo is displayed in the control 1622. When the mobile phone 161 detects a click operation on the control 1622, the photo preview screen 1623 is displayed. The photo displayed on the photo preview screen 1623 is the image captured by the ultra-wide-angle macro camera when the zoom ratio is 1.5x.

[0304] It should be noted that during the zoom process in the macro shooting mode, from the ultra-wide-angle focal length to the telephoto focal length, the above-mentioned image alignment solution can be used to perform image offset cropping to make the picture transition during the zoom process smoother and more natural.

[0305] When the zoom ratio is 2x-3.5x, the electronic device 100 determines the telephoto FOV area in the ultra-wide-angle image according to the updated image offset value, and takes a photo by fusing the ultra-wide-angle image and the telephoto image.

[0306] For example, see Fig. 16B Another schematic diagram of macro photography is shown, based on Fig.16A In the scenario, the mobile phone 161 detects a drag operation on the zoom control 1615 in the photo preview state, responds to the drag operation, and outputs the image on the preview interface.

[0307] When the mobile phone 161 detects that the zoom control 1615 is at 3x, it displays a photo preview interface 1624. The photo preview interface 1624 includes a navigation window 1625 for displaying a navigation screen, and a rectangular frame 1626 for identifying the relative position relationship between the preview screen and the navigation screen. At this time, the zoom ratio 3x falls within the focal length of the telephoto macro camera, and the mobile phone 161 uses the ultra-wide-angle partial FOV image as the preview screen and the ultra-wide-angle full FOV image as the navigation screen.

[0308] When the zoom ratio is 3x, the mobile phone 161 detects a click operation on the photo control 1627, and in response to the click operation, performs a photo operation. At this time, since the zoom ratio 3x is greater than 2x and less than 3.5x, the ultra-wide-angle image and the telephoto image are fused to take pictures, that is, the image captured by the ultra-wide-angle macro camera and the image captured by the telephoto macro camera are fused to obtain a fused image, and the fused image is displayed in the control 1628. When the mobile phone 161 detects a click operation on the control 1628, the photo preview screen 1629 is displayed. The photo displayed on the photo preview screen 1629 is a fused image obtained by image fusion when the zoom ratio is 3x.

[0309] When the zoom ratio is greater than or equal to 3.5x, the electronic device 100 uses a telephoto lens to take photos.

[0310] For example, see Fig. 16C Another schematic diagram of macro photography is shown, based on Fig.16AIn the scene, after entering the macro shooting mode, the mobile phone 161 detects the drag operation on the zoom control 1615, and in response to the drag operation, zooms and focuses by pushing the zoom motor and the focus motor, and outputs the image on the preview interface. At this time, when the mobile phone 161 detects that the zoom control 1615 is at 4x, the photo preview interface 1630 is displayed. The photo preview interface 1630 includes a navigation window 1631 for displaying a navigation screen, and a rectangular frame 1632 for identifying the relative position relationship between the preview screen and the navigation screen. At this time, the mobile phone 161 has switched from the ultra-wide-angle macro camera to the telephoto macro camera. The zoom ratio of 4x falls within the focal length of the telephoto macro camera. The mobile phone 161 uses the picture captured by the ultra-wide-angle macro camera as the navigation screen, and uses the picture captured by the telephoto macro camera as the preview screen.

[0311] When the zoom ratio is 4x, the mobile phone 161 detects a click operation on the photo control 1633, and in response to the click operation, performs a photo operation. At this time, since the zoom ratio 4x is greater than 3.5x, the telephoto output is used to take a photo, that is, the image captured by the telephoto macro camera is used as the photo, and the photo is displayed in the control 1634. When the mobile phone 161 detects a click operation on the control 1634, a photo preview screen 1635 is displayed. The photo displayed on the photo preview screen 1635 is the image captured by the telephoto macro camera when the zoom ratio is 4x.

[0312] It should be noted that in Figures 16A to 16C In the photo-taking scene shown, during the zooming process of the mobile phone 161 in the macro shooting mode, when the zoom ratio reaches the camera switching ratio, the ultra-wide-angle macro camera is switched to the telephoto macro camera. In addition, during the zooming process from the ultra-wide-angle focal length to the telephoto focal length, the image offset value between the ultra-wide-angle image center and the telephoto image center is determined by image feature matching, and the image offset cropping is performed according to the image offset value to achieve alignment of the ultra-wide-angle image center and the telephoto image center. In addition, during the preview stage in the macro mode, the picture captured by the corresponding macro camera is used as the navigation picture and the preview picture according to the zoom ratio.

[0313] It should be noted that although the above is an illustrative introduction based on the photo preview scenario, the macro shooting solution based on at least two macro cameras, the solution for automatically entering and exiting the macro shooting mode, the image center alignment solution, and the preview navigation solution provided in the embodiments of the present application can also be applied to other scenarios, for example, video preview telephoto and video shooting scenarios, etc., which are not limited here.

[0314] For example, see Fig.17A A schematic diagram of a macro video preview scene is shown, where the mobile phone 171 includes a main camera, a telephoto macro camera, and an ultra-wide-angle macro camera.

[0315] like Fig.17A As shown, after the mobile phone 171 detects a click operation on the camera 1711, in response to the click operation, a photo preview interface 1712 is displayed. The photo preview interface 1712 includes a shooting subject 1713, a zoom bar 1714 and a zoom control 1715. At this time, the zoom control 1714 is at the 1x position. At this time, the photo preview interface displays the picture captured by the main camera.

[0316] When the mobile phone 171 detects a click operation on the video control 1716 in the photo preview state, it enters the video mode in response to the click operation and displays the video preview interface 1717. The video preview interface 1717 displays the picture captured by the main camera.

[0317] After the mobile phone 171 enters the video recording mode, in the video recording preview state, if it is detected that the shooting distance of N consecutive frames of images is less than the first threshold, it is determined that the conditions for automatically entering the macro shooting mode are met, the macro shooting mode is entered, and the video recording preview interface 1718 is displayed. The video recording preview interface 1718 includes a prompt message 1719, an ultra-wide-angle macro shortcut control 1720, and a telephoto macro shortcut control 1721. At this time, when the mobile phone 171 enters the macro shooting mode, the ultra-wide-angle macro camera is used by default, and the ultra-wide-angle macro shortcut control 1720 on the video recording preview interface 1718 is in the on state, and the telephoto macro shortcut control 1721 is in the off state. In addition, the video recording preview interface 1718 displays the picture captured by the ultra-wide-angle macro camera.

[0318] The user increases the zoom ratio from 1x to 1.5x by dragging the zoom control 1715. In response to the dragging operation on the zoom control 1715, the mobile phone 171 determines that the current zoom ratio is 1.5x. If the camera switching ratio is not reached, the ultra-wide-angle macro camera is continued to be used for macro shooting, and the picture captured by the ultra-wide-angle macro camera is displayed on the video preview interface 1722. At this time, the current zoom ratio of 1.5x falls within the focal length of the ultra-wide-angle macro camera, and the full FOV picture captured by the ultra-wide-angle macro camera is used as the navigation picture, and the partial FOV picture captured by the ultra-wide-angle macro camera is used as the preview picture, that is, the navigation window 1723 for displaying the navigation picture is displayed on the video preview interface 1722, and the relative position relationship between the preview picture and the navigation picture is indicated by a rectangular frame 1724.

[0319] The user increases the zoom ratio from 1.5x to 3.5x by dragging the zoom control 1715. In response to the dragging operation on the zoom control 1715, the mobile phone 171 determines that the current zoom ratio is 3.5x, reaching the camera switching ratio, and switches from the ultra-wide-angle macro camera to the telephoto macro camera, and displays the picture captured by the telephoto macro camera on the video preview interface 1725 as a preview picture; the picture captured by the ultra-wide-angle macro camera is also used as a navigation picture, which is superimposed on the preview picture, that is, a navigation window 1726 for displaying the navigation picture is displayed, and the relative position relationship between the telephoto image and the ultra-wide-angle image is marked by a rectangular frame 1727.

[0320] In addition, since the mobile phone 171 switches from the ultra-wide-angle macro to the telephoto macro when the zoom ratio is 3.5x, the ultra-wide-angle macro shortcut control 1720 on the video preview interface 1725 is in a closed state, and the telephoto macro shortcut control 1721 is in an open state.

[0321] It can be understood that in the process of zooming from the ultra-wide-angle focal length to the telephoto focal length, the mobile phone 171 can calculate the image offset value between the center of the ultra-wide-angle image and the center of the telephoto image by image feature matching, and perform image offset cropping according to the image offset value to obtain the image for display, thereby making the picture transition during the macro zoom process more natural and smooth.

[0322] For example, see Fig. 17B A schematic diagram of macro video based on Fig.17A In the scenario, after the mobile phone 171 enters the recording mode and displays the recording preview interface 1717, a click operation on the control 1742 is detected. In response to the click operation, the recording is performed and the recording interface 1728 is displayed. At this time, the recording interface 1728 displays the picture captured by the main camera.

[0323] In the recording state, if the mobile phone 171 detects that the shooting distance of N consecutive frames of images is less than the first threshold, it determines that the conditions for automatically entering the macro shooting mode are met, enters the macro shooting mode, and displays the recording interface 1729. The recording interface 1729 includes a prompt message 1730, an ultra-wide-angle macro shortcut control 1731, and a telephoto macro shortcut control 1732. When the mobile phone 171 enters the macro mode, it enters the ultra-wide-angle macro by default, then the ultra-wide-angle macro shortcut control 1731 is in the on state, and the telephoto macro shortcut control 1732 is in the off state. At this time, the recording screen is switched from the main camera to the ultra-wide-angle macro camera, that is, the recording interface 1729 displays the screen captured by the ultra-wide-angle macro camera.

[0324] The user increases the zoom ratio from 1x to 1.5x by dragging the zoom control 1715. In response to the dragging operation on the zoom control 1115, the mobile phone 171 determines that the current zoom ratio is 1.5x, and the camera switching ratio has not been reached, so the ultra-wide-angle macro camera continues to be used for recording, and the picture captured by the ultra-wide-angle macro camera is displayed on the recording interface 1733. In addition, since the zoom ratio of 1.5x belongs to the ultra-wide-angle high ratio, the full FOV picture captured by the ultra-wide-angle macro camera is used as the navigation picture, which is superimposed on the recording picture, as shown in the navigation window 1734 for displaying the navigation picture on the recording interface 1733, and the relative position relationship between the recording picture and the navigation picture is indicated by the rectangular frame 1735.

[0325] The user increases the zoom ratio from 1.5x to 3.5x by dragging the zoom control 1715. In response to the dragging operation on the zoom control 1715, the mobile phone 171 determines that the current zoom ratio is 3.5x, and then switches from the ultra-wide-angle macro camera to the telephoto macro camera, and displays the picture captured by the telephoto macro camera on the video recording interface 1736; the picture captured by the ultra-wide-angle macro camera is used as a navigation picture, which is superimposed on the video recording picture, that is, a navigation window 1737 for displaying the navigation picture is displayed, and the relative position relationship between the video recording picture and the navigation picture is marked by a rectangular frame 1738.

[0326] The user increases the zoom ratio from 3.5x to 4x by dragging the zoom control 1715. In response to the dragging operation on the zoom control 1715, the mobile phone 171 determines that the current zoom ratio is 4x, which belongs to the telephoto focal length, and then continues to display the partial FOV picture captured by the telephoto macro camera on the video recording interface 1739; in addition, the full FOV picture captured by the telephoto macro camera is also used as a navigation picture, which is superimposed on the video recording picture, as shown in the navigation window 1740 for displaying the navigation picture on the video recording interface 1739, and the relative position relationship between the video recording picture and the navigation picture is indicated by a rectangular frame 1741.

[0327] It should be noted that Fig. 17B In the scene, during the zoom process of macro shooting, mobile phone 171 can also calculate the image offset value between the center of the ultra-wide-angle image and the center of the telephoto image by image matching, and perform image offset cropping according to the image offset value to obtain the display image, thereby making the picture transition more natural and smooth during the process of zooming from the ultra-wide-angle focal length to the telephoto focal length.

[0328] The electronic device provided in the embodiment of the present application may include a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, a method as in any one of the above method embodiments is implemented. The embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by the processor, the steps in the above method embodiments can be implemented. The embodiment of the present application provides a computer program product, and when the computer program product runs on an electronic device, the electronic device can implement the steps in the above method embodiments when it is executed. The embodiment of the present application also provides a chip system, the chip system includes a processor, the processor is coupled to the memory, and the processor executes the computer program stored in the memory to implement the methods as in the above method embodiments. The chip system can be a single chip, or a chip module composed of multiple chips.

[0329] In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. It should be understood that the size of the sequence number of each step in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. In addition, in the description of the present application specification and the attached claims, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance. The reference to "one embodiment" or "some embodiments" described in the present application specification means that one or more embodiments of the present application include specific features, structures or characteristics described in combination with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. Finally, it should be noted that the above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed in the present application should be covered within the scope of protection of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A macro shooting method, characterized in that: Applied to an electronic device, the electronic device includes a main camera and at least two macro cameras, the method includes: Starting a camera application and displaying the image captured by the main camera in the viewfinder; If the distance between the electronic device and the subject is less than a preset threshold, the picture displayed in the viewfinder is switched from the picture captured by the main camera to the picture captured by the first macro camera; In response to a user input operation of increasing the zoom ratio, when the zoom ratio is within a first ratio range and the distance between the electronic device and the photographed subject is less than the preset threshold, continuing to display the image captured by the first macro camera in the viewfinder; In response to a user input operation of continuing to increase the zoom ratio, when the zoom ratio is in a second ratio range and the distance between the electronic device and the photographed subject is less than the preset threshold, switching the picture displayed in the viewfinder from the picture captured by the first macro camera to the picture captured by the second macro camera; Wherein, the focal length of the first macro camera is different from the focal length of the second macro camera; The at least two macro cameras include the first macro camera and the second macro camera; the first macro camera is an ultra-wide-angle macro camera or a wide-angle macro camera, the second macro camera is a telephoto macro camera, and the second magnification range is greater than the first magnification range.

2. The method according to claim 1, characterized in that The method further comprises: Displaying a navigation window superimposed on the preview picture displayed in the viewfinder, wherein the navigation window is used to display the navigation picture; The viewing angle of the navigation screen is greater than the viewing angle of the preview screen.

3. The method according to claim 2, characterized in that When the zoom ratio is within the second ratio range, the navigation picture is a picture captured by the first macro camera, and the preview picture is a picture captured by the second macro camera.

4. The method according to claim 2, characterized in that: When the zoom ratio is within the first ratio range, the navigation picture and the preview picture are both pictures captured by the first macro camera; And / or, when the zoom ratio is within the second ratio range, the navigation picture and the preview picture are both pictures captured by the second macro camera.

5. The method according to claim 2, characterized in that: When the zoom ratio is within a first ratio range, the navigation picture and the preview picture are both pictures captured by the first macro camera; When the zoom ratio is within the second ratio range, the navigation picture is a picture captured by the first macro camera, and the preview picture is a picture captured by the second macro camera.

6. The method according to claim 2, characterized in that When the zoom ratio is within a first ratio range, the navigation picture and the preview picture are both pictures captured by the first macro camera; When the zoom ratio is within a third ratio range, the navigation picture is a picture captured by the first macro camera, and the preview picture is a picture captured by the second macro camera; When the zoom ratio is in the fourth ratio range, the navigation picture and the preview picture are both pictures captured by the second macro camera; The second magnification range includes the third magnification range and the fourth magnification range, and the third magnification range is smaller than the fourth magnification range.

7. The method according to claim 2, characterized in that The zoom ratio is within a first ratio range; Displaying the picture captured by the first macro camera in a viewfinder, and superimposing a navigation window on the preview picture displayed in the viewfinder, comprising: During the zooming process in the first magnification range, performing image offset cropping on the image captured by the first macro camera according to an initially calibrated image offset value or a last updated image offset value to obtain a first cropped image; According to the image offset value, a position of the first cropped image in the image captured by the first macro camera is identified by an identification pattern to obtain a first identification image; Displaying the first cropped image in the viewfinder, and displaying the first identification image in the navigation window of the viewfinder; The identification pattern is used to identify the relative position relationship between the preview screen and the navigation screen.

8. The method according to claim 7, characterized in that The method further comprises: Performing image feature matching based on the first cropped image and the image captured by the second macro camera to obtain a first image feature matching result; An image offset value of the image captured by the first macro camera is determined according to the first image feature matching result and the zoom ratio of the image captured by the first macro camera, and the offset value is updated according to the image offset value of the image captured by the first macro camera.

9. The method according to claim 8, characterized in that The determining, according to the first image feature matching result and the zoom ratio of the image captured by the first macro camera, an image offset value of the image captured by the first macro camera comprises: By formula , obtaining an image offset value of the image captured by the first macro camera; in, represents an offset value from the center of the field of view of the first macro camera to the center of the field of view of the second macro camera, which is a pre-calibrated offset value, or the image feature matching result; Indicates the image offset value of the previous frame. Indicates the image offset value of the current frame image; represents the zoom ratio of the previous frame image, Indicates the zoom ratio of the current frame image; The zoom ratio indicating the end of image shift is less than or equal to the camera switching ratio; The zoom ratio indicating the start of image shift is greater than or equal to the minimum zoom ratio of the first macro camera.

10. The method according to claim 7, characterized in that The zoom ratio is within the second ratio range or the third ratio range; the second ratio range includes the third ratio range; Displaying the picture captured by the second macro camera in the viewfinder, and superimposing a navigation window on the preview picture displayed in the viewfinder, comprising: During the zooming process in the second magnification range or the third magnification range, digitally zooming and cropping the image captured by the second macro camera to obtain a second cropped image; According to the initially calibrated image offset value or the last updated image offset value, the position of the second cropped image in the target image is marked by a marking pattern to obtain a second marked image; Displaying the second cropped image in the viewfinder, and displaying the second identification image in the navigation window of the viewfinder; The target image is a third cropped image obtained by performing image offset cropping on the image captured by the first macro camera according to an initially calibrated image offset value or an updated image offset value; or a full field of view image captured by the first macro camera.

11. The method according to claim 10, characterized in that The method further comprises: Performing image feature matching on the second cropped image and the target image to obtain a second image feature matching result; An image offset value of the target image is determined according to the second image feature matching result and the zoom ratio of the target image, and the offset value is updated according to the image offset value of the target image.

12. The method according to claim 10, characterized in that The zoom ratio is in a fourth ratio range; Displaying the picture captured by the second macro camera in the viewfinder, and superimposing a navigation window on the preview picture displayed in the viewfinder, comprising: Digitally zoom and crop the image captured by the second macro camera to obtain a fourth cropped image; Marking the position of the fourth cropped image in the image captured by the second macro camera by using the marking pattern to obtain a third marking image; Displaying the fourth cropped image in the viewfinder, and displaying the third identification image in the navigation window of the viewfinder; The second magnification range includes the third magnification range and the fourth magnification range, and the third magnification range is smaller than the fourth magnification range.

13. The method according to claim 1, characterized in that The method further comprises: If the zoom ratio is within the fifth ratio range and a photographing operation is detected, outputting an image captured by the first macro camera in response to the photographing operation; If the zoom ratio is within the sixth ratio range and a photographing operation is detected, in response to the photographing operation, the image captured by the first macro camera and the image captured by the second macro camera are fused to obtain a fused image, and the fused image is output; If the zoom ratio is within the second ratio range and a photographing operation is detected, outputting an image captured by the second macro camera in response to the photographing operation; The first magnification range includes the fifth magnification range and the sixth magnification range, and the fifth magnification range is smaller than the sixth magnification range.

14. The method according to claim 13, characterized in that The fused image includes a target area and a non-target area, and the image quality of the target area is higher than that of the non-target area; The target area is an area in the image captured by the first macro camera that corresponds to the image captured by the second macro camera.

15. The method according to any one of claims 1 to 14, characterized in that If the distance between the electronic device and the subject is less than a preset threshold, switching the picture displayed in the viewfinder from the picture captured by the main camera to the picture captured by the first macro camera includes: Detecting a first shooting distance of an image captured by the main camera; If the first shooting distance of N consecutive frames of images is less than a first threshold, the macro shooting mode is automatically entered, the main camera is switched to the first macro camera, and the image captured by the first macro camera is displayed in the viewfinder.

16. The method according to claim 15, characterized in that The method further comprises: If the first shooting distances of N consecutive frames of images are all smaller than the first threshold, a prompt message is displayed in the viewfinder, where the prompt message is used to prompt that the macro shooting mode has been entered.

17. The method according to claim 15, characterized in that After entering the macro shooting mode, the method further includes: Detecting a second shooting distance of an image currently captured by the macro camera; If the second shooting distances of the consecutive M frames of images are all greater than a second threshold, switching from the current macro camera to the default camera, and displaying the image captured by the default camera in the viewfinder; The current macro camera is the first macro camera or the second macro camera.

18. The method according to claim 1, characterized in that The viewfinder further includes a first shortcut control and a second shortcut control; The first shortcut control is used to switch the picture displayed in the viewfinder from the picture captured by the first macro camera to the picture captured by the second macro camera; The second shortcut control is used to switch the image displayed in the viewfinder from the image captured by the second macro camera to the image captured by the first macro camera.

19. The method according to claim 1, characterized in that The first magnification range is 0.5x~3.5x, and the second magnification range is 3.5x~15x or greater than or equal to 3.5x; Alternatively, the first magnification range is 0.5x to 3.5x, the second magnification range is 3.5x to 15x or greater than or equal to 3.5x, the third magnification range is 3.5x to 10x, and the fourth magnification range is greater than 10x or 10x to 15x; Alternatively, the first magnification range is 0.5x~3.5x, the fifth magnification range is 0.5x~2x, and the sixth magnification range is 2x~3.5x.

20. A macro photography method, characterized in that: Applied to electronic equipment, the method comprises: Launch the camera app and display the image captured by the main camera in the viewfinder; If the distance between the electronic device and the subject is less than a preset threshold, switching the picture displayed in the viewfinder from the picture captured by the main camera to the picture captured by the first macro camera; In response to an operation of increasing the zoom ratio input by a user, if the zoom ratio is greater than or equal to the target ratio and the distance between the electronic device and the photographed subject is less than a preset threshold, a navigation window is superimposed on the preview screen displayed in the viewfinder, and the navigation window is used to display a navigation screen, and the field of view of the navigation screen is greater than the field of view of the preview screen; the navigation screen is a screen captured by the first macro camera, or a screen captured by the second macro camera, and the field of view of the second macro camera is greater than the field of view of the first macro camera.

21. The method according to claim 20, characterized in that The navigation picture is a picture captured by the first macro camera; Displaying a picture captured by a first macro camera in a viewfinder, and superimposing a navigation window on the preview picture displayed in the viewfinder, including: Performing image offset cropping on the image captured by the first macro camera according to the initially calibrated image offset value or the last updated image offset value to obtain a first cropped image; According to the image offset value, a position of the first cropped image in the image captured by the first macro camera is identified by an identification pattern to obtain a first identification image; Displaying the first cropped image in the viewfinder, and displaying the first identification image in the navigation window of the viewfinder; The identification pattern is used to identify the relative position relationship between the preview screen and the navigation screen.

22. The method according to claim 21, characterized in that The method further comprises: Performing image feature matching based on the first cropped image and the image captured by the second macro camera to obtain a first image feature matching result; An image offset value of the image captured by the first macro camera is determined according to the first image feature matching result and the zoom ratio of the image captured by the first macro camera, and the offset value is updated according to the image offset value of the image captured by the first macro camera.

23. The method according to claim 22, characterized in that The determining, according to the first image feature matching result and the zoom ratio of the image captured by the first macro camera, an image offset value of the image captured by the first macro camera comprises: By formula , obtaining an image offset value of the image captured by the first macro camera; in, represents an offset value from the center of the field of view of the first macro camera to the center of the field of view of the second macro camera, which is a pre-calibrated offset value, or the image feature matching result; Indicates the image offset value of the previous frame. Indicates the image offset value of the current frame image; represents the zoom ratio of the previous frame image, Indicates the zoom ratio of the current frame image; The zoom ratio indicating the end of image shift is less than or equal to the camera switching ratio; The zoom ratio indicating the start of image shift is greater than or equal to the minimum zoom ratio of the first macro camera.

24. The method according to claim 20, characterized in that The navigation picture is a picture captured by the second macro camera; Displaying a picture captured by the first macro camera in a viewfinder, and superimposing a navigation window on the preview picture displayed in the viewfinder, including: Digitally zoom and crop the image captured by the first macro camera to obtain a second cropped image; According to the initially calibrated image offset value or the last updated image offset value, the position of the second cropped image in the target image is marked by a marking pattern to obtain a second marked image; Displaying the second cropped image in the viewfinder, and displaying the second identification image in the navigation window of the viewfinder; The target image is a third cropped image obtained by performing image offset cropping on the image captured by the second macro camera according to an initially calibrated image offset value or an updated image offset value; or, a full field of view image captured by the second macro camera.

25. The method according to claim 24, characterized in that The method further comprises: Performing image feature matching on the second cropped image and the target image to obtain a second image feature matching result; An image offset value of the target image is determined according to the second image feature matching result and the zoom ratio of the target image, and the offset value is updated according to the image offset value of the target image.

26. The method according to claim 20, characterized in that The navigation picture is a picture captured by the first macro camera; Displaying a picture captured by the first macro camera in a viewfinder, and superimposing a navigation window on the preview picture displayed in the viewfinder, including: Performing digital zoom cropping on the image captured by the first macro camera to obtain a fourth cropped image; Marking the position of the fourth cropped image in the image captured by the first macro camera by using the marking pattern to obtain a third marking image; The fourth cropped image is displayed in the viewfinder, and the third identification image is displayed in the navigation window of the viewfinder.

27. The method according to claim 20, characterized in that The method further comprises: When a photographing operation is detected, in response to the photographing operation, the image captured by the first macro camera and the image captured by the second macro camera are fused to obtain a fused image, and the fused image is output; The fused image includes a target area and a non-target area, and the image quality of the target area is higher than that of the non-target area; The target area is an area in the image captured by the first macro camera that corresponds to the image captured by the second macro camera.

28. The method according to any one of claims 21 to 27, characterized in that If the distance between the electronic device and the subject is less than a preset threshold, switching the picture displayed in the viewfinder from the picture captured by the main camera to the picture captured by the first macro camera includes: Detecting a first shooting distance of an image captured by a main camera; If the first shooting distance of N consecutive frames of images is less than a first threshold, the macro shooting mode is automatically entered, the main camera is switched to the first macro camera, and the image captured by the first macro camera is displayed in the viewfinder.

29. The method according to claim 28, characterized in that The method further comprises: If the first shooting distances of N consecutive frames of images are all smaller than the first threshold, a prompt message is displayed in the viewfinder, where the prompt message is used to prompt that the macro shooting mode has been entered.

30. The method according to claim 28, characterized in that After entering the macro shooting mode, the method further includes: Detecting a second shooting distance of the image captured by the first macro camera; If the second shooting distances of the consecutive M frames of images are all greater than the second threshold, the first macro camera is switched to the default camera, and the image captured by the default camera is displayed in the viewfinder.

31. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 19 or 20 to 30 is implemented.

32. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 19 or 20 to 30 is implemented.

Images