Image shooting method and related equipment

Abstract

An image shooting method and related equipment. By implementing the image shooting method provided in an embodiment of the present application, an electronic device can change the direction of a camera to shoot N frames of images to be stitched, then stitch the N frames of images to be stitched, and display the stitched preview image in a stitching preview window, so that the stitched target image has the image resolution of multiple frames and presents more scenes than any frame of the captured image, and at the same time, a visual preview of the image stitching is realized during the image shooting process.

Description

Technical Field

[0001] The present application relates to the field of terminal and communication technology, and in particular to an image capturing method and related equipment. Background Art

[0002] With the development of technology, the shooting functions of electronic devices such as mobile phones have gradually improved, and the same electronic device can have multiple cameras, such as a wide-angle camera and a telephoto camera. Images taken with a telephoto camera have higher image resolution than images taken with a wide-angle camera, but fewer scenes are presented.

[0003] In the photo taking function, the image preview is displayed simultaneously with the taking of the photo.

[0004] How to capture an image with the resolution of an image captured by a telephoto camera while presenting the scenes or even more scenes in the image captured by a wide-angle camera, and realize effective preview of the captured images in this process is a direction worthy of research. Summary of the invention

[0005] The present application provides an image capturing method and related equipment, which can capture multiple frames of images through cameras arranged in an electronic device by changing the direction of one of the cameras and stitching the captured images to obtain a target image, thereby ensuring that the target image has the image resolution of multiple frames of images and presents more scenes than any frame of the captured image, and effectively previewing the image stitching based on the images captured by the two cameras.

[0006] In a first aspect, an image capturing method is provided, which is applied to an electronic device including a first camera and a second camera, wherein a field of view of the first camera is greater than a field of view of the second camera, and the image capturing method includes:

[0007] The electronic device enters the camera preview interface, responds to the photo trigger operation, uses the first camera to obtain the first image, responds to the photo trigger operation, changes the direction of the second camera, uses the second camera to obtain N frames of images to be stitched, and stitches the N frames of images to be stitched to obtain the target image. N is an integer greater than 2.

[0008] The camera preview interface includes a stitching preview window. On this basis, a stitching preview image is displayed in the stitching preview window, and the stitching preview image corresponds to the target image, the image to be stitched, or the first image.

[0009] The corresponding relationship between the stitching preview image and the target image includes, but is not limited to, using the target image as the stitching preview image, and using the target image as the stitching preview image after cropping or further processing.

[0010] The correspondence between the stitching preview image and the image to be stitched includes, but is not limited to, using the image to be stitched as the stitching preview image, using the stitching image generated in the process of stitching the images to be stitched frame by frame as the stitching preview image, and using the image to be stitched as the stitching preview image after cropping or further processing.

[0011] The corresponding relationship between the stitched preview image and the first image includes, but is not limited to, using the first image as the stitched preview image, and using the first image as the stitched preview image after cropping or further processing.

[0012] The stitching preview image corresponds to the image stitching process. In the process of stitching the images to obtain the target image, the stitching preview image can be displayed in the stitching preview window.

[0013] When performing image capture, the photos taken by cameras with different field of view angles in the electronic device have different resolutions and different amounts of scenes presented. In this case, a camera with a rotatable lens is set in the electronic device and the direction of the camera is changed to capture multiple frames of images. Each frame of the image captured has a certain resolution and presents a certain scene. The multiple frames of images are then spliced ​​together so that the spliced ​​target image has the image resolution of multiple frames and presents more scenes than any one frame of the image.

[0014] At the same time, during the image capture process, a visual preview of image stitching is realized, and an equivalent preview of the stitched image is realized by using the image obtained from the image stitching processing of a rotatable camera or the image captured by a camera with a larger field of view, thereby realizing a preview display of the image shooting effect of the image stitching mode.

[0015] In combination with the first aspect, in some implementations of the first aspect, the camera preview interface includes an image preview area, and the stitching preview window is located in the image preview area.

[0016] After entering the camera preview interface, the electronic device uses the first camera to obtain a camera preview image and displays the camera preview image in the image preview area, wherein the first image corresponds to the camera preview image displayed in the image preview area at the triggering moment of the photo triggering operation.

[0017] In this way, the splicing preview window forms a small preview window, and the image preview area forms a large preview window, which together realize the preview display of the image of the current scene to be photographed.

[0018] In combination with the first aspect, in some implementations of the first aspect, in response to a photo triggering operation, changing the orientation of the second camera, and acquiring N frames of images to be stitched using the second camera include:

[0019] In response to the photo triggering operation, the driving motor changes the direction of the second camera according to the set camera trajectory, and uses the N photo taking points of the second camera in the camera trajectory to obtain N frames of images to be stitched.

[0020] In this way, each time the second camera in the electronic device moves to a photographing point along the camera movement trajectory, a frame of image to be processed is captured, and finally N frames of image to be processed are obtained at N photographing points in the camera movement trajectory.

[0021] In combination with the first aspect, in certain implementations of the first aspect, it is necessary to determine the camera movement trajectory in advance.

[0022] In this case, the image capturing method further includes:

[0023] An inner starting point for photographing and an outer ending point for photographing are selected from the field of view of the first camera. Based on the starting point for photographing and the ending point for photographing, a photographing movement sequence among N-2 intermediate photographing points in the field of view is determined. According to the photographing movement sequence, a target movement trajectory starting from the starting point for photographing, passing through the N-2 intermediate photographing points, and reaching the ending point for photographing is determined as the set lens movement trajectory.

[0024] This method of point selection ensures that the camera movement trajectory starts from the center of the lens' movable range as much as possible, saving the time of moving the second camera from the center position to the shooting starting point, achieving fast image output, reducing unnecessary movement distance, and improving the stability of the captured image.

[0025] In combination with the first aspect, in some implementations of the first aspect, selecting an inner starting point for photographing and an outer ending point for photographing from the field of view of the first camera includes:

[0026] A first point located inside the field of view of the first camera is selected as the starting point for photographing, and a second point located outside the field of view of the first camera and having a distance from the first point less than a threshold value is selected as the ending point for photographing.

[0027] In this way, it can be ensured that the end point of shooting is as inner as possible, reducing the movement time of the camera from the end point of shooting to the center position of the movable range of the lens, achieving fast image output, and further reducing unnecessary movement distance, thereby improving the stability of the captured image.

[0028] In combination with the first aspect, in some implementations of the first aspect, displaying the stitching preview image in the stitching preview window includes:

[0029] The first image is cropped to obtain a second image, and the second image is displayed as a stitching preview image in the stitching preview window.

[0030] The image taken by the first camera with a large field of view is cropped to obtain an equivalent preview image corresponding to the target image obtained by stitching, so as to realize a fast preview display of the image showing multiple scenes and high resolution that can be formed after stitching.

[0031] In combination with the first aspect, in some implementations of the first aspect, cropping the first image to obtain the second image includes:

[0032] A second image consistent with the maximum field of view angle covered by the second camera when the orientation is adjusted is cropped from the first image.

[0033] In this way, the image taken by the first camera with a large field of view is cropped to be consistent with the maximum field of view of the rotatable telephoto camera, ensuring the equivalent effect between the cropped preview image and the target image obtained by stitching N images to be stitched.

[0034] In combination with the first aspect, in some implementations of the first aspect, cropping a second image consistent with a maximum field of view angle covered by the second camera when the orientation is adjusted from the first image includes:

[0035] Obtain the zoom ratio of the equivalent focal length of the maximum field of view angle covered by the second camera when the orientation is adjusted relative to the first focal length of the first camera; determine the cropping area coordinates from the first image based on the width and height dimensions and the zoom ratio of the first image; crop the first image based on the cropping area coordinates to obtain the second image.

[0036] In this way, a picture with the same field of view as the stitched image of the second camera can be cropped from the picture taken by the first camera to display the preview effect.

[0037] In combination with the first aspect, in some implementations of the first aspect, displaying the stitching preview image in the stitching preview window includes:

[0038] The target image is displayed as a stitching preview image in the stitching preview window.

[0039] Or it includes cropping the target image to obtain a third image, and displaying the third image as a stitching preview image in the stitching preview window.

[0040] Or it includes, in the process of stitching N frames of images to be stitched to obtain the target image, each stitched image obtained by stitching one frame of images to be stitched is displayed in the stitching preview window as a stitching preview image.

[0041] In this way, a quick preview display of the stitching result based on the image stitching or a dynamic preview display of the stitching process based on the image stitching is achieved.

[0042] In combination with the first aspect, in certain implementations of the first aspect, a display grid is provided in the stitching preview window, the display grid including N grid positions, each grid position corresponding to a photographing point in the camera movement trajectory of the second camera, and a photographing point is used to obtain a frame of image to be stitched using the second camera.

[0043] In this case, in the process of stitching N frames of images to be stitched to obtain the target image, each stitched image obtained by stitching one frame of images to be stitched is displayed in the stitching preview window as a stitching preview image, including:

[0044] From the display grid, in the process of stitching N frames of images to be stitched to obtain a target image, at least one target grid corresponding to each stitched image obtained by stitching a pair of images to be stitched is determined, and the stitched images are sequentially covered and displayed on at least one target grid in the stitching preview window.

[0045] This process integrates the camera movement trajectory into the graphic preview, and combines the camera movement process of dynamic image acquisition with the dynamic image stitching preview display. While realizing the dynamic preview display of image stitching, it improves the realism of the image acquisition and stitching preview display.

[0046] In combination with the first aspect, in some implementations of the first aspect, performing image stitching on N frames of images to be stitched to obtain a target image includes:

[0047] After the first frame of the image to be stitched is acquired by the second camera, each time a frame of the image to be stitched is acquired by the second camera, the frame of the image to be stitched is stitched to obtain a stitched image, until the Nth frame of the image to be stitched is acquired by the second camera, and the Nth frame of the image to be stitched is stitched to obtain a target image.

[0048] Here, N frames of images to be stitched are stitched frame by frame, and each time a frame of images to be stitched is stitched, a stitched image is generated until the target image is finally obtained, so as to implement dynamic preview display of image stitching.

[0049] In combination with the first aspect, in some implementations of the first aspect, each time a frame of the image to be stitched is acquired by using the second camera, stitching the frame of the image to be stitched to obtain a stitched image includes:

[0050] In the case where the first image to be stitched is acquired by the second camera, the center position coordinates of the first image to be stitched and the second image to be stitched are acquired, based on the center position coordinates, the image stitching region between the first image to be stitched and the second image to be stitched is determined, image feature points are detected from the image stitching region, the most similar M pairs of feature points are determined from the image feature points, the affine transformation matrix of the first image to be stitched is calculated according to the M pairs of feature points, the first image to be stitched is affine transformed according to the affine transformation matrix to obtain a third image to be stitched, and the third image to be stitched is fused with the second image to be stitched to obtain a stitched image. M is an integer greater than or equal to 3.

[0051] The second image to be stitched is the first frame of the image to be stitched or is a stitched image that has been stitched.

[0052] In this process, when performing image stitching, the center position coordinates of the two frames of stitched images are introduced each time the image is stitched, and the image is aligned based on the center position coordinates. Then, on this basis, feature point detection, matching and affine transformation are implemented to achieve image fusion and stitching, thereby ensuring the accuracy of image stitching processing.

[0053] In a second aspect, the present application provides an image capturing device, including a first camera and a second camera, wherein the field of view of the first camera is greater than the field of view of the second camera, and the device includes:

[0054] A mode trigger module is used to enter the camera preview interface, which includes a stitching preview window;

[0055] A first shooting module, configured to acquire a first image using a first camera in response to a photo triggering operation;

[0056] A second shooting module is used to change the direction of the second camera in response to a photo triggering operation, and use the second camera to obtain N frames of images to be stitched, where N is an integer greater than 2;

[0057] An image processing module is used to stitch N frames of images to be stitched together to obtain a target image;

[0058] The image preview module is used to display a stitching preview image in the stitching preview window, where the stitching preview image corresponds to the target image, the image to be stitched, or the first image.

[0059] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the methods of the first aspect when executing the computer program.

[0060] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, which can implement any one of the methods of the first aspect when the computer program is executed by a processor.

[0061] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it can implement any one of the methods of the first aspect.

[0062] It can be understood that the image capture device provided in the second aspect, the electronic device provided in the third aspect, the computer-readable storage medium provided in the fourth aspect, and the computer program product provided in the fifth aspect are all used to execute any one of the methods provided in the first aspect of the present application. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1a-Figure 1c A set of exemplary images of scenes captured by a wide-angle camera are shown;

[0064] Figure 2a-2c A group of schematic diagrams showing an electronic device taking images using a telephoto camera;

[0065] Figure 3a-3d A schematic diagram of a camera preview display in an electronic device is shown;

[0066] Figure 3e A schematic diagram of the relationship between camera parameters in a telephoto camera is shown;

[0067] Figure 4 is a schematic flow chart of a shooting method involved in the first scene;

[0068] Figure 5a-5e A set of schematic diagrams for obtaining N frames of images to be stitched and performing image stitching preview in the second scene;

[0069] Figure 6 is a schematic flow chart of a shooting method involved in the second scene;

[0070] Figure 7 A schematic flow chart of splicing N frames of images to be spliced ​​frame by frame;

[0071] Figure 8 A schematic flow chart of a one-cycle processing process for performing image stitching;

[0072] Figure 8a-8f A set of schematic diagrams for determining the camera movement trajectory and performing image stitching preview based on the camera movement trajectory in the third scene;

[0073] Fig. 9 is a schematic flow chart of a shooting method involved in the third scene;

[0074] Fig.10 is a schematic diagram of an electronic device control device according to an embodiment of the present application;

[0075] Fig.11 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0077] It should be understood that the terms "first", "second", etc. in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

[0078] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0079] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that the user can recognize. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operations displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of an electronic device.

[0080] In the same electronic device, multiple cameras and photo-taking functions supported by different cameras can be set according to the device functions.

[0081] Among them, the electronic device is, for example, a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA) or a dedicated camera (for example, a SLR camera, a card camera) and other devices with an image capture function. The embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0082] The camera may be, for example, a normal camera (flat-angle camera), a wide-angle camera, an ultra-wide-angle camera, and a telephoto camera, and the photo function may be, for example, a normal photo function, a wide-angle photo function, and a photo stitching function proposed later in the embodiments of the present application. Different photo functions may be configured in the camera application in the form of photo modes. Under different image capture requirements, different cameras need to be combined to achieve corresponding image capture effects.

[0083] When shooting distant scenes, the zoom ratio is usually increased. At this time, the electronic device will switch to the telephoto camera to shoot. Although the scenery in the picture will be clearer, the scenery captured at the same time will be less.

[0084] If you use a wide-angle camera to shoot at the same viewpoint at this time, although you can capture more scenery, the clarity after magnification will not be as good as that of a telephoto camera.

[0085] In the present application, the electronic device includes two cameras with different field of view angles, wherein the field of view angle of the first camera is larger than that of the second camera, the first camera is, for example, a wide-angle camera or an ultra-wide-angle camera, and the second camera is, for example, a normal camera or a telephoto camera.

[0086] In one solution, in order to present more content in the captured image, the electronic device can use a first camera to shoot. The first camera can be a wide-angle camera or an ultra-wide-angle camera.

[0087] like Figure 1a-Figure 1c A set of exemplary images of scenes captured by a wide-angle camera are shown. Figure 1a The user interface 10 is a main interface of the electronic device. In response to a user's function start-up operation (eg, a click operation, a slide operation, etc.) on the camera application 101, the electronic device can start the camera function.

[0088] like Figure 1b As shown, assuming that the electronic device is in the first position, and for the scene in area 101, the electronic device can shoot a partial scene through the telephoto camera, but can use a wide-angle camera with a larger field of view (larger than the telephoto camera) to shoot the entire scene. In response to the user's photo triggering operation (such as a click operation, a sliding operation, etc.), the electronic device performs the photo operation to obtain image A.

[0089] For viewing the captured image, the electronic device can display Figure 1c The user interface 11 is shown in FIG.

[0090] In the user interface 11, image B may be displayed, and image B may be image A or an image obtained after some processing of image A, so as to realize viewing and displaying of the captured image. The image resolution of image B is low, resulting in unclear image B. For example, a part of image B may be displayed in area 102, and after the part of image is magnified X times, it may be as shown in the image in area 103, and may be unclear.

[0091] Thus, although the wide-angle camera has a larger field of view and can capture more scenes, the image resolution of the image captured is low, resulting in unclear images.

[0092] In this application, in order to present more shooting content in the captured image and have a higher image resolution, the electronic device can use the second camera with adjustable direction to shoot in the aforementioned first position, obtain multiple frames of images of the scene in the aforementioned area 101, and obtain the target image after splicing the images. Among them, the second camera is a rotatable camera, and the hardware can rely on the motor to rotate the lens to achieve the adjustment of the camera lens direction, and rely on this to increase the limitation of the second camera in the field of view angle and expand the shooting coverage area.

[0093] There is a difference in the size of the field of view between the first camera and the second camera. The field of view of the first camera is larger than the field of view of the second camera itself. And there is a conflict between the image resolution and the amount of scenery that can be presented in the image captured by the two cameras.

[0094] In this situation, the embodiment of the present application adds a photo stitching mode to the camera function. In this mode, the second camera obtains multiple frames of images and stitches them to replace the first camera to obtain an image that can present more scenes but has a higher overall image resolution.

[0095] Optionally, the first camera is a wide-angle camera or an ultra-wide-angle camera, and the second camera can be a telephoto camera accordingly. In this case, when shooting a distant scene, the image captured by the second camera is relatively clearer than the image A.

[0096] In this case, in the photo stitching mode added to the camera function, the telephoto camera obtains multiple frames of images and stitches them to replace the wide-angle camera to obtain an image that can present more scenes but has a higher overall image resolution. This mode can be called the "telephoto stitching" mode. Of course, when the first camera and the second camera use other cameras respectively, this mode can also be called other corresponding mode names, and is not limited to this.

[0097] Figure 2a The middle user interface 20 is a function setting item display interface in the camera application. A telephoto stitching control 200 corresponding to the "telephoto stitching" mode can be set in the display interface. In response to the user's operation on the telephoto stitching control 200 (such as a click operation), the electronic device can enter the telephoto stitching mode.

[0098] Figure 2b , Figure 2c A set of schematic diagrams showing an electronic device taking images using a telephoto camera.

[0099] Refer to the above Figure 1bIn the schematic diagram shown, the electronic device is in the first position and does not change. In response to the user's photo triggering operation (such as a click operation, a slide operation, etc.), the electronic device changes the direction of the telephoto camera. Figure 2b In this way, it can first shoot the scene in area 201, and then shoot the scenes in area 202, area 203 and area 204 in sequence, to obtain a total of 4 frames of images to be stitched, and then the electronic device can stitch the 4 frames of images to be stitched to obtain image C.

[0100] like Figure 2c As shown, the image displayed in the user interface 21 may be image D, which may be image C or an image obtained after some processing of image C, so as to realize the viewing and display of the captured image. The image D may display the scene in the area 101, and its resolution is higher than that of image B. For example, a part of the image D may be displayed in the area 211, and after the part of the image is magnified X times, it may be shown as the image shown in the area 212, and compared with the image shown in the aforementioned area 103, there will be no unclear problem.

[0101] It can be understood that if the first camera mentioned above is an ultra-wide-angle camera, the second camera may also be a wide-angle camera.

[0102] In this way, the electronic device can use a camera with a small field of view to capture a wider range of scenes, so that more scenes can be presented in the image, while the image resolution will not be reduced, making the image clear.

[0103] The present application provides an image shooting method. According to the image shooting method, in a scenario where the resolution of photos taken by cameras with different field of view angles is different and the amount of scenes presented is different, a camera with a rotatable lens is set in an electronic device, and the direction of the camera is changed to shoot multiple frames of images, each frame of the image shot has a certain resolution and the scene presented, and then the multiple frames of images are spliced, so that the spliced ​​target image has the image resolution of multiple frames and presents more scenes than any frame of the image shot.

[0104] At the same time, during the image capture process, a visual preview of image stitching is realized, and an equivalent preview of the stitched image is realized by using the image obtained from the image stitching processing of a rotatable camera or the image captured by a camera with a larger field of view, thereby realizing a preview display of the image shooting effect of the image stitching mode.

[0105] In one solution, in order to make the image stitching processing operation have a visual display effect, a stitching preview window is set in the camera preview interface of the camera function, and a preview image of the image stitching is displayed in the stitching preview window.

[0106] Figure 3a , Figure 3b A schematic diagram of a camera preview interface for calling out a stitching preview window in an electronic device is shown.

[0107] like Figure 3a As shown, the user interface 30 is a camera preview interface. The electronic device starts the camera function and enters the camera preview interface. After the electronic device enters the preview interface in the camera application, the first camera is turned on and starts to collect images. The preview box 301 of the camera preview interface displays the image content collected by the first camera. When the electronic device triggers the "telephoto stitching mode" in the camera application, in response to the user's photo triggering operation on the shooting control 302 in the camera preview interface, in response to the photo triggering operation, the second camera starts to collect images from the same viewpoint, and Figure 3b As shown in the user interface 31, the electronic device displays a stitching preview window 311 in the camera preview interface, and synchronously changes the lens direction of the second camera to capture N images through the second camera.

[0108] Figure 3b In the user interface 31, the stitching preview window 311 can be displayed in the camera preview interface in the form of a small preview window.

[0109] The preview frame 301 forms a large preview window, and the image captured by the first camera is synchronously displayed in the camera preview interface through the large preview window. Optionally, the display position of the splicing preview window 311 is located in the large preview window formed by the preview frame 301. In this way, the splicing preview small window and the wide-angle preview large window jointly realize the preview display of the image of the scene to be photographed.

[0110] The stitching preview image corresponding to the stitching of the image of the second camera is displayed in the stitching preview window 311. The stitching preview display may be a preview display based on the image with more scenes taken by the first camera, or a preview display based on the target image generated after stitching of N images to be stitched taken by the rotating second camera, or a preview display based on the image stitching process of N images to be stitched taken by the rotatable second camera in the process of stitching the target image.

[0111] In the implementation process, the first camera may be a wide-angle camera or an ultra-wide-angle camera, and the second camera may be a telephoto camera. Alternatively, the first camera may be an ultra-wide-angle camera, and the second camera may be a wide-angle camera. Alternatively, the first camera may be a normal camera, and the second camera may be a telephoto camera.

[0112] The following takes the first camera as a wide-angle camera and the second camera as a telephoto camera as an example, and combines three specific implementation application scenarios to describe in detail the visual preview display method of image stitching in the image capture method of the embodiment of the present application.

[0113] First scene: preview and display the first image captured by the first camera.

[0114] The user interface involved in this process can refer to the above Figure 1a , Figure 2a , Figure 2b , Figure 3a-3d The user interface and images in are shown.

[0115] like Figure 1a As shown, the user interface 10 is a main interface of the electronic device. In response to the user's function start operation on the camera application 101, the electronic device can start the camera function and enter the camera preview interface, see Figure 3a The user interface 30 in FIG.

[0116] After entering the camera preview interface, the electronic device first uses the wide-angle camera to obtain an image. The current camera preview interface displays the image captured by the wide-angle camera, realizing a preview display of the image captured by the current camera.

[0117] In the camera function, users can choose Figure 2a In the function setting item display interface shown, the telephoto stitching control 200 is selected to trigger the electronic device to enter the telephoto stitching mode of the camera function. Alternatively, the user can adjust the zoom ratio to a set value based on the current photo taking needs, for example, adjust the zoom ratio to 3.5 times the zoom ratio, and the electronic device enters the "telephoto stitching mode" of the camera function.

[0118] In response to the user's photo triggering operation on the shooting control 302 in the camera preview interface, the electronic device enters Figure 3b In the user interface 31 shown in FIG. 3 , the electronic device can continue to use the wide-angle camera to obtain a camera preview image and display the preview image in the preview frame 301, and call out a spliced ​​preview window 311. In response to the photo triggering operation, the electronic device performs a photo operation.

[0119] The photo operation includes two parts, one of which is to shoot the image captured by the wide-angle camera previewed and displayed in the preview box 301 at the triggering moment of the photo trigger operation to obtain image A (i.e., the first image captured by the first camera). At the same time, the other part is to adjust the lens direction of the telephoto camera, obtain N frames of images to be stitched, and stitch them to obtain the target image.

[0120] In the process of acquiring N frames of images to be stitched, a stitching preview image is synchronously displayed in the stitching preview window 311 . The stitching preview image needs to be obtained based on the image captured by the wide-angle camera.

[0121] During the stitching preview process, image A can be used as a stitching preview image, or image A can be cropped or superimposed with more other processing as a stitching preview image and displayed in the stitching preview window 311, so as to realize a quick preview display of the image with multiple scenes and high resolution that can be formed after stitching.

[0122] After acquiring N frames of images to be stitched, stitch or superimpose more other processing on the N frames of images to be stitched to obtain and save the target image. Wherein, N is a positive integer greater than or equal to 2, usually a positive integer raised to the power of 2, and the value of N is determined by the field of view of the current wide-angle camera and the field of view of the telephoto camera.

[0123] Among them, in a preview processing process, combined with Figure 3c As shown, considering that there are black edges in the image when the wide-angle camera lens is used for imaging, the captured image A needs to be cropped at the edges according to a reasonable size to remove the black edges, thereby obtaining a spliced ​​preview image.

[0124] Alternatively, in a preview processing process, considering the difference in field of view angles between the wide-angle camera and the telephoto camera, and combined with the maximum angular range in which the lens of the telephoto camera can rotate, there may be a difference in the coverage of the field of view angle between the wide-angle camera and the rotatable telephoto camera. It is necessary to crop from image A an image that is consistent with the maximum field of view angle that can be covered when the telephoto camera is adjusted in direction, and then obtain a stitched preview image.

[0125] Combination Figure 1b , Figure 2b , Figure 3d As shown, the coverage of the field of view of a single wide-angle camera is the range of area 101, and the coverage of the field of view of a single telephoto camera is the range of area 201. In the current example, the lens orientation of the telephoto camera can be rotated and adjusted to be oriented toward area 201, area 202, area 203, and area 204, respectively. If the lens rotation angle range of the telephoto camera under the four lens orientations is taken as the maximum rotatable angle range of the lens, then the coverage of the field of view of the rotatable telephoto camera under the maximum rotatable angle range of the lens is the area range formed by area 201, area 202, area 203, and area 204, that is, the range of area 301, see Figure 3d shown.

[0126] Here, the image corresponding to the maximum field of view angle that can be covered by the rotatable telephoto camera when the direction is adjusted is the target image obtained by stitching N images to be stitched, and the target image corresponds to the field of view angle coverage area 301 under the maximum angle range of the lens rotation.

[0127] Combination Figure 3d As shown, the wide-angle camera captures an image A corresponding to the area 101, and the area 302 in the image A is a cropped area, which corresponds to the field of view angle coverage area 301 of the rotatable telephoto camera. According to the cropped area, the image A is cropped to be consistent with the maximum field of view of the rotatable telephoto camera, and an equivalent preview image (i.e., a spliced ​​preview image) corresponding to the target image obtained by splicing N images to be spliced ​​is obtained, so as to realize a fast preview display of the image that can be formed after splicing and presents multiple scenes and has a high resolution.

[0128] Among them, in order to make up for the difference in coverage of the field of view angle between the wide-angle camera and the rotatable telephoto camera, it is necessary to pre-calibrate the image scaling ratios corresponding to the wide-angle camera and the rotatable telephoto camera already equipped in the electronic device, that is, the corresponding Zoom ratio, and use the Zoom ratio to crop from image A an image that is consistent with the maximum field of view angle that can be covered when the rotatable telephoto camera is adjusted in direction.

[0129] Therefore, before setting the “telephoto stitching mode”, you need to first set the Zoom ratio that is compatible with the “telephoto stitching mode”.

[0130] Specifically, it is necessary to first measure the field of view angle of the rotatable telephoto camera when the lens is rotated to the maximum angle. , combined with the diagonal length d of the image sensor in the telephoto camera, the formula is used to calculate the field angle Equivalent focal length , see Figure 3e As shown:

[0131] ;

[0132] ;

[0133] Then, calculate the equivalent focal length Focal length with wide-angle camera The ratio between:

[0134] ;

[0135] Get the field of view angle corresponding to the rotatable telephoto camera The zoom ratio relative to the field of view of the wide-angle camera, i.e. value.

[0136] In the subsequent step, based on the zoom ratio and the width and height of image A, Figure 3d As shown, the coordinates of the starting position and the ending position of the cropping area are determined from image A. For the determination of the coordinates of the cropping area, the coordinates of the cropping starting point Q can be determined as ( ), the coordinates of the clipping end point P are ( ).

[0137] Based on the coordinates, a picture with the same field of view as the stitched image of the telephoto camera is cropped from the picture taken by the wide-angle camera to display the preview effect.

[0138] In the application, the first camera can be the main camera. After the main camera image is cropped to be consistent with the maximum field of view of the rotatable telephoto camera, it is sent for display to achieve main camera display.

[0139] In combination with the above description of the first scenario, the process of an electronic device acquiring a target image and an image preview using the shooting method in the embodiment of the present application is described in detail below.

[0140] Figure 4 It is a schematic flowchart of the shooting method involved in the first scene.

[0141] In the first scenario, the process of the electronic device acquiring the target image and image preview using the shooting method in the embodiment of the present application can refer to the following description of steps 101 to 107.

[0142] Step 101: the electronic device enters a camera preview interface.

[0143] The camera preview interface includes an image preview area, which is used to preview the image captured by the camera.

[0144] After entering the camera preview interface, the electronic device uses a 1x zoom ratio to shoot the scene by default. At this time, the electronic device can use the wide-angle camera to collect images. The user can set the zoom ratio when the electronic device collects images, for example, setting the zoom ratio from 1x zoom ratio to 3x zoom ratio.

[0145] Under different zoom ratios, electronic devices can use different cameras to capture images and send them for display. For example, when the zoom ratio is greater than 3.5 times, the electronic device uses a wide-angle camera and / or a telephoto camera. When the zoom ratio is less than 1.0 times, an ultra-wide-angle camera is used, and only a wide-angle camera can be used in other cases. It should be understood that this is only an exemplary description, and the impact of the zoom ratio on the camera used to capture images has different configurations in different electronic devices, and no examples will be given.

[0146] After entering the camera preview interface, the electronic device can capture images based on the wide-angle camera in the default state, and preview and display them in the image preview area in the camera preview interface.

[0147] Differently, a stitching preview window can be set in the camera preview interface. When the zoom ratio is greater than 3.5 times, the electronic device uses the telephoto camera to take N images and stitch the images, and the image stitching is previewed in the stitching preview window.

[0148] Step 102: Acquire a camera preview image using a first camera.

[0149] The electronic device includes a first camera and a second camera, and the field of view of the first camera is greater than that of the second camera.

[0150] Here, the first camera may be a wide-angle camera or an ultra-wide-angle camera, and the second camera may be a telephoto camera. The first camera may be an ultra-wide-angle camera, and the second camera may be a wide-angle camera. The first camera may be a normal camera, and the second camera may be a telephoto camera.

[0151] Step 103: display the preview image in the image preview area.

[0152] The electronic device uses the first camera to capture images, and two or more frames of images can constitute an image sequence. The electronic device can send the captured images for display, that is, after pre-processing the captured images, send them to the display screen for preview in the image preview area of ​​the camera preview interface. An exemplary user interface involved in the preview can be the aforementioned Figure 3a The user interface 30 is shown in FIG.

[0153] Step 104: In response to a photo-taking trigger operation, a first image is acquired using a first camera.

[0154] Exemplarily, the first image corresponds to a preview image displayed in the image preview area at the triggering moment of the photographing triggering operation.

[0155] Specifically, the first image is a frame of image in an image sequence acquired by the first camera.

[0156] The photo triggering operation may be the aforementioned Figure 3a In the user interface 30 shown, an operation (eg, a click operation) on the shooting control 302. When the electronic device detects a photo triggering operation of the user on the shooting control 302, a frame of image acquired at the time of the operation triggering in the image sequence is used as the first image.

[0157] Step 105 , in response to the photo triggering operation, changing the direction of the second camera, and using the second camera to obtain N frames of images to be stitched.

[0158] Wherein, N is an integer greater than 2.

[0159] Here, the second camera is a rotatable camera. Figure 3a After the user triggers the photo shooting operation on the shooting control 302, the electronic device can control the motor or other driving device to adjust the lens direction of the second camera to expand the coverage range of the image captured by the second camera lens as much as possible and expand the field of view that can be achieved by the camera shooting images.

[0160] The process of the second camera changing its direction to capture an image can be referred to Figure 2b As shown in the figure, each time the camera lens direction is changed, the area that the camera can shoot changes accordingly. The camera collects an image of the current shooting area. After changing N times ( Figure 2b After changing 4 times) N images to be stitched are obtained ( Figure 2b 4 images to be stitched are obtained).

[0161] Step 106 , stitching the N frames of images to be stitched together to obtain a target image.

[0162] After obtaining N images to be stitched, the N images to be stitched can be stitched to obtain a target image.

[0163] In this way, the field of view corresponding to the obtained target image is greater than the field of view of any image in the N frames of images to be stitched. The target image finally stitched together can present more scenes without reducing the image resolution, making the image clear.

[0164] Among them, when performing image stitching, if there are image areas with overlapping content between the N frames of images to be stitched, it is necessary to perform feature deduplication on these overlapping image areas to achieve image stitching and fusion. If there are no image areas with overlapping content between the N frames of images to be stitched, the pixel feature matching points between different images can be determined based on the edge image content of each frame of the image to be stitched, and the image stitching and fusion can be achieved based on the feature matching points.

[0165] Step 107 , cropping a second image consistent with the maximum field of view angle covered by the second camera when the orientation is adjusted from the first image, and displaying the second image as a stitching preview image in the stitching preview window.

[0166] The location of the stitching preview window in the camera preview interface can be set according to the user's usage habits or aesthetic preferences. The stitching preview window is located in the image preview area. Figure 3b As shown, a preview frame can be set in the camera preview interface to form an image preview area, and a splicing preview window 311 is set in the preview frame 301 .

[0167] Here, the first image needs to be cropped to obtain the second image.

[0168] Combination Figure 3d , Figure 3e As shown, in the process of cropping out the second image consistent with the maximum field of view angle covered when the second camera is adjusted in direction from the first image, the field of view angle corresponding to the image shooting range when the telephoto camera is rotated to the maximum angle is measured, and based on the field of view angle, the zoom ratio of the telephoto camera relative to the wide-angle camera is calculated, that is, the zoom ratio of the equivalent focal length of the maximum field of view angle covered when the second camera is adjusted in direction relative to the first focal length of the first camera is obtained.

[0169] Then, based on the zoom ratio and in combination with the width and height of the first image, cropping coordinates are determined from the first image, and the first image is cropped based on the cropping coordinates to obtain a second image to replace the target image obtained by stitching for image preview display.

[0170] Optionally, the image content in the target image is the same as the image content in the second image.

[0171] In this process, the stitching preview image corresponds to the first image, and the stitching preview image is displayed in the stitching preview window. The image taken by the wide-angle camera is used as the basis for implementing the image stitching preview in the telephoto stitching mode, so as to realize a fast preview display of the image that can be formed after stitching and presents multiple scenes and has a high resolution.

[0172] Second scenario: based on the image captured by the second camera, the image stitching is previewed and displayed.

[0173] In this scenario, there are two implementation methods. One is to preview and display the target image finally obtained by stitching N images to be stitched, that is, to preview and display the image stitching result (target image). The other is to preview and display the different stitching images obtained in the stitching process of N images to be stitched, that is, to preview and display the image stitching process (the stitching process of the target image). In this way, a corresponding relationship is established between the stitching preview image and the target image and the image to be stitched.

[0174] The electronic device implements preview display of image stitching based on the images captured by the second camera under the two different implementation modes.

[0175] Refer to the above Figure 3aSchematic diagram of user interface 30 is shown. After the electronic device enters the "telephoto stitching mode" of the camera function, the electronic device uses the first camera to obtain a camera preview image at a zoom ratio of 3.5 times, and the electronic device can continue to use the wide-angle camera to obtain a camera preview image and display the preview image in the preview box 301 of the user interface.

[0176] In response to the user's photo triggering operation on the shooting control 302 in the camera preview interface, refer to the aforementioned Figure 3b In the schematic diagram of the user interface 31 shown, the electronic device calls up a stitching preview window 311 and synchronously performs a photo-taking operation, specifically, adjusting the lens orientation of the telephoto camera, using the telephoto camera to obtain N frames of images to be stitched, and then stitching or superimposing more other processing on the N frames of images to be stitched to obtain a target image.

[0177] Figure 5a-5d This is a set of schematic diagrams for obtaining N frames of images to be stitched in the second scene.

[0178] like Figure 5a As shown, assuming that at this time, the electronic device determines that N is 4 according to the field of view of the wide-angle camera and the field of view of the telephoto camera, the motor can be moved 4 times according to the preset trajectory to change the direction of the second camera and obtain 4 frames of images to be stitched. It allows it to first shoot the scenery in area 501, and then shoot the scenery in area 502, area 503 and area 504 in turn to obtain 4 frames of images to be stitched. When the direction of the second camera is changed when the motor is moved twice in a row, the scenery shot twice by the second camera can have an overlapping area, so that when shooting two consecutive frames of images to be stitched, there is an overlapping area between the two frames of images to be stitched. For example, area 505 is the overlapping area when the electronic device shoots area 501 and area 502. The area of ​​the overlapping area can be 15%-25% of the image, for example 20%.

[0179] Take the example of an electronic device acquiring a first frame of images to be stitched and a second frame of images to be stitched as an example:

[0180] like Figure 5b As shown, it is assumed that the initial orientation of the second camera can enable it to capture the scene in area 506. The motor moves for the first time to change the orientation of the second camera, so that the second camera can capture the scene in area 501, and obtain the first frame of the image to be stitched. Then, the motor moves for the second time to change the orientation of the second camera.

[0181] like Figure 5c As shown, the second camera can capture the scene in area 502 to obtain a second frame of image to be stitched. At this time, the overlapping parts in area 501 and area 502 can be as follows Figure 5c Area 505 is shown in FIG.

[0182] The process of the electronic device acquiring the third frame of the image to be stitched and the fourth frame of the image to be stitched can refer to the above Figure 5a-5c The description is not repeated here.

[0183] like Figure 5d As shown, it is a schematic diagram of an electronic device acquiring 4 frames of images to be processed. For example, the 4 frames of images may be images 1 to 4. Among them, area 507 in image 1 and area 507 in image 2 are overlapping areas. Area 508 in image 2 and area 508 in image 3 are overlapping areas. Area 509 in image 3 and area 509 in image 4 are overlapping areas.

[0184] Then, based on the image captured by the second camera, the image stitching is performed to preview the image.

[0185] Among them, in one of the aforementioned implementations, the electronic device can use the overlapping areas in the four frames of images to be processed to stitch them into one frame of image as the target image, or process the one frame of image to obtain the target image. The obtained target image can refer to the aforementioned Figure 2c The image D is shown in the user interface 21.

[0186] During the image stitching process, there are N images to be stitched and a target image obtained by stitching.

[0187] Therefore, when previewing the stitching, the target image obtained by stitching can be directly displayed in the stitching preview window as a stitching preview image, so as to realize a quick preview display of the image presenting multiple scenes and high resolution that can be formed after stitching; or, the target image obtained by stitching can be further cropped and then displayed in the stitching preview window as a stitching preview image after additional processing, so as to satisfy the presentation effect of the preview display and realize a quick preview display of the image presenting multiple scenes and high resolution that can be formed after stitching.

[0188] See above Figure 3b As shown, a spliced ​​preview image is displayed in a preview window formed by the spliced ​​preview window 311. The spliced ​​preview image is a target image obtained by splicing or an image obtained by splicing after additional processing such as cropping of the target image.

[0189] Alternatively, in another embodiment described above, in the process of stitching N frames of images to be stitched to obtain a target image, each time a pair of images to be stitched are stitched, a corresponding stitched image is generated, and the stitched image is displayed in a stitching preview window, thereby realizing a dynamic stitching display of the process of stitching N frames of images to be stitched to obtain an icon image, and realizing a visual dynamic display of image stitching.

[0190] During the implementation of this method, it is necessary to stitch the N images to be stitched together taken by the telephoto camera one by one, wherein the stitching operation can occur during the shooting process of the N images to be stitched together, that is, stitching one image at a time, or after the shooting of the N images to be stitched together is completed, stitching the N images to be stitched together one by one in sequence.

[0191] Figure 5e It is a schematic diagram of stitching N frames of images to be stitched one by one and previewing them in the stitching preview window.

[0192] Combination Figure 5d As shown, the electronic device obtains 4 frames of images to be processed, and the 4 frames of images may be image 1 to image 4.

[0193] Figure 5e The middle window 60 is a schematic diagram of the stitching preview window. After the first frame of the image to be stitched, i.e., image 1, is acquired, image 1 is used as preview image 1 or processed as preview image 1 and displayed in window 60. Then the next frame of the image is acquired. After image 2 is acquired, image 2 is stitched with image 1 to obtain preview image 2, which is displayed in window 60. At this time, preview image 1 displayed in window 60 is replaced by preview image 2. Later, image 3 is acquired. After image 3 is acquired, image 3 is stitched with preview image 2 to obtain preview image 3, which is displayed in window 60. At this time, preview image 2 displayed in window 60 is replaced by preview image 3. Finally, the fourth frame of the image to be stitched is acquired. After image 4 is acquired, image 4 is stitched with preview image 3 to obtain preview image 4, which is displayed in window 60. At this time, preview image 3 displayed in window 60 is replaced by preview image 4. And because image 4 is the last frame of the image to be stitched, the preview image 4 obtained by stitching based on image 4 is the preview image corresponding to the target image formed after all the images to be stitched are stitched together.

[0194] In this process, one image to be stitched is acquired and processed at a time, and the generated stitched image is output and displayed in the stitching preview window in real time, which can effectively display the dynamic stitching processing of N frames of images to be stitched during the acquisition process, realize the dynamic stitching display of the process of stitching N frames of images to be stitched to obtain the icon image, and realize the visualization of the dynamic stitching processing of the image.

[0195] Through the above-mentioned various implementations, when the preview display of image stitching is realized, different corresponding relationships can be established between the stitched preview image and the target image generated by stitching, and the preview display of image stitching based on the image captured by the second camera can be realized in different implementations.

[0196] In combination with the above description of the second scenario, the process of an electronic device acquiring a target image and an image preview using the shooting method in the embodiment of the present application is described in detail below.

[0197] Figure 6 It is a schematic flowchart of the shooting method involved in the second scene.

[0198] In the second scenario, the process of the electronic device acquiring the target image and image preview using the shooting method in the embodiment of the present application can refer to the following description of steps 201 to 209.

[0199] Step 201: the electronic device enters a camera preview interface.

[0200] The camera preview interface includes a stitching preview window.

[0201] Step 202: Acquire a camera preview image using a first camera.

[0202] The camera preview interface includes an image preview area.

[0203] Step 203: display the preview image in the image preview area.

[0204] Step 204: In response to a photo-taking trigger operation, a first image is acquired using a first camera.

[0205] The first image corresponds to the preview image displayed in the image preview area at the triggering moment of the photographing triggering operation.

[0206] Step 205 , in response to the photo triggering operation, the orientation of the second camera is changed, and N frames of images to be stitched are acquired by using the second camera.

[0207] Wherein, N is an integer greater than 2.

[0208] The implementation process of the above steps 201 to 205 can refer to the implementation process described in the first scenario of steps 101 to 105, and refer to Figure 5a-5e The related implementation process of obtaining N frames of images to be stitched will not be described in detail here.

[0209] Different from the first scenario in which an equivalent preview of the stitched image is performed based on the image taken by the wide-angle camera, in the second scenario, after obtaining N frames of images to be stitched, an image stitching preview is performed based on the N images to be stitched taken by the telephoto camera and the final stitched target image.

[0210] In one embodiment, based on the N frames of images to be stitched acquired by the second camera, an image stitching preview may be performed during the image stitching process. Specifically, step 206 is performed.

[0211] Step 206 , each time a pair of images to be stitched are stitched together to obtain a stitched image, the current stitched image is used as a stitching preview image and displayed in the stitching preview window.

[0212] Here, the N frames of images to be stitched are stitched frame by frame, and each time a frame of images to be stitched is stitched, a stitched image is generated. And each time a stitched image is generated, the stitched image is displayed in the stitching preview window for preview. In the image stitching process, a dynamic image stitching preview is implemented along with the image stitching process. For specific image stitching preview display, please refer to Figure 5e shown.

[0213] In one embodiment, the target image can be finally obtained by stitching N frames of images to be stitched obtained by the second camera, and image stitching preview can be performed for the image stitching processing result. Specifically, step 207 and step 208 are performed, or step 207 and step 209 are performed.

[0214] Step 207 , after completing image stitching based on N frames of images to be stitched, a target image is obtained.

[0215] The target image is an image obtained after stitching N frames of images to be stitched.

[0216] The target image can be generated by uniformly matching and stitching the N frames of images to be stitched together, or by stitching them together in a frame-by-frame manner as described in step 206 .

[0217] Figure 7 The figure is a schematic flow chart for stitching N frames of images to be stitched frame by frame.

[0218] In the frame-by-frame stitching method, the image stitching process can refer to the following Figure 7 Description of steps 301 to 304 in FIG.

[0219] Step 301: Use a second camera to acquire a first frame of image to be stitched.

[0220] The electronic device moves the motor to change the direction of the second camera, and obtains the first frame of the image to be spliced ​​through the second camera.

[0221] The electronic device moves the position of the motor from the initial position to the first position. Moving the motor can change the direction of the second camera and the shooting range of the second camera, so that the scene shot by the second camera changes. The scene corresponding to the first frame of the image to be spliced ​​obtained by the electronic device can be the aforementioned Figure 5b As shown in . The initial position of the motor can be located at the exact center, for example, Figure 5bArea 506 is the shooting range of the second camera when the motor is at the exact center, and area 501 is the shooting range of the second camera after the motor moves for the first time.

[0222] After the first frame of the image to be stitched is acquired by using the second camera, step 302 is executed.

[0223] In step 302, the electronic device moves a motor to change the direction of the second camera, and obtains the next frame of the image to be stitched through the second camera. Each time a frame of the image to be stitched is obtained by the second camera, the frame of the image to be stitched is stitched to obtain a stitched image.

[0224] In this process, after acquiring the first frame of the image to be stitched, the electronic device changes the direction of the second camera by moving the motor, and continues to acquire the next frame of the image to be stitched. Image stitching processing is performed once each time a frame of the image to be stitched is acquired to obtain a stitched image. The stitched image is stitched together with another frame of the image to be stitched, and this cycle is repeated continuously to stitch two frames of images together until all N frames of images are stitched together to finally obtain the target image.

[0225] An example of this process can be found in the second scenario described above. Figure 5a-5d The description content of the associated acquisition of N frames of images to be stitched is not repeated here.

[0226] In addition, in some embodiments, after the electronic device captures the first frame of the image to be stitched and the second frame of the image to be stitched through the second camera, one frame is used as the first image to be stitched and the other frame is used as the second image to be stitched, the first image to be stitched and the second image to be stitched are stitched to obtain the first stitched image, the first stitched image can be used as the first image to be stitched in the next stitching cycle, and the next frame of the image to be stitched is continuously acquired as the second image to be stitched, the stitching process of the first image to be stitched and the second image to be stitched is performed once, and the updated first image to be stitched is further obtained. The above process is continuously repeated until the N frames of images are stitched together to obtain the target image.

[0227] Among them, in order to ensure the image stitching effect of each frame of the image to be stitched during the stitching process, in each cycle of obtaining a frame of the image to be stitched using the second camera and stitching the frame of the image to be stitched to obtain a stitched image, the center position coordinates of the two frames of stitched images can be introduced each time the images are stitched.

[0228] Combination Figure 8 As shown, a one-cycle processing process of performing image stitching can refer to the following description of steps 401 to 406.

[0229] Step 401, after the first frame of the image to be stitched is acquired by the second camera, when the first image to be stitched is acquired by the second camera, the center position coordinates of the first image to be stitched and the second image to be stitched are acquired.

[0230] Here, when the electronic device uses the second camera to obtain N frames of images to be stitched, it will first obtain the first frame of images to be stitched, and then perform a cyclic image stitching process.

[0231] As a cyclic processing process, the first frame of the image to be stitched and the second frame of the image to be stitched may be stitched. In this case, the first frame of the image to be stitched is the second frame of the image to be stitched obtained by the second camera, and the second frame of the image to be stitched is the first frame of the image to be stitched.

[0232] As another cycle processing process, a new frame of image to be stitched may be stitched with the previously stitched image. In this case, the first image to be stitched is the new frame of image to be stitched obtained by the second camera, and the second image to be stitched is the previously stitched image.

[0233] Among them, since the N images to be stitched are all taken by the same camera (the second camera), and the stitching between images is also performed on the basis of the N images to be stitched, the center position coordinates of different images to be stitched can be determined in the camera coordinate system of the second camera.

[0234] Reference Figure 5a-5d As shown, in Figure 5a When the second camera is rotated to change the lens orientation, the center points in area 501 to area 504 correspond to the photographing center points of the second camera respectively.

[0235] but, Figure 5d In the images 1 to 4 captured, the center position of each image corresponds to the aforementioned photographing center point when the second camera takes photographs of different areas.

[0236] When performing image stitching, the center position coordinates of the two frames of stitched images are introduced each time the images are stitched, and the image alignment is performed using the center position coordinates, and the relative spatial relationship between the two images to be stitched is indicated to ensure the accuracy of the image stitching processing.

[0237] Step 402: determining an image stitching region between the first image to be stitched and the second image to be stitched based on the relative coordinates.

[0238] Taking area 501 and area 502 as an example, there is an overlapping area 505 between the two areas, and correspondingly there is also a corresponding overlapping area 507 between image 1 and image 2. When image 1 and image 2 are stitched together, first based on the center position coordinates of the two images, after image 1 and image 2 are aligned, combined with the image size, determine what the overlapping areas between the two images are, and use the overlapping areas between the images as the image stitching areas that need to be integrated for content.

[0239] Step 403: Detect and obtain image feature points from the image stitching area.

[0240] The feature points are points where the grayscale values ​​of the first image to be stitched and the second image to be stitched change dramatically or pixels with large curvature on the edge of the image. They have significant features and can reflect the essential characteristics of the image. The electronic device can use the scale-invariant feature transform (SIFT) algorithm or the speeded up robust feature (SURF) algorithm to calculate the feature points in the first image to be stitched and the second image to be stitched.

[0241] Step 404: determine the most similar M pairs of feature points from the image feature points.

[0242] The value range of M is a positive integer greater than or equal to 3, for example, M can be 3. Matching of feature points is matching of images. The electronic device can calculate the Euclidean distance between the feature points of the first image to be stitched and the second image to be stitched by the K-nearest neighbor algorithm. The smaller the Euclidean distance between any feature point of the second image to be stitched and any feature point of the first image to be stitched, the more similar the two feature points are.

[0243] In some embodiments, the electronic device may determine three pairs of feature points with the smallest Euclidean distance.

[0244] In other embodiments, the electronic device may set a preset threshold, and stop calculating when the calculated Euclidean distance of three pairs of feature points is less than the preset threshold, and regard the three pairs of feature points as the three most similar pairs of feature points.

[0245] It should be understood that, in addition to calculating the Euclidean distance between the feature points of the first image to be stitched and the second image to be stitched using the K-nearest neighbor algorithm, there may be other ways to determine the most similar M pairs of feature points, which is not limited in this embodiment of the present application.

[0246] Step 405: Calculate the affine transformation matrix of the first image to be stitched according to M pairs of feature points.

[0247] The affine transformation matrix is ​​used to perform a linear transformation and a translation on the second image to be stitched, and transform it into the vector space where the first image to be stitched is located.

[0248] Step 406 , performing affine transformation on the first image to be stitched according to the affine transformation matrix to obtain a third image to be stitched, and fusing the third image to be stitched with the second image to be stitched to obtain a stitched image.

[0249] The stitched image is then formed as the second image to be stitched in the next cycle, or is formed as the final target image.

[0250] The electronic device performs affine transformation on the second image to be stitched according to the image affine transformation matrix to obtain a third image to be stitched, and performs Laplace fusion on the third image to be stitched with the first image to be stitched to obtain a first stitched image.

[0251] The electronic device uses the radiation transformation matrix to perform an affine transformation on the second image to be stitched to obtain a third image to be stitched. The third image to be stitched is in the same vector space as the first image to be stitched. Then the electronic device performs Laplace fusion on the third image to be stitched and the first image to be stitched to obtain a first stitched image. The first stitched image obtained by the fusion is used as the updated first image to be stitched.

[0252] It should be understood that, in addition to the Laplace fusion algorithm, the electronic device may also use other methods to fuse the first image to be stitched and the third image to be stitched, which is not limited in the present embodiment of the application.

[0253] It should be understood that in addition to the above-mentioned stitching method, in other embodiments, there are other ways to stitch the N frames of images, for example, the N frames of images are divided into two groups, and the above-mentioned methods are used to stitch them respectively to obtain two frames of images, and then the two frames of images are stitched to obtain the target image. The embodiment of the present application does not limit the way in which the electronic device stitches the target image.

[0254] After a frame of the image to be stitched is acquired by using the second camera, the current frame of the image to be stitched is stitched to obtain a stitched image, and then step 303 is continued.

[0255] Step 303: determine whether the acquired images to be stitched have reached N frames.

[0256] That is, it is determined whether the second camera has acquired the Nth frame of the image to be stitched. If it is determined that the Nth frame has been acquired, step 304 is executed to stitch together the final target image.

[0257] Otherwise, it is necessary to enter the image stitching loop process and return to execute step 302. Step 302 can be looped N-1 times until the Nth frame of the image to be stitched is obtained.

[0258] Step 304 , stitching the Nth frame of images to be stitched together to obtain a target image.

[0259] In the above steps, after the electronic device obtains the first frame of the image to be stitched using the second camera, each time it obtains a frame of the image to be stitched using the second camera, it stitches the frame of the image to be stitched to obtain a stitched image, until the N-th frame of the image to be stitched is obtained using the second camera, and the N-th frame of the image to be stitched is stitched to obtain the target image, thus completing the frame-by-frame image stitching of all the N frames of the image to be stitched.

[0260] The above process realizes splicing the target image based on N frames of images to be spliced, and step 207 is completed.

[0261] After executing step 207 , in one embodiment, step 208 may be executed.

[0262] Step 208: Display the target image as a stitching preview image in the stitching preview window.

[0263] Alternatively, after executing step 207, in another implementation, step 209 may be executed.

[0264] Step 209 , cropping the target image to obtain a third image, and displaying the third image as a stitching preview image in the stitching preview window.

[0265] In these two embodiments, the target image is used as a stitching preview image, or the target image is used as a stitching preview image after cropping and displayed in a stitching preview window, and different image processing methods are used to achieve a fast preview display of the image that presents multiple scenes and has a high resolution after stitching.

[0266] The third scene: based on the image captured by the second camera, the image stitching is previewed and displayed in combination with the camera movement trajectory.

[0267] The second camera provided in the electronic device is a rotatable camera. Then, in the process of acquiring N images to be stitched through the second camera, the camera movement trajectory in the process of acquiring images can be set.

[0268] The electronic device controls the second camera to rotate along one or several paths according to the set lens movement trajectory to adjust the lens direction.

[0269] The third scenario here is based on the second implementation method in the aforementioned second scenario, that is, based on the preview implementation method of preview display based on the stitching process of image stitching, the camera movement trajectory is integrated into the graphic preview, and the camera movement process of dynamic image acquisition is combined with the dynamic stitching preview display of the image, so as to realize the dynamic preview display of image stitching while improving the realism of the image acquisition and stitching preview display.

[0270] The electronic device needs to first combine the mirror movement trajectory and collect N frames of images to be stitched through the second camera.

[0271] Refer to the above Figure 3a After the electronic device enters the "telephoto stitching mode" of the camera function, it can continue to use the first camera to obtain a camera preview image and display the preview image in the preview box 301 of the user interface. In response to the user's photo triggering operation on the shooting control 302 in the camera preview interface, refer to the aforementioned Figure 3b In the schematic diagram of the user interface 31 shown, the electronic device calls up a stitching preview window 311 and synchronously performs a photo operation, specifically, the electronic device drives a motor to change the direction of the second camera according to a set mirror movement trajectory, and uses the N photo points of the second camera in the mirror movement trajectory to obtain N frames of images to be stitched.

[0272] The camera trajectory will contain multiple shooting points. The number of shooting points is the same as the number of images to be stitched. One shooting point is used to obtain one frame of image to be stitched using the second camera. If N frames of images to be stitched need to be taken, N shooting points are set in the camera trajectory.

[0273] The electronic device can drive the second camera to move between N photo points along a preset trajectory while the position remains unchanged. Each time the motor drives the second camera to move, the direction of the second camera changes, thereby realizing the camera movement, and the scene that can be photographed by the second camera changes. When the second camera after the direction change is at a set photo point, the second camera can photograph the changed scene at the current photo point.

[0274] In this way, the second camera can capture a frame of image to be processed each time it moves to a photographing point along the camera movement trajectory, and finally obtain N frames of image to be processed at N photographing points in the camera movement trajectory.

[0275] See also Figure 2bAs shown, a camera movement track can be shown by arrows in the figure, and the camera movement track includes 4 shooting points. The second camera can first shoot the scene in area 201 according to the 4 shooting points included in the camera movement track, and then shoot the scenes in area 202, area 203 and area 204 in sequence, to obtain a total of 4 frames of images to be spliced, and then the electronic device can splice the 4 frames of images to be spliced.

[0276] In the "telephoto stitching mode", the camera movement trajectory can be designed and generated in advance in the mode, or set and generated based on the user's current shooting needs.

[0277] Figure 8a-8f This is a set of schematic diagrams for determining the camera trajectory and performing image stitching preview based on the camera trajectory in the third scene.

[0278] The first camera in the electronic device has a larger field of view than the second camera. In order to ensure that the image captured by the second camera can cover as much of the field of view of the first camera as possible, the field of view shooting coverage of the first camera with a larger field of view can be used to formulate the camera movement trajectory of the second camera.

[0279] Here, the camera movement trajectory of the second camera can be determined based on the field of view of the first camera.

[0280] like Figure 8a-8c As shown, the electronic device uses the first camera to obtain the camera preview image, and the field of view of the first camera is a fixed parameter, so the area can be divided based on the range covered by the field of view of the first camera. And the mirror movement trajectory of the second camera is planned based on the divided areas.

[0281] The number of divided areas is the same as the number of images to be stitched taken by the second camera. Assuming that at this time, the electronic device determines that N is 9 based on the field of view of the wide-angle camera and the field of view of the telephoto camera, 9 areas are divided from the range covered by the field of view of the first camera. Figure 8a-8c In the 9-area image, the range covered by the field of view of the first camera is divided into area 801 to area 809, a total of 9 areas, each of which covers different scene contents. Based on the 9 divided areas, point 1 to point 9 corresponding to the field of view of the first camera can be planned, a total of 9 photo points. The photo points can be previewed in the stitching preview area 810. Figure 8a-8c In the stitching preview area 810 , photographing points 1 to 9 are shown, and different camera movement routes are planned between photographing points 1 to 9.

[0282] A moving trajectory is planned among the nine photographing points, so that the electronic device controls the second camera to move to the corresponding photographing point in the moving trajectory, and the scenes of area 801-area 809 are photographed at the nine photographing points, covering the field of view angle coverage of the first camera as much as possible.

[0283] For the planning of the movement sequence between the photographing points, as shown in 8a, you can choose to start from the photographing point 1 corresponding to the upper left area 801 of the shooting area to the photographing point 9 corresponding to the lower right area 809, and use point 1 as the starting point A for photographing, and use point 9 as the ending point B for photographing, and obtain the movement trajectory between point 1 and point 9: point 1-point 2-point 3-point 6-point 5-point 4-point 7-point 8-point 9, forming a camera movement trajectory.

[0284] Alternatively, you can also choose to start from the photo point 3 corresponding to the upper right area 803 of the shooting area and end at the photo point 7 corresponding to the lower left area 807 to obtain the movement trajectory between point 3 and point 7 to form a camera movement trajectory, which will not be repeated here.

[0285] In the electronic device, the rotatable second camera is located at the center of the movable range before and after shooting. That is, during the lens rotation shooting process, the motor drives the camera to start from the center position and finally return to the center position.

[0286] In order to ensure the rationality of the movement of the second camera lens, the 9 areas covered by the field of view of the first camera can be referred to, and the inner starting point of the shooting and the outer ending point of the shooting can be selected from the shooting points 1 to 9.

[0287] The inner starting point of the photo shooting is the center of the movable range of the second camera, or a point closer to the center of the movable range of the second camera. The outer ending point of the photo shooting is a point farther from the center of the movable range of the second camera than the starting point of the photo shooting.

[0288] This method of point selection ensures that the camera movement trajectory starts from the center position of the movable range of the lens as much as possible, eliminating the time required for the movable second camera to move from the center position to the shooting starting point, achieving fast image output, reducing unnecessary movement distance, and improving the stability of the captured image.

[0289] Moreover, in order to further ensure the rationality of the movement of the second camera lens, when selecting the outer end point of shooting, the distance between the end point of shooting and the start point of shooting can also be taken into consideration, and the distance between the selected end point of shooting and the start point of shooting needs to be less than a threshold.

[0290] In this way, it can be ensured that the end point of shooting is as inner as possible, reducing the movement time of the camera from the end point of shooting to the center position of the movable range of the lens, achieving fast image output, and further reducing unnecessary movement distance, thereby improving the stability of the captured image.

[0291] exist Figure 8b In the figure, the inner photo point 5 is selected as the starting point A for photo shooting, the outer photo point 1 is selected as the ending point B for photo shooting, and the photo shooting movement sequence between the remaining 7 middle photo points is planned to obtain the movement trajectory between point 5 and point 1: point 5-point 2-point 3-point 6-point 9-point 8-point 7-point 4-point 1, forming a camera movement trajectory.

[0292] exist Figure 8c In the example, the inner photo point 5 is selected as the starting point A, the outer photo point 2 is selected as the ending point B, and the photo movement sequence between the remaining 7 middle photo points is planned to obtain the movement trajectory between point 5 and point 2: point 5-point 3-point 6-point 9-point 8-point 7-point 4-point 1-point 2, forming a camera movement trajectory.

[0293] After the electronic device enters the "telephoto stitching mode" of the camera function, in response to the user's photo triggering operation, the driving motor changes the direction of the second camera according to the set mirror movement trajectory, and uses the N photo points of the second camera in the mirror movement trajectory to obtain N frames of images to be stitched. In the process of obtaining the N frames of images to be stitched, refer to the above Figure 3b In the schematic diagram of the user interface 31 shown, the electronic device calls out a stitching preview window 311 to display a preview of the image stitching.

[0294] In this embodiment, a display grid can be set in the stitching preview window of the camera preview interface. The display grid can be in the form of a chessboard or a diamond grid.

[0295] Specifically, N grid positions need to be set in the display grid, and the number of grid positions needs to be consistent with the number of images to be stitched that the second camera needs to capture, so that the stitched image formed by stitching the images to be stitched can be previewed in the corresponding grid positions. Each grid position corresponds to a photographing point in the camera movement track of the second camera, so that when the stitched image is previewed, it can be associated with the photographing point and the camera movement track for previewing.

[0296] exist Figure 8b Based on the camera trajectory given in Figure 8d-8f A preview display image of the spliced ​​image in the display grid is shown in FIG.

[0297] Figure 8d-Figure 8fIn the splicing preview window 810, the display grid is displayed. The display grid in the splicing preview window 810 includes 9 grid positions. Each grid position corresponds to a shooting point in the camera movement track of the second camera. Among them, grid position 5 corresponds to the shooting start point A in the camera movement track, grid position 2 corresponds to the second shooting point in the camera movement track, grid position 3 corresponds to the third shooting point in the camera movement track, and grid position 1 corresponds to the shooting end point B in the camera movement track. The corresponding relationship between other grid positions and shooting points can be seen in the figure, and will not be explained one by one.

[0298] When the electronic device uses the second camera to obtain N frames of images to be stitched at N shooting points in the camera movement trajectory, it is necessary to preview and display the images at corresponding grids in the stitching preview window 810 .

[0299] in, Figure 8d In the embodiment, the electronic device controls the second camera to capture the image of area 805 along the shooting starting point in the camera movement trajectory to obtain the first frame of the image to be stitched, and the shooting starting point A of area 805 corresponds to the grid position 5 in the stitching preview window 810. Therefore, the first frame of the image to be stitched needs to be displayed on the grid position 5 in the stitching preview window 810.

[0300] Then, Figure 8e In the embodiment, the electronic device controls the second camera to move along the mirror movement trajectory, and adjusts the shooting starting point to the second shooting point to shoot the image of area 802 to obtain a second frame of the image to be spliced, and splices the first frame of the image to be spliced ​​with the second frame of the image to be spliced ​​to obtain a first spliced ​​image. The shooting point of area 802 corresponds to the grid position 2 in the splicing preview window 810, so the first spliced ​​image corresponds to two grid positions, namely, grid position 5 and grid position 2 (to show the preview display effect, Figure 8e The grid numbers are not shown in the figure. The numbers can be found in Figure 8d Therefore, the first frame of the image to be stitched that has been displayed in the grid plate 5 needs to be overwritten by the current first stitched image, and the first stitched image is displayed on the grid position 5 and the grid position 2 in the stitching preview window 810.

[0301] Further, Figure 8f In the embodiment, the electronic device controls the second camera to continue to move along the camera movement trajectory, and adjusts from the second photographing point to the third photographing point to photograph the area 803 to obtain a third frame of image to be spliced, and splices the third frame of image to be spliced ​​with the first spliced ​​image to obtain a second spliced ​​image. The photographing point of the area 803 corresponds to the grid position 3 in the splicing preview window 810, so the second spliced ​​image corresponds to three grid positions, namely, grid position 5, grid position 2 and grid position 3 (to show the preview display effect, Figure 8e The grid numbers are not shown in the figure. The numbers can be found in Figure 8dTherefore, the current second stitched image needs to overwrite the first image to be stitched that has been displayed in grid plate 5 and grid plate 2, and the second stitched image is displayed on grid position 5, grid position 2 and grid position 3 in the stitching preview window 810.

[0302] The electronic device subsequently controls the second camera to continue moving along the mirror trajectory, adjusts the photo-taking point, shoots the image to be stitched, and displays the stitched image on the corresponding grid position of the grid display in the stitching preview window. The processing process can be referred to the above description until the stitching and preview of N frames of images to be stitched are completed.

[0303] Fig. 9 It is a schematic flowchart of the shooting method involved in the third scene.

[0304] In the third scenario, the process of the electronic device acquiring the target image and image preview using the shooting method in the embodiment of the present application can refer to the following description of steps 501 to 506.

[0305] Step 501, selecting an inner starting point for photographing and an outer ending point for photographing from the field of view of a first camera.

[0306] The inner starting point of the photo shooting is the center of the movable range of the second camera, or a point closer to the center of the movable range of the second camera. The outer ending point of the photo shooting is a point farther from the center of the movable range of the second camera than the starting point of the photo shooting.

[0307] This method of point selection ensures that the camera movement trajectory starts from the center position of the movable range of the lens as much as possible, eliminating the time required for the movable second camera to move from the center position to the shooting starting point, achieving fast image output, reducing unnecessary movement distance, and improving the stability of the captured image.

[0308] In order to further ensure the rationality of the movement of the second camera lens, when selecting the outer end point of shooting, the distance between the end point of shooting and the start point of shooting can also be taken into consideration.

[0309] A first point located innermost from the field of view of the first camera is selected as the starting point for photographing, and a second point located outermost from the first point and having a distance less than a threshold from the field of view of the first camera is selected as the ending point for photographing, so that the distance between the selected ending point for photographing and the starting point for photographing is less than the threshold.

[0310] In this way, it can be ensured that the end point of shooting is as inner as possible, reducing the movement time of the camera from the end point of shooting to the center position of the movable range of the lens, achieving fast image output, and further reducing unnecessary movement distance, thereby improving the stability of the captured image.

[0311] Step 502: Based on the starting point and the ending point of the photographing, determine the photographing movement sequence between N-2 intermediate photographing points in the field of view.

[0312] Step 503, according to the photographing movement sequence, determine the target movement trajectory starting from the photographing start point, passing through N-2 intermediate photographing points, and reaching the photographing end point as the set camera movement trajectory.

[0313] Therefore, the lens movement trajectory of the second camera is formulated by utilizing the field of view shooting coverage of the first camera having a larger field of view, thereby ensuring image acquisition and telephoto stitching image effects of the second camera.

[0314] Step 504, in response to the photo triggering operation, the driving motor changes the direction of the second camera according to the set camera trajectory, and uses the N photo taking points of the second camera in the camera trajectory to obtain N frames of images to be stitched.

[0315] The camera movement track will contain multiple shooting points. If you need to shoot N frames of images to be stitched, then set N shooting points in the camera movement track.

[0316] After the camera movement trajectory is determined, the second camera needs to shoot N frames of images to be stitched according to the camera movement trajectory.

[0317] The electronic device drives the motor to drive the second camera to move between N photographing points along a preset trajectory, and each time the motor drives the second camera to move, the scene that the second camera can photograph changes. In this way, the second camera can capture a frame of image to be processed each time it moves to a photographing point along the camera movement trajectory, and finally obtains N frames of image to be processed at the N photographing points in the camera movement trajectory.

[0318] Step 505 , determining, from the display grid set in the stitching preview window, at least one target grid corresponding to each stitched image obtained by stitching one frame of the images to be stitched in the process of stitching N frames of the images to be stitched to obtain the icon image.

[0319] Among them, a display grid is set in the stitching preview window, which includes N grid positions, each grid position corresponds to a photographing point in the camera movement track of the second camera, and a photographing point is used to obtain a frame of image to be stitched using the second camera.

[0320] Step 506 , sequentially overlaying and displaying the stitched images on at least one target grid in the stitching preview window.

[0321] In the process of stitching N frames of images to be stitched to obtain an icon image, each stitched image obtained by stitching one frame of images to be stitched is displayed as a stitching preview image in the stitching preview window, thereby realizing a dynamic preview display of the image stitching process.

[0322] This process integrates the camera movement trajectory into the graphic preview, and combines the camera movement process of dynamic image acquisition with the dynamic image stitching preview display. While realizing the dynamic preview display of image stitching, it improves the realism of the image acquisition and stitching preview display.

[0323] In the embodiments of the present application, there are some similarities and differences in the image capture and image preview processing in different scenarios. The relevant description contents of these parts can be borrowed and integrated with each other, and there are no obstacles to the implementation of technical combination due to differences in scene descriptions.

[0324] Fig.10 Schematic diagram of an image capturing device according to an embodiment of the present application. Fig.10 As shown, the device 2000 includes a mode trigger module 2001, a first shooting module 2002, a second shooting module 2003, an image processing module 2004 and an image preview module 2005. The device 2000 can be integrated in electronic devices such as mobile phones, tablet computers, and smart wearable devices.

[0325] The device 2000 can be used to execute any one of the above image capturing methods.

[0326] In one implementation, the apparatus 2000 may further include a storage unit for storing data such as images, recognized image features, etc. The storage unit may be integrated in any one of the above units, or may be a unit independent of all the above units.

[0327] Fig.11 Schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. Fig.11As shown, the electronic device 900 may include a processor 910, an external memory interface 920, an internal memory 921, a universal serial bus (USB) interface 930, a charging management module 940, a power management module 941, a battery 942, an antenna 1, an antenna 2, a mobile communication module 950, a wireless communication module 960, an audio module 970, a speaker 970A, a receiver 970B, a microphone 970C, an earphone interface 970D, a sensor module 980, a button 990, a motor 991, an indicator 992, a camera 993, a display screen 994, and a subscriber identification module (SIM) card interface 995, etc. Among them, the sensor module 980 may include a pressure sensor 980A, a gyroscope sensor 980B, an air pressure sensor 980C, a magnetic sensor 980D, an acceleration sensor 980E, a distance sensor 980F, a proximity light sensor 980G, a fingerprint sensor 980H, a temperature sensor 980J, a touch sensor 980K, an ambient light sensor 980L, a bone conduction sensor 980M, etc.

[0328] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 900. In other embodiments of the present application, the electronic device 900 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0329] For example, Fig.11 The processor 910 shown may include one or more processing units, for example, the processor 910 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0330] The controller may be the nerve center and command center of the electronic device 900. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0331] The processor 910 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 910 is a cache memory. The memory may store instructions or data that the processor 910 has just used or cyclically used. If the processor 910 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 910, and thus improves the efficiency of the system.

[0332] In some embodiments, the processor 910 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0333] In some embodiments, the I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). The processor 910 may include multiple groups of I2C buses. The processor 910 may be coupled to the touch sensor 980K, the charger, the flash, the camera 993, etc. through different I2C bus interfaces. For example, the processor 910 may be coupled to the touch sensor 980K through the I2C interface, so that the processor 910 communicates with the touch sensor 980K through the I2C bus interface, thereby realizing the touch function of the electronic device 900.

[0334] In some embodiments, the I2S interface can be used for audio communication. The processor 910 can include multiple I2S buses. The processor 910 can be coupled to the audio module 970 via the I2S bus to achieve communication between the processor 910 and the audio module 970.

[0335] In some embodiments, the audio module 970 can transmit audio signals to the wireless communication module 960 through the I2S interface to implement the function of answering calls through a Bluetooth headset.

[0336] In some embodiments, the PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. The audio module 970 and the wireless communication module 960 can be coupled via a PCM bus interface.

[0337] In some embodiments, the audio module 970 can also transmit audio signals to the wireless communication module 960 through the PCM interface to implement the function of answering calls through a Bluetooth headset. It should be understood that both the I2S interface and the PCM interface can be used for audio communication.

[0338] In some embodiments, the UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. The UART interface is generally used to connect the processor 910 and the wireless communication module 960. For example, the processor 910 communicates with the Bluetooth module in the wireless communication module 960 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 970 can transmit an audio signal to the wireless communication module 960 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0339] In some embodiments, the MIPI interface can be used to connect the processor 910 with peripheral devices such as the display screen 994 and the camera 993. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. The processor 910 and the camera 993 communicate through the CSI interface to realize the shooting function of the electronic device 900. The processor 910 and the display screen 994 communicate through the DSI interface to realize the display function of the electronic device 900.

[0340] In some embodiments, the GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. The GPIO interface can be used to connect the processor 910 with the camera 993, the display screen 994, the wireless communication module 960, the audio module 970, the sensor module 980, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0341] Exemplarily, the USB interface 930 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 900, and can also be used to transfer data between the electronic device 900 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0342] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 900. In other embodiments of the present application, the electronic device 900 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0343] The charging management module 940 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 940 may receive charging input from a wired charger through the USB interface 930. In some wireless charging embodiments, the charging management module 940 may receive wireless charging input through a wireless charging coil of the electronic device 900. While the charging management module 940 is charging the battery 942, it may also power the electronic device through the power management module 941.

[0344] The power management module 941 is used to connect the battery 942, the charging management module 940 and the processor 910. The power management module 941 receives input from the battery 942 and / or the charging management module 940, and supplies power to the processor 910, the internal memory 921, the external memory, the display screen 994, the camera 993, and the wireless communication module 960. The power management module 941 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 941 can also be set in the processor 910. In other embodiments, the power management module 941 and the charging management module 940 can also be set in the same device.

[0345] The wireless communication function of the electronic device 900 can be implemented through antenna 1, antenna 2, mobile communication module 950, wireless communication module 960, modem processor and baseband processor.

[0346] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 900 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0347] The mobile communication module 950 can provide a wireless communication solution applied to the electronic device 900, such as at least one of the following solutions: a second generation (2G) mobile communication solution, a third generation (3G) mobile communication solution, a fourth generation (5G) mobile communication solution, and a fifth generation (5G) mobile communication solution. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 950 may receive electromagnetic waves from the antenna 1, and perform processing such as filtering and amplifying the received electromagnetic waves, and then transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 950 may also amplify the signal modulated by the modulation and demodulation processor, and the amplified signal is converted into an electromagnetic wave and radiated out through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be arranged in the processor 910. In some embodiments, at least some of the functional modules of the mobile communication module 950 may be arranged in the same device as at least some of the modules of the processor 910.

[0348] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 970A, a receiver 970B, etc.), or displays an image or video through a display screen 994. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 910 and may be set in the same device as the mobile communication module 950 or other functional modules.

[0349] The wireless communication module 960 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 900. The wireless communication module 960 can be one or more devices integrating at least one communication processing module. The wireless communication module 960 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 910. The wireless communication module 960 can also receive the signal to be sent from the processor 910, modulate the frequency of the signal, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0350] In some embodiments, the antenna 1 of the electronic device 900 is coupled to the mobile communication module 950, and the antenna 2 of the electronic device 900 is coupled to the wireless communication module 960, so that the electronic device 900 can communicate with the network and other electronic devices through wireless communication technology. The wireless communication technology may include at least one of the following communication technologies: global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, IR technology. The GNSS may include at least one of the following positioning technologies: global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and satellite based augmentation systems (SBAS).

[0351] The electronic device 900 implements the display function through a GPU, a display screen 994, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 994 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 910 may include one or more GPUs, which execute program instructions to generate or change display information.

[0352] The display screen 994 is used to display images, videos, etc. The display screen 994 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini-LED, Micro-Led, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 900 may include 1 or N display screens 994, where N is a positive integer greater than 1.

[0353] The electronic device 900 can realize the shooting function through an ISP, a camera 993, a video codec, a GPU, a display screen 994, and an application processor.

[0354] ISP is used to process the data fed back by camera 993. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 993.

[0355] The camera 993 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 900 may include 1 or N cameras 993, where N is a positive integer greater than 1.

[0356] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 900 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0357] Video codecs are used to compress or decompress digital videos. The electronic device 900 may support one or more video codecs. Thus, the electronic device 900 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0358] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 900 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0359] The external memory interface 920 can be used to connect an external memory card, such as a secure digital (SD) card, to expand the storage capacity of the electronic device 900. The external memory card communicates with the processor 910 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0360] The internal memory 921 can be used to store computer executable program codes, which include instructions. The processor 910 executes various functional applications and data processing of the electronic device 900 by running the instructions stored in the internal memory 921. The internal memory 921 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 900 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 921 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0361] The electronic device 900 can implement audio functions such as music playing and recording through the audio module 970, the speaker 970A, the receiver 970B, the microphone 970C, the headphone jack 970D, and the application processor.

[0362] The audio module 970 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 970 can also be used to encode and decode audio signals. In some embodiments, the audio module 970 can be arranged in the processor 910, or some functional modules of the audio module 970 can be arranged in the processor 910.

[0363] The speaker 970A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 900 can listen to music or listen to a hands-free call through the speaker 970A.

[0364] The receiver 970B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 900 receives a call or voice message, the voice can be received by placing the receiver 970B close to the human ear.

[0365] Microphone 970C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to the microphone 970C to input the sound signal into the microphone 970C. The electronic device 900 can be provided with at least one microphone 970C. In other embodiments, the electronic device 900 can be provided with two microphones 970C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 900 can also be provided with three, four or more microphones 970C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0366] The earphone interface 970D is used to connect a wired earphone and can be the USB interface 930 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0367] The pressure sensor 980A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 980A can be set on the display screen 994. There are many types of pressure sensors 980A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 980A, the capacitance between the electrodes changes. The electronic device 900 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 994, the electronic device 900 detects the touch operation intensity according to the pressure sensor 980A. The electronic device 900 can also calculate the touch position according to the detection signal of the pressure sensor 980A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

[0368] The gyroscope sensor 980B can be used to determine the motion posture of the electronic device 900. In some embodiments, the angular velocity of the electronic device 900 around three axes (i.e., the x, y, and z axes) can be determined by the gyroscope sensor 980B. The gyroscope sensor 980B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 980B detects the angle of the electronic device 900 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 900 through reverse movement to achieve anti-shake. The gyroscope sensor 980B can also be used for navigation and somatosensory game scenes.

[0369] The air pressure sensor 980C is used to measure air pressure. In some embodiments, the electronic device 900 calculates the altitude through the air pressure value measured by the air pressure sensor 980C to assist in positioning and navigation.

[0370] The magnetic sensor 980D includes a Hall sensor. The electronic device 900 can use the magnetic sensor 980D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 900 is a flip phone, the electronic device 900 can detect the opening and closing of the flip cover according to the magnetic sensor 980D; according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, set the flip cover to automatically unlock and other features.

[0371] The acceleration sensor 980E can detect the magnitude of the acceleration of the electronic device 900 in various directions (generally three axes). When the electronic device 900 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0372] The distance sensor 980F is used to measure the distance. The electronic device 900 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 900 can use the distance sensor 980F to measure the distance to achieve fast focusing.

[0373] The proximity light sensor 980G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The light-emitting diode may be an infrared light-emitting diode. The electronic device 900 emits infrared light outward through the light-emitting diode. The electronic device 900 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 900. When insufficient reflected light is detected, the electronic device 900 can determine that there is no object near the electronic device 900. The electronic device 900 can use the proximity light sensor 980G to detect that the user holds the electronic device 900 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 980G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.

[0374] The ambient light sensor 980L is used to sense the ambient light brightness. The electronic device 900 can adaptively adjust the brightness of the display screen 994 according to the perceived ambient light brightness. The ambient light sensor 980L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 980L can also cooperate with the proximity light sensor 980G to detect whether the electronic device 900 is in a pocket to prevent accidental touches.

[0375] The fingerprint sensor 980H is used to collect fingerprints. The electronic device 900 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.

[0376] The temperature sensor 980J is used to detect temperature. In some embodiments, the electronic device 900 uses the temperature detected by the temperature sensor 980J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 980J exceeds a threshold, the electronic device 900 reduces the performance of the processor located near the temperature sensor 980J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 900 heats the battery 942 to avoid abnormal shutdown of the electronic device 900 due to low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 900 performs a boost on the output voltage of the battery 942 to avoid abnormal shutdown caused by low temperature.

[0377] The touch sensor 980K is also called a "touch panel". The touch sensor 980K can be set on the display screen 994, and the touch sensor 980K and the display screen 994 form a touch screen, also called a "touch screen". The touch sensor 980K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 994. In other embodiments, the touch sensor 980K can also be set on the surface of the electronic device 900, which is different from the position of the display screen 994.

[0378] The bone conduction sensor 980M can obtain a vibration signal. In some embodiments, the bone conduction sensor 980M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 980M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 980M can also be set in an earphone and combined into a bone conduction earphone. The audio module 970 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 980M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 980M to realize a heart rate detection function.

[0379] The key 990 includes a power key, a volume key, etc. The key 990 may be a mechanical key or a touch key. The electronic device 900 may receive key input and generate key signal input related to user settings and function control of the electronic device 900.

[0380] Motor 991 can generate vibration prompts. Motor 991 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 994, motor 991 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0381] Indicator 992 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0382] The SIM card interface 995 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 900 by inserting it into the SIM card interface 995 or pulling it out from the SIM card interface 995. The electronic device 900 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 995 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 995 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 995 can also be compatible with different types of SIM cards. The SIM card interface 995 can also be compatible with external memory cards. The electronic device 900 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 900 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 900 and cannot be separated from the electronic device 900.

[0383] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0384] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0385] An embodiment of the present application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor can implement the steps in any of the above methods when executing the computer program.

[0386] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0387] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0388] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / electronic device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0389] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0390] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0391] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0392] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0393] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0394] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0395] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0396] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0397] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0398] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An image capturing method, applied to an electronic device including a first camera and a second camera, characterized in that: The field of view of the first camera is greater than the field of view of the second camera, and the method includes: The electronic device enters a camera preview interface, wherein the camera preview interface includes a splicing preview window; a display grid is provided in the splicing preview window, wherein the display grid includes N grid positions; and the splicing preview window is provided on a side of the camera preview interface; In response to a photo triggering operation, acquiring a first image using the first camera; In response to the photo triggering operation, changing the orientation of the second camera, and acquiring N frames of images to be stitched using the second camera, where N is an integer greater than 2; Performing image stitching on the N frames of images to be stitched to obtain a target image; A stitching preview image is displayed in the stitching preview window, the stitching preview image corresponds to the target image, the image to be stitched, or the first image; the stitching preview image includes a second image cropped from the first image based on a starting position coordinate and an ending position coordinate, the starting position coordinate is , the cut-off position coordinates are ; wherein said is the zoom ratio of the field of view angle corresponding to the second rotatable camera when the lens is rotated to the maximum angle relative to the field of view angle of the first camera; is the width of the first image, the is the height of the first image; the zoom ratio is calculated based on the field of view corresponding to the second rotatable camera when the lens is rotated to the maximum angle, the diagonal length of the image sensor in the second camera, and the focal length of the first camera; Displaying the stitching preview image in the stitching preview window includes: Determining, from the display grid, at least one target grid position corresponding to each stitched image obtained by stitching one frame of the images to be stitched in a process of stitching the N frames of the images to be stitched to obtain the target image; The stitched images are sequentially covered and displayed on the at least one target grid in the stitching preview window.

2. The method according to claim 1, characterized in that The camera preview interface includes an image preview area; After the electronic device enters the camera preview interface, the electronic device uses the first camera to obtain a camera preview image; Displaying the camera preview image in the image preview area; The first image corresponds to the camera preview image displayed in the image preview area at the triggering moment of the photographing triggering operation; The stitching preview window is located in the image preview area.

3. The method according to claim 1, characterized in that In response to the photo triggering operation, changing the direction of the second camera and acquiring N frames of images to be stitched by using the second camera includes: In response to the photographing triggering operation, the driving motor changes the orientation of the second camera according to the set mirror movement trajectory, and uses the N photographing points of the second camera in the mirror movement trajectory to obtain N frames of images to be stitched.

4. The method according to claim 3, characterized in that The method further comprises: Selecting an inner starting point for photographing and an outer ending point for photographing from the field of view of the first camera; Based on the photographing start point and the photographing end point, determining a photographing movement sequence between N-2 intermediate photographing points in the field of view; According to the photographing movement sequence, a target movement trajectory starting from the photographing starting point, passing through N-2 intermediate photographing points, and reaching the photographing ending point is determined as the set camera movement trajectory.

5. The method according to claim 4, characterized in that The selecting an inner starting point for photographing and an outer ending point for photographing from the field of view of the first camera includes: Selecting a first inner point from the field of view of the first camera as the starting point for photographing; A second point whose distance from the first point is less than a threshold and is outside the first point is selected from the field of view of the first camera as the photographing end point.

6. The method according to any one of claims 1 to 5, characterized in that: Displaying the stitching preview image in the stitching preview window includes: Displaying the target image as the stitching preview image in the stitching preview window; or, The target image is cropped to obtain a third image, and the third image is displayed in the splicing preview window as the splicing preview image.

7. The method according to claim 1, characterized in that Each of the grid positions corresponds to a photographing point in the camera movement track of the second camera, and one of the photographing points is used to obtain a frame of the image to be spliced ​​using the second camera.

8. The method according to claim 1, characterized in that The step of performing image stitching on the N frames of images to be stitched to obtain a target image includes: After the first frame of the image to be stitched is acquired by the second camera, each time a frame of the image to be stitched is acquired by the second camera, stitching the frame of the image to be stitched is performed to obtain a stitched image; Until the Nth frame of the image to be stitched is acquired by the second camera, the Nth frame of the image to be stitched is stitched to obtain the target image.

9. The method according to claim 8, characterized in that The method of acquiring a frame of the image to be stitched by the second camera each time and stitching the frame of the image to be stitched to obtain a stitched image comprises: When the second camera is used to obtain a first image to be stitched, obtaining the center position coordinates of the first image to be stitched and a second image to be stitched, where the second image to be stitched is the first frame of the image to be stitched or a stitched image; Based on the center position coordinates, determining an image stitching region between the first image to be stitched and the second image to be stitched; Detecting and obtaining image feature points from the image stitching area; Determine the most similar M pairs of feature points from the image feature points; M is an integer greater than or equal to 3; Calculating an affine transformation matrix of the first image to be stitched according to the M pairs of feature points; Performing an affine transformation on the first image to be stitched according to the affine transformation matrix to obtain a third image to be stitched; The third image to be stitched is fused with the second image to be stitched to obtain the stitched image.

10. An image capture device, comprising a first camera and a second camera, characterized in that: The field of view of the first camera is greater than the field of view of the second camera, and the device includes: A mode trigger module, used for entering a camera preview interface, wherein the camera preview interface includes a splicing preview window; a display grid is provided in the splicing preview window, and the display grid includes N grid positions; the splicing preview window is provided on the side of the camera preview interface; A first shooting module, configured to acquire a first image using the first camera in response to a photo triggering operation; A second shooting module, configured to change the orientation of the second camera in response to the photo triggering operation, and obtain N frames of images to be stitched using the second camera, where N is an integer greater than 2; An image processing module is used to stitch the N frames of images to be stitched together to obtain a target image; An image preview module is used to display a splicing preview image in the splicing preview window, wherein the splicing preview image corresponds to the target image, the image to be spliced, or the first image; the splicing preview image includes a second image cropped from the first image based on a starting position coordinate and an ending position coordinate, wherein the starting position coordinate is ( ), the cut-off position coordinates are ( ); wherein, the is the zoom ratio of the field of view angle corresponding to the second rotatable camera when the lens is rotated to the maximum angle relative to the field of view angle of the first camera; is the width of the first image, the is the height of the first image; the zoom ratio is calculated based on the field of view corresponding to the second rotatable camera when the lens is rotated to the maximum angle, the diagonal length of the image sensor in the second camera, and the focal length of the first camera; The image preview module is specifically used for: Determining, from the display grid, at least one target grid position corresponding to each stitched image obtained by stitching one frame of the images to be stitched in a process of stitching the N frames of the images to be stitched to obtain the target image; The stitched images are sequentially covered and displayed on the at least one target grid in the stitching preview window.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images