Panoramic shooting method and electronic equipment

Abstract

The embodiment of the invention discloses a panoramic shooting method and electronic equipment. The electronic equipment can display a first shooting interface, and the first shooting interface comprises a viewing frame, a first identifier and a splicing preview area. A preview of the first picture is displayed in the splicing preview area; the viewing frame is used for displaying a preview picture; the first identifier is displayed in the viewing frame and is used for marking the center of the viewing frame; the first identifier does not coincide with the second identifier, the second identifier is used for prompting the user to move the electronic equipment to obtain a second picture, and the second picture is the next picture spliced by the first picture in panoramic splicing. And the electronic equipment detects the movement of the electronic equipment, responds to the movement, refreshes the preview picture in the viewing frame, and shoots the second picture when the second picture appears in the viewing frame and the first identifier coincides with the second identifier. The electronic equipment can update the preview of the first picture in the splicing preview area into the preview of the third picture, and the third picture is formed by splicing the first picture and the second picture.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and in particular, to a panoramic shooting method and an electronic device. Background Art

[0002] In the field of terminal photography, panoramic shooting technology is widely used in scenarios such as scenic spot shooting and architectural photography. The panoramic shooting technology realizes the omnidirectional display of a scene by seamlessly stitching multiple images into a single image with a wide field of view.

[0003] However, the current panoramic shooting technology has the following problems: large distortion in the resulting image, difficult control of direction and speed, difficult central composition, and inability to perform real-time preview and comparison. These problems indicate that the image stitching mode and guiding method used in the existing panoramic shooting methods actually pose a relatively high operation threshold for panoramic shooting, thereby increasing the user's learning cost and shooting difficulty, and reducing the usability and user experience of panoramic shooting. Summary of the Invention

[0004] This application provides a panoramic shooting method and an electronic device. In the above panoramic shooting method, the electronic device can guide the user through a first identifier and a second identifier in the panoramic shooting interface. Among them, the first identifier is displayed in the viewfinder and is used to mark the center of the viewfinder; the second identifier is used to prompt the user to move the electronic device to obtain a second image, and the second image is the next image to be stitched to the first image in panoramic stitching. The electronic device detects the movement of the electronic device, and in response to the above movement, refreshes the preview image in the viewfinder until the second image appears in the viewfinder and the first identifier coincides with the second identifier, and then shoots the second image; the electronic device updates the preview image of the first image in the stitching preview area to the preview image of a third image, and the third image is composed of the first image and the second image stitched together. Through the above panoramic shooting method, the electronic device can also realize the guidance of multiple panoramic stitching modes, so as to effectively guide the user to perform panoramic stitching during the panoramic shooting process and reduce the operation difficulty of the user for panoramic shooting.

[0005] In a first aspect, the present application provides a panoramic shooting method, which includes: an electronic device displays a first shooting interface, and the first shooting interface includes: a viewfinder, a first identifier, and a stitching preview area; wherein, a preview image of a first picture is displayed in the stitching preview area; the viewfinder is used to display a preview picture; the first identifier is displayed in the viewfinder and is used to mark the center of the viewfinder; the first identifier does not coincide with a second identifier, and the second identifier is used to prompt the user to move the electronic device to obtain a second picture, and the second picture is the next picture to be stitched to the first picture in panoramic stitching; when the electronic device detects that the electronic device moves, in response to the movement, the preview picture in the viewfinder is refreshed until the second picture appears in the viewfinder and the first identifier coincides with the second identifier, and then the second picture is taken; the electronic device updates the preview image of the first picture in the stitching preview area to a preview image of a third picture, and the third picture is formed by stitching the first picture and the second picture.

[0006] When implementing the method provided in the first aspect, the electronic device can display guiding identifiers in the panoramic shooting interface (i.e., the above-mentioned first shooting interface) to guide the user to move the electronic device, automatically trigger shooting of pictures for panoramic stitching at specific positions, and display the stitched pictures in the stitching preview area. Compared with the existing guiding method that simply uses horizontal straight lines and direction identifiers, implementing the method provided in the first aspect can provide a more intuitive and user-friendly guiding method for panoramic shooting for users, thereby greatly reducing the operation threshold of panoramic shooting and enhancing the user experience.

[0007] In combination with the method provided in the first aspect, in some embodiments, a direction identifier is also displayed in the first shooting interface, and the direction identifier points from the first identifier to the second identifier and is used to indicate the viewfinder movement direction.

[0008] When implementing the method provided in the above embodiments, the electronic device can display a direction identifier (such as an arrow) in the panoramic shooting interface, so as to be able to provide clear guiding for panoramic shooting for users, help users move the shooting device more accurately in accordance with the preset direction. This can significantly improve the success rate and quality of panoramic shooting, enable users to complete the panoramic shooting operation more easily, enhance the user experience, reduce the shooting difficulty, and make panoramic shooting more convenient and efficient.

[0009] In combination with the method provided in the first aspect, in some embodiments, the method further includes: saving a panoramic image, and the picture of the panoramic image is the third picture or includes the third picture.

[0010] When implementing the method provided in the above embodiments, the electronic device can save the panoramic image after panoramic shooting, and the saved panoramic image should contain or be the picture whose preview image is displayed in the stitching preview area after stitching, so as to facilitate the user to view the panoramic image. Optionally, the electronic device can save the above panoramic image in the gallery of the electronic device.

[0011] In some embodiments, in combination with the method provided in the first aspect, the conditions for saving a panoramic image include: the specification of the panoramic image reaches a preset panoramic image specification, or the user ends panoramic shooting.

[0012] When implementing the method provided in the above embodiments, the above panoramic shooting can be automatically ended based on program settings, that is, when the specification of the captured panoramic image reaches the preset panoramic image specification, the panoramic shooting ends. The above panoramic shooting can also be manually ended by the user. For example, when the electronic device detects a user operation on the stop shooting control in the panoramic shooting interface, the electronic device can also end the current panoramic shooting and save the panoramic image. The electronic device provides two ways for the user to save the panoramic image (that is, end the current panoramic shooting), which can cope with different usage scenarios and meet different user needs.

[0013] In some embodiments, in combination with the method provided in the first aspect, the electronic device captures the first frame of the panoramic stitching; before capturing the first frame, the first identifier and the second identifier coincide.

[0014] When implementing the method provided in the above embodiments, during panoramic shooting, since the frame in the viewfinder will definitely participate in the panoramic stitching as the first frame, before the electronic device captures the first frame, in the panoramic shooting interface, the first identifier (such as a circular identifier) and the second identifier (such as a round identifier) coincide. The design of the coincidence of the above first identifier and the second identifier can provide a clear shooting starting point for the user and enable the user to focus more on the shooting content when capturing the first frame of the panoramic stitching without paying too much attention to the operation details of the device.

[0015] In some embodiments, in combination with the method provided in the first aspect, when a preview image of the first frame is displayed in the stitching preview area, the method further includes: the electronic device further displays a third identifier in the stitching preview area, and the third identifier is used to indicate the position of the second frame relative to the first frame.

[0016] When implementing the method provided in the above embodiments, the electronic device can also display a third identifier (such as a positioning block diagram) in the stitching preview area to indicate the position of the next frame (that is, the above second frame) of the panoramic stitching, so as to help the user understand the position of the second frame in the entire panoramic image.

[0017] In some embodiments, in combination with the method provided in the first aspect, after capturing the second frame, the method further includes: the electronic device changes the position of the third identifier in the stitching preview area, and the third identifier is used to indicate the position of the fourth frame relative to the second frame in the stitching preview area, and the fourth frame is the next frame stitched to the third frame in the panoramic stitching.

[0018] When implementing the method provided in the above embodiments, after the first identifier and the second identifier overlap, the electronic device can automatically capture the next frame for panoramic stitching. Subsequently, the electronic device can move the third identifier (such as a positioning block diagram) to the position of the next frame for panoramic stitching in the stitching preview area, thereby helping the user to understand the current panoramic shooting progress in real time.

[0019] Combined with the method provided in the first aspect, in some embodiments, during the panoramic stitching process, the viewfinder movement trajectory is a trajectory that moves in the horizontal or vertical direction towards the first direction.

[0020] When implementing the method provided in the above embodiments, the electronic device provides the user with a multi-frame stitching mode. In the above multi-frame stitching mode, the trajectory of the user moving the electronic device, that is, the viewfinder movement trajectory, is a trajectory that moves in the horizontal or vertical direction towards the first direction, thereby meeting the shooting needs of the user in shooting scenarios such as landscape photography and architectural photography.

[0021] Combined with the method provided in the first aspect, in some embodiments, during the panoramic stitching process, the viewfinder movement trajectory is a trajectory that first moves in the horizontal or vertical direction towards the first direction and then moves towards the second direction; the second direction is the opposite direction of the first direction.

[0022] When implementing the method provided in the above embodiments, the electronic device provides the user with a three-frame stitching mode. In the above three-frame stitching mode, the user can first capture the frame at the center position of the panoramic image, then sequentially capture the frames on the left or right of the center position, and finally capture the frame on the other side of the center frame. Based on the above three-frame shooting mode, the user can first capture the center frame, thereby helping the user to better compose the picture and reducing the operation threshold for the user to perform panoramic shooting.

[0023] Combined with the method provided in the first aspect, in some embodiments, during the panoramic stitching process, the viewfinder movement trajectory is a rectangular trajectory that starts from the viewfinder starting position and moves around the viewfinder starting position.

[0024] When implementing the method provided in the above embodiments, the electronic device provides the user with a grid-frame stitching mode. In the above grid-frame stitching mode, the user can first capture the center frame, and then sequentially capture the subsequent stitching frames around the center frame. Compared with the traditional panoramic shooting method, the above grid-frame stitching mode expands the picture specification and at the same time avoids the problem that the picture content is too scattered due to the overly wide ratio, thereby meeting the shooting needs of the user in different shooting scenarios such as landscape, architecture, and portrait. The panoramic image obtained by shooting also conforms more to the human visual habit.

[0025] In some embodiments, the method provided in combination with the first aspect further includes: before displaying the first shooting interface, a second preview interface is also displayed, and the second preview interface includes a stitching mode switching control for a user to select the following stitching modes: a first stitching mode, a second stitching mode, and a third stitching mode; wherein, in the first stitching mode, during the panoramic stitching process, the viewfinder movement trajectory is a trajectory that first moves in a first direction horizontally or vertically and then moves in a second direction, and the second direction is the opposite direction of the first direction; in the second stitching mode, during the panoramic stitching process, the viewfinder movement trajectory is a rectangular trajectory that starts from the viewfinder starting position and moves around the viewfinder starting position; in the third stitching mode, during the panoramic stitching process, the viewfinder movement trajectory is a trajectory that moves in a first direction horizontally or vertically.

[0026] When implementing the method provided in the above embodiments, the electronic device may display a stitching mode switching control 205 in the panoramic preview interface (i.e., the above-mentioned second preview interface). The user can operate the above-mentioned stitching mode switching control 205 to jump to the stitching mode selection interface to select different panoramic stitching modes, so that the user can select different panoramic stitching modes according to their own shooting needs, thereby improving the flexibility and applicability of the above panoramic shooting method.

[0027] In some embodiments, the method provided in combination with the first aspect further includes: if the distance between the center of the second picture and the center of the first picture is greater than half of the viewfinder, the electronic device does not display the second identifier in the viewfinder; if the distance between the center of the second picture and the center of the first picture is less than or equal to half of the viewfinder, the electronic device displays the second identifier in the viewfinder.

[0028] When implementing the method provided in the above embodiments, the electronic device may not display the above-mentioned second identifier in the viewfinder when the distance between the center of the second picture and the center of the first picture is too large. Thus, on the one hand, the electronic device can remove unnecessary identifiers in the viewfinder when the center of the second picture has not yet appeared in the viewfinder, reducing visual interference. On the other hand, the electronic device can continue to guide the movement of the electronic device through the direction identifier, enhancing the overall shooting experience.

[0029] In some embodiments, if the distance between the center of the second picture and the center of the first picture is greater than half of the viewfinder, the method further includes: when it is detected that the electronic device moves in the first viewfinder direction until the composition center of the second picture appears in the viewfinder, the electronic device displays the second identifier in the viewfinder.

[0030] When implementing the method provided in the above embodiment, if the electronic device does not display the second identifier at a certain moment, as the user moves the electronic device in the guided framing direction until the center of the second screen appears in the viewfinder, the electronic device can redisplay the second identifier in the viewfinder. The design of the above guiding method provides real-time feedback to the user and improves the user experience of panoramic shooting.

[0031] In combination with the method provided in the first aspect, in some embodiments, the panoramic image is composed of multiple images stitched together. The multiple images are successively captured at multiple times, and the multiple times are the multiple times when the first identifier coincides with the second identifier before saving the panoramic image.

[0032] In combination with the method provided in the first aspect, in some embodiments, before capturing the second screen, the method further includes: when the first identifier moves to coincide with the second identifier, the preview screen in the viewfinder is in an out-of-focus state, and the electronic device does not highlight the first identifier and the second identifier; after successful focusing, the electronic device highlights the first identifier and the second identifier.

[0033] When implementing the method provided in the above embodiment, the electronic device can capture the image with panoramic stitching only after successful focusing, and distinguish whether the electronic device is in successful focus by whether to highlight the guiding identifier on the panoramic shooting interface. The above design enables the user to intuitively understand whether the current focusing is successful through the highlighting state of the guiding identifier. This real-time feedback mechanism allows the user to quickly adjust the focus during the shooting process. And capturing the image after successful focusing can effectively avoid shooting failure caused by blurred focus, thereby improving the success rate of panoramic image capture and effectively reducing the operation threshold of panoramic shooting.

[0034] In a second aspect, the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory; the processor executes the computer program to implement the method described in the first aspect and any possible implementation manner in the first aspect.

[0035] In a third aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the method described in the first aspect and any possible implementation manner in the first aspect.

[0036] In a fourth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the method described in the first aspect and any possible implementation manner in the first aspect.

[0037] Understandably, the electronic device provided in the second aspect, the computer storage medium provided in the third aspect, and the computer program product provided in the fourth aspect are all used to execute the method provided in this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding method, which will not be elaborated here. Description of the Drawings

[0038] Figure 1 is a schematic diagram of a panoramic shooting interface provided by an embodiment of the present application;

[0039] Figure 2 is a schematic diagram of a panoramic preview interface provided by an embodiment of the present application;

[0040] Figure 3A is a schematic diagram of a preview interface of a three-frame stitching mode provided by an embodiment of the present application;

[0041] Figure 3B is a schematic diagram of a preview interface of a multi-frame stitching mode provided by an embodiment of the present application;

[0042] Figure 3C is a schematic diagram of a preview interface of a grid-frame stitching mode provided by an embodiment of the present application;

[0043] Figure 4 is a schematic diagram of a shooting interface of panoramic shooting provided by an embodiment of the present application;

[0044] Figures 5A - 5C is a set of guiding schematic diagrams of panoramic shooting in a three-frame mode provided by an embodiment of the present application;

[0045] Figures 6A - 6C is a set of guiding schematic diagrams of panoramic shooting in a grid-frame mode provided by an embodiment of the present application;

[0046] Figures 7A - 7F is a set of user interaction schematic diagrams of panoramic shooting in a grid-frame stitching mode provided by an embodiment of the present application;

[0047] Figures 8A - 8H is a set of user interaction schematic diagrams of panoramic shooting in a three-frame stitching mode provided by an embodiment of the present application;

[0048] Figures 9A - 9H is a set of user interaction schematic diagrams of panoramic shooting in a multi-frame stitching mode provided by an embodiment of the present application;

[0049] Figure 10 is a schematic diagram of a user interface when looking back at the content of panoramic shooting provided by an embodiment of the present application;

[0050] Figure 11 is another schematic diagram of a user interface when looking back at the content of panoramic shooting provided by an embodiment of the present application;

[0051] Figures 12A - 12B It is a schematic diagram of panoramic shooting with vertical screen longitudinal splicing provided by an embodiment of the present application;

[0052] Figures 13A - 13B It is a schematic diagram of panoramic shooting with horizontal screen horizontal splicing provided by an embodiment of the present application;

[0053] Figures 14A - 14B It is a schematic diagram of panoramic shooting with horizontal screen vertical splicing provided by an embodiment of the present application;

[0054] Figure 15 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application;

[0055] Figure 16 It is a schematic diagram of the software architecture of a panoramic shooting method provided by an embodiment of the present application. Detailed implementation manners

[0056] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0057] In the field of terminal photography, panoramic shooting technology is widely used in scenarios such as scenic spot shooting and architectural photography. The panoramic shooting technology realizes the all-round display of the scene by seamlessly splicing multiple images into an image with a wide field of view.

[0058] Figure 1 It is a schematic diagram of a panoramic shooting interface provided by an embodiment of the present application.

[0059] As Figure 1 shown, the above panoramic shooting interface may include a preview area, a guiding area, a panoramic imaging display area, and a control area. Among them, the above preview area allows the user to view the picture in the current viewfinder in real time, the guiding area provides visual assistance to help the user understand how to move the electronic device for panoramic shooting, the panoramic imaging display area is used to display the spliced panoramic image during panoramic shooting, and various interaction buttons may be displayed in the above control area, such as Figure 1 which includes a stop shooting control, etc. In the panoramic shooting mode, after detecting a user operation on the start shooting control, the electronic device may display as Figure 1The panoramic shooting interface shown. At this time, the electronic device can use the preview image in the above preview area when the above start shooting control is triggered as the starting image for stitching the panoramic image. Exemplarily, as the user rotates the electronic device to the right according to the rotation direction prompted in the guidance area, the electronic device can perform real-time stitching of the newly captured image and the previously captured image to obtain a stitched image, and display the stitched image in the panoramic imaging display area. The above panoramic shooting can end after reaching a predetermined stitching range or detecting a user operation on the stop shooting control. Subsequently, the electronic device can save the above stitched image as the panoramic image obtained from this panoramic shooting.

[0060] Through the above panoramic shooting method, users can capture a complete picture with a wider perspective at one time, without the need for multiple shootings and post-stitching. The panoramic image obtained based on the above panoramic shooting method can also provide users with a more immersive viewing experience, making panoramic shooting an ideal choice for recording and displaying vast scenes.

[0061] However, the current panoramic shooting technology has the following problems, which limit the popularization and effect of panoramic shooting:

[0062] 1. Large distortion in the final image: In the existing technology, large distortion is easily generated during the image stitching process, which not only affects the beauty of the image, but also increases the operation difficulty during shooting by users and raises the possibility of shooting errors.

[0063] 2. Difficult to control direction and speed: During the shooting process, it is not easy for users to accurately grasp the moving direction and speed of the camera, which may lead to difficulties in aligning the captured images. If the moving speed is too fast, there may be a situation where part of the stitched image is blurred, thus affecting the final stitching effect.

[0064] 3. Problem with central composition stitching: The existing technology cannot effectively complete the stitching of group photos or grid pictures from the central composition, which is a challenge for shooting scenarios that require precise composition (such as group photos).

[0065] 4. Problem with real-time preview and comparison: During the shooting process, the existing technology cannot provide real-time preview and relative position comparison with the existing finished images when the user moves the viewfinder, which makes it difficult for users to evaluate and adjust the shooting effect.

[0066] These problems indicate that the image stitching mode and guidance method used in the existing panoramic shooting methods actually make there be a relatively high operation threshold for panoramic shooting, thus increasing the learning cost and shooting difficulty of users and reducing the usability and user experience of panoramic shooting. Improving these aspects can significantly enhance the user experience and image quality of panoramic shooting, making the panoramic shooting technology more practical and efficient.

[0067] In view of this, an embodiment of the present application provides a panoramic shooting method, which is applied to an electronic device with a camera and a display screen, such as a mobile phone or a tablet computer. In the panoramic shooting method provided by the embodiment of the present application, the electronic device can display a first shooting interface, and the first shooting interface includes: a viewfinder, a first identifier, and a stitching preview area; wherein, a preview image of a first picture is displayed in the stitching preview area; the viewfinder is used to display a preview picture; the first identifier is displayed in the viewfinder and is used to mark the center of the viewfinder; the first identifier does not coincide with a second identifier, and the second identifier is used to prompt the user to move the electronic device to obtain a second picture, and the second picture is the next picture to be stitched to the first picture in panoramic stitching. The electronic device can detect that the electronic device moves. In response to the movement, the electronic device can refresh the preview picture in the viewfinder until the second picture appears in the viewfinder and the first identifier coincides with the second identifier, and then shoot the second picture. The electronic device can update the preview image of the first picture in the stitching preview area to a preview image of a third picture, and the third picture is formed by stitching the first picture and the second picture. Among them, the field of view of the third picture is larger than that of the first picture and the second picture.

[0068] In the above panoramic shooting method, the electronic device provides a more intuitive and user-friendly way to guide panoramic shooting, thus greatly reducing the operation threshold of panoramic shooting and improving the user experience.

[0069] Not limited to mobile phones and tablet computers, the above electronic device can also be a handheld computer, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, a smart home device, and / or a smart city device. The embodiment of the present application does not impose special restrictions on the specific type of the electronic device.

[0070] Figure 2 It is a schematic diagram of a panoramic preview interface provided by an embodiment of the present application. After the user selects the panoramic mode on the camera main page, the electronic device can display a panoramic preview interface 201 as shown in Figure 2 The above panoramic preview interface is also referred to as a second preview interface.

[0071] As shown in Figure 2 The above panoramic preview interface 201 may include a panoramic stitching mode identifier 202, a preview picture 203, a composition center identifier 204, a stitching mode switching control 205, a focal length selection control 206, a panoramic stitching direction control 207, a gallery control 208, and a start shooting control 209.

[0072] Among them, the above panoramic stitching mode identifier 202 is used to indicate the currently selected panoramic stitching mode. Among them, the square in the panoramic stitching mode identifier 202 is used to indicate the position of the first picture for stitching in the panoramic image in this panoramic stitching mode, and the direction identifier (such as an arrow) in the panoramic stitching mode identifier 202 is used to indicate the moving direction of the viewfinder in this panoramic stitching mode; the above preview picture 203 is located in the viewfinder and is used to display in real time the image captured by the user through the electronic device camera; the above composition center identifier 204 is located at the center of the viewfinder to guide the user to align and shoot correctly; the above panoramic stitching mode switching control 205 is used to display the currently selected panoramic stitching mode and jump to display the stitching mode selection interface after detecting a user operation to meet different shooting needs of the user. The above panoramic stitching mode may include a three-frame stitching mode, a multi-frame stitching mode, and a grid-frame stitching mode. For example, based on Figure 2 The shown panoramic stitching mode selection control, it can be known that in Figure 2 the panoramic stitching adopts a three-frame stitching mode. The above focal length selection control 206 is used to adjust the focal length to obtain the best shooting effect. The above panoramic stitching direction control 207 is used to indicate to the user how to move the electronic device to complete the panoramic stitching, where the stitching direction being horizontal means that the images are stitched left and right, and the stitching direction being vertical means that the panoramic stitching is for the images to be stitched up and down. The above gallery control 208 is used to provide the user with an entry to quickly access the gallery, and the captured panoramic images can be stored in the gallery. The above start shooting control 209 is used to trigger the panoramic shooting function after detecting a user operation on the start shooting control 209. Without being limited to the above components, the above panoramic preview interface 201 may include more or fewer controls, and the embodiments of the present application do not make special restrictions on this.

[0073] When the electronic device detects a user operation on the above panoramic stitching mode switching control 205, the electronic device jumps from Figure 2 to Figure 3A , Figure 3B , Figure 3C to any one of the interfaces, and the above Figure 3A , Figure 3B , Figure 3C can be collectively referred to as the stitching mode selection interface. Among them, the user operations on the controls described in this article include, but are not limited to, clicking operations on the controls, and the embodiments of the present application do not make special restrictions on the user operations in the above expressions. It can be understood that which interface the electronic device specifically jumps from Figure 2 depends on Figure 2 which panoramic stitching mode the panoramic stitching mode switching control 205 currently displays in Figure 2If the identifier of the three-frame stitching mode is displayed on the panoramic stitching mode switching control 205, after detecting a user operation on the panoramic stitching mode switching control 205, the electronic device can switch from Figure 2 to Figure 3A . Similarly, if the identifier of the multi-frame stitching mode is displayed on the panoramic stitching mode switching control 205 in Figure 2 , after detecting a user operation on the panoramic stitching mode switching control 205, the electronic device can switch from Figure 2 to Figure 3B ; if the identifier of the grid-frame stitching mode is displayed on the panoramic stitching mode switching control 205 in Figure 2 , after detecting a user operation on the panoramic stitching mode switching control 205, the electronic device can switch from Figure 2 to Figure 3C .

[0074] Next, the stitching mode selection interface will be introduced by taking Figure 3A as an example. Compared with Figure 2 , in the stitching mode selection interface shown in Figure 3A , the panoramic stitching mode switching control 205, the focal length selection control 206, and the panoramic stitching direction control 207 may not be displayed, and a panoramic stitching mode capsule control 301 is displayed at the original position of the focal length selection control 206.

[0075] As can be seen from Figures 3A - 3C , in the above stitching mode selection interface, the panoramic stitching mode indicated by the panoramic stitching mode identifier 202 corresponds one-to-one with the panoramic stitching mode differently displayed in the panoramic stitching mode capsule control 301. As shown by the panoramic stitching mode identifier 202 in Figure 3A , the above three-frame stitching mode refers to a panoramic stitching mode in which multiple (e.g., three) images are stitched horizontally to form a panoramic image, and the first frame of the panoramic stitching is used as the composition center of the panoramic image. As shown by the panoramic stitching mode identifier 202 in Figure 3B , the above multi-frame stitching mode refers to a panoramic stitching mode in which multiple images are stitched horizontally to form a panoramic image, and the first frame of the panoramic stitching is placed at the boundary of the panoramic image. As shown by the panoramic stitching mode identifier 202 in Figure 3C , the above grid-frame stitching mode refers to a panoramic stitching mode in which multiple images are stitched according to a predefined grid layout (e.g., a 3×3 nine-square grid) to form a panoramic image, and the first frame of the panoramic stitching is used as the composition center of the panoramic image. It can be understood that the first frame of the above panoramic stitching can be the first image captured by the electronic device or can be obtained by fusing the first n images captured by the electronic device. The embodiments of the present application do not make special limitations on this.

[0076] After the electronic device detects a user operation on the panoramic stitching mode capsule control 301 described above, the electronic device can switch to the panoramic stitching mode. For example, after the electronic device detects a user's selection operation for the "multi-frame" panoramic stitching mode in the panoramic stitching mode capsule control 301 described above, the electronic device can switch from Figure 3A to Figure 3B . The above panoramic stitching mode identifier 202 can change with the change of the panoramic stitching mode. Similarly, after the electronic device detects a user's selection operation for the "grid frame" panoramic stitching mode in the panoramic stitching mode capsule control 301 described above, the electronic device can switch from Figure 3A to Figure 3C .

[0077] In some embodiments, the electronic device can switch back to the Figure 2 shown panoramic preview interface when the duration of no user operation on the above stitching mode selection interface detected by the electronic device reaches a first duration. In some embodiments, the electronic device may also be provided with a confirmation control ( Figures 3A - 3C not shown in the figure) in the above stitching mode selection interface; when a user operation on the above confirmation control is detected, the electronic device can switch back to the Figure 2 shown panoramic shooting preview interface. The embodiments of the present application do not impose special restrictions on how the Figures 3A - 3C shown stitching mode selection interface specifically switches back to the Figure 2 shown panoramic shooting preview interface.

[0078] In the Figure 2 shown preview interface or the Figures 3A - 3C shown stitching mode selection interface, when a user operation on the start shooting control 209 is detected, the electronic device jumps to the panoramic shooting interface as shown in Figure 4 . The above panoramic shooting interface is also called the first shooting interface.

[0079] As shown in Figure 4 , the above panoramic shooting interface may include a stitching preview area 401, a viewfinder 402, a guiding area 403, a prompt area 404, and a control area 405.

[0080] The above stitching preview area 401 is used to present a preview image of the stitched image and indicate the stitching progress of the panoramic image. The above preview image may include a thumbnail. The above stitched image is also called the first image. The stitching preview area 401 may include a preview image of the stitched image and a third identifier. The third identifier may be Figure 4The positioning block diagram shown above. The above positioning block diagram can be used to indicate the position of the next picture (i.e., the second picture) to be stitched to the first picture in panoramic stitching relative to the first picture. In addition, the above positioning block diagram can also be used to indicate the position of the second picture in the entire panoramic picture. As Figure 4 shown, the above stitching preview area 401 is located in the area above the above viewfinder 402. In some embodiments, for example Figure 6B shown, the above stitching preview area 401 can also be located in the above viewfinder 402. The specific positional relationship between the above stitching preview area 401 and the viewfinder 402 in the embodiments of the present application is not particularly limited. Not limited to Figure 4 the positioning block diagram shown above, the specific form of the above third identifier in the embodiments of the present application is not particularly limited.

[0081] The above viewfinder 402 is used to present the preview picture of the current shot. The above viewfinder 402 may include the above guiding area 403. The above guiding area 403 is used to present guiding identifiers in the above viewfinder 402. For example, in Figure 4 the panoramic shooting interface shown above, the above guiding identifiers may include a circular identifier, a circular logo, and a direction identifier (such as Figure 4 the arrow shown above), where the above circular identifier is also called the first identifier and is located at the center of the viewfinder 402. The above circular logo is also called the second identifier and is used to mark the composition center of the next picture for panoramic stitching. The next picture for panoramic stitching can also be called the second picture. It can be understood that in the panoramic shooting method provided by the embodiments of the present application, the pixel distance between the above second identifier and the first identifier is related to the overlapping distance, and the overlapping distance is the straight-line distance of the overlapping part of the first image and the second image in the stitching direction during panoramic stitching. The greater the overlapping distance, the smaller the pixel distance between the second identifier and the first identifier. Specifically, the above pixel distance can be calculated based on the size of the viewfinder 402 and the overlapping distance. Exemplarily, the above pixel distance can be equal to the pixel size of the viewfinder 402 in the stitching direction minus the overlapping distance.

[0082] The above direction identifier is used to indicate the viewfinder movement direction, and this direction identifier points from the circular identifier to the circular logo. The above guiding identifiers may also include shooting auxiliary lines (such as Figure 4 the horizontal line shown above), which are used to assist the user in moving the electronic device. Not limited to the above form, the specific form of the above guiding identifiers (such as the specific shapes of the first identifier and the second identifier) in the embodiments of the present application is not particularly limited.

[0083] Optionally, the above viewfinder 402 may further include a prompt area 404. The above prompt area 404 is used to present prompt words, and the above prompt words are used to prompt the user to turn the direction, operation actions, and the current shooting progress. As Figure 4As shown, if the current prompt is "Rotate horizontally and take the second photo", it means that the first photo for stitching the panoramic image has been taken, and the user needs to continue rotating the electronic device horizontally to take the second photo for stitching the panoramic image. Similarly, the first photo mentioned here is the first frame of the above panoramic stitching.

[0084] The above control area 405 is used to display the stop shooting control 406. As Figure 4 shown, the above stop shooting control 406 can adopt a recording button. After detecting a user operation on the above-displayed stop shooting control 406, the electronic device can end the shooting of the above panoramic image and save the stitched image as a panoramic image. In addition, when the above panoramic image reaches the stitching limit, that is, when the above stitched image fills the above stitching preview area 401, the electronic device can also end the shooting of the above panoramic image.

[0085] In addition to the traditional multi-frame stitching mode, based on the functions in the above panoramic preview interface and the above panoramic shooting interface, the electronic device can also implement other different panoramic stitching modes, such as a three-frame stitching mode and a grid-frame stitching mode. Specifically, when it is detected that the user selects the three-frame stitching mode in the panoramic preview interface as Figure 5A shown, the electronic device can first jump to display a shooting interface as Figure 5B shown, and as Figure 5C shown, take the preview image in the viewfinder as Figure 5B the first frame for panoramic stitching and use this frame as the center of the panoramic image. And, as Figure 5C shown, in the three-frame stitching mode, the electronic device can use the above guiding marks to guide the user to rotate the electronic device to the left or right to take the next frame for panoramic stitching, so as to implement the above three-frame stitching. Among them, Figure 5C the two circular marks in are respectively the composition centers of the left image and the right image. It can be understood that the above three-frame stitching mode is not limited to stitching a panoramic image from three images. In the embodiments of the present application, the above three-frame stitching may also include a stitching mode of first stitching the middle image and then successively stitching other images on both sides. For example, the electronic device can first determine the first frame for panoramic stitching as the center of the panoramic image, and then successively stitch 5 images on the left side of the first frame for panoramic stitching. After the stitching on the left side is completed, 5 images are successively stitched on the right side of the above first frame. Then, the panoramic image stitched based on the above process can also be recognized as a panoramic image taken in the three-frame stitching mode. The above three-frame stitching mode is also called the first stitching mode.

[0086] When the user selects the grid-frame stitching mode in the preview image as Figure 6A shown, the electronic device can, as Figure 6BAs shown in Figure 6B the preview image in the viewfinder in is taken as the first image for panoramic stitching, and this image is used as the center of the panoramic image. Then, through Figure 6B the guiding signs in, the user is guided to turn the electronic device left or right to capture the next image for panoramic stitching. After detecting that the user turns left and captures the next image, the electronic device can determine that the user stitches the images in the clockwise direction, and presents guiding signs as shown in Figure 6C to guide the user to turn the electronic device upward, so as to obtain the upper image for stitching. The images are not limited to being stitched in the clockwise manner. In the above grid format stitching mode, the electronic device can also stitch the images in the counterclockwise direction or other preset paths. The embodiments of the present application do not make special limitations on this. It can be understood that the above grid format stitching mode is not limited to a nine-grid format, and can also include more formats such as a sixteen-grid format, or fewer formats such as a four-grid format. The embodiments of the present application do not make special limitations on the specific specifications of the grid format. The above grid format stitching mode is also called the second stitching mode. In addition, the multi-format stitching mode is also called the third stitching mode.

[0087] It can be understood that in the panoramic shooting method provided by the embodiments of the present application, no special limitations are made on the image processing algorithm for how the electronic device specifically stitches multiple images into a stitched image in various modes. Optionally, the electronic device can locate the images based on the overlapping parts of two images, and stitch the images based on this location.

[0088] Next, the user interaction in the three panoramic stitching modes in the above panoramic shooting method will be introduced in detail.

[0089] Grid format stitching mode

[0090] Figures 7A - 7F is a schematic diagram of user interaction for panoramic shooting in a group of grid format stitching modes provided by the embodiments of the present application.

[0091] After the user selects the grid format stitching mode, the panoramic preview interface is as shown in Figure 7A After detecting a user operation on the shooting control, the electronic device switches from the panoramic preview interface shown in Figure 7A to the panoramic shooting interface. At the beginning of displaying the panoramic shooting interface, that is, before shooting the first image, the circular mark and the circular mark coincide. At this time, the electronic device can shoot the first image for panoramic stitching. It can be understood that since in the panoramic shooting process, the first image after entering the shooting interface will definitely participate in the above panoramic stitching, before shooting the first image, the above circular mark and the circular mark coincide. Optionally, successful focusing can be used as a prerequisite for shooting the next image for panoramic stitching. Exemplarily, as shown in Figure 7BAs shown, the above guiding identifier 701 appears black. At this time, the electronic device may be in an unstable state or the picture may be out of focus. When the picture is in focus, at this time, the circular identifier in the above guiding identifier 701 is located at the center of the annular identifier (which also means that the circular identifier coincides with the annular identifier), then the electronic device can highlight the above guiding identifier 701( Figure 7C changing the guiding identifier 701 to gray in the [document] can be used to indicate that the electronic device highlights the guiding identifier 701). Subsequently, the electronic device can capture the first picture for panoramic stitching. Not limited to highlighting, the electronic device can also use other methods, such as color change, etc. to distinguish the guiding identifier 701 before and after the picture is in focus. The embodiments of the present application do not make special limitations on this. It can be understood that before the above guiding identifier 701 is highlighted, the above electronic device has not captured the first picture for panoramic stitching. Therefore, in Figure 7B and Figure 7C 's stitching preview area 702, the electronic device does not display any preview result pictures, and in the above stitching preview area 702, the positioning block diagram is still at the center position of the stitching preview area 702.

[0092] After capturing the first picture for panoramic stitching, as Figure 7D shown, the electronic device displays the above picture as a preview picture of the stitched image in the above stitching preview area 702. At this time, the first picture can be the first picture for panoramic stitching. Since in the grid format stitching mode, the user can rotate the electronic device clockwise to stitch the image, or rotate the electronic device counterclockwise to stitch the panoramic image, therefore, the next picture for panoramic stitching can be on the left side of the first picture for panoramic stitching, or on the right side of the first picture for panoramic stitching. It can be understood that since the next picture for panoramic stitching has not been captured at this time, there are two possibilities for the next image for the electronic device. At this time, the electronic device may not display the positioning block diagram in the above stitching preview area 702. In Figure 7D , the electronic device can display an annular identifier, two circular identifiers, shooting auxiliary lines, and direction identifiers (i.e., two arrows) in the guiding area 703. Among them, the above circular identifiers are respectively used to display the composition centers of the left and right two pictures, and the above two arrows respectively indicate rotating the electronic device to the left or right. And the electronic device can display the following prompt words "Rotate horizontally to capture the 2nd picture" in the prompt area of the panoramic shooting interface.

[0093] It is detected that the user is based on Figure 7DAfter the prompt in the guiding area 703 of the middle prompts to rotate the electronic device, in response to the above movement, the electronic device can refresh the preview image in the viewfinder. Moreover, in the guiding area 703 of the above viewfinder, the above annular mark can move between the two circular marks. It can be understood that the movement of the above annular mark between the circular marks means that the panoramic shooting method provided in the embodiments of the present application actually overlaps the center of the viewfinder and the composition center of the next image for panoramic stitching by rotating the electronic device, so as to realize the guidance for the user. However, actually in the viewfinder, what the user actually sees is that the annular mark remains stationary, because the annular mark itself represents the center of the viewfinder. Therefore, if the user horizontally moves the electronic device, the circular marks will gradually approach the annular mark until they coincide. If the user tilts the electronic device when moving it, the above shooting auxiliary line will also tilt. It can be understood that the above shooting auxiliary line functions similar to a spirit level in a camera. Exemplarily, as Figure 7E shown, when the electronic device detects that the above circular mark coincides with the above annular mark, the electronic device can capture a second image. Optionally, the following prompt words "Shooting, please keep stable" are displayed in the prompt area of the shooting interface.

[0094] After the second image is captured, based on the fact that the second image is on the left side of the first image in the panoramic stitching (i.e., the first image of the panoramic stitching), the electronic device can determine that the current stitching is in the clockwise direction, so as to calculate the position of the composition center of the next image (i.e., the 3rd image). It can be understood that the order of stitching images by the electronic device in the above grid frame mode can be set in the electronic device. Subsequently, the electronic device can switch from Figure 7E to Figure 7F .

[0095] In Figure 7F , the electronic device can stitch the above second image and the first image (here also the first image of the panoramic stitching) to obtain a third image, and update the preview image of the original first image in the stitching preview area 702 to the preview image of the third image. It can be understood that the field of view range of the above third image is larger than that of the first image and the second image. Moreover, in the stitching preview area 702 of Figure 7F , the electronic device can also display a positioning block diagram in the upper left corner. That is, after capturing the second image, the electronic device can change the position of the positioning block diagram in the stitching preview area. The above positioning block diagram is also used to indicate the position of the next image to be stitched in the stitching preview area relative to the second image, and the above fourth image is the next image of the third image in the panoramic stitching. In other words, the position of the above positioning block diagram in the stitching preview area changes with the progress of the panoramic shooting. As Figure 7FAs shown, the electronic device can display a circular identifier to identify the composition center of the current preview image, and display a circular identifier above the circular identifier. An arrow and a shooting auxiliary line are also displayed between the circular identifier and the circular identifier to indicate the direction of rotating the electronic device to the user.

[0096] In the grid format stitching mode, the guiding method in the subsequent interface can be analogized with reference to the relevant descriptions in the above text, and will not be elaborated here. After shooting, the electronic device can save the panoramic image, and the picture of the panoramic image includes the picture of the third picture.

[0097] Optionally, in the grid format mode, when the stitched image is non-rectangular, the electronic device can prohibit the use of the above stop shooting control. Exemplarily, as Figures 7A - 7F shown, when the third picture is taken, the electronic device can make the stop shooting control display in gray on the panoramic shooting interface at this time. At this time, the user cannot perform a user operation on the stop shooting control to end the shooting, so as to ensure that the finally saved panoramic image is a regular rectangle.

[0098] Optionally, in the grid format mode, when it is detected that the user stops shooting when the stitched image is non-rectangular, such as detecting a user operation on the stop shooting control, the electronic device can process the stitched image so that the saved panoramic image is a regular rectangle. Continuing the above example, after the third picture is taken and a user operation on the stop shooting control is detected, the electronic device can actively discard the third picture, thereby outputting the stitched image containing the first two pictures as the panoramic image and saving it in the gallery.

[0099] It can be understood that in the grid format mode, during the panoramic stitching process, the viewfinder movement trajectory is a rectangular trajectory starting from the viewfinder starting position and moving around the viewfinder starting position. Among them, the above viewfinder starting position is also the position when the first picture of the panoramic stitching is taken, and it is also the viewfinder position of the mobile phone when the two identifiers coincide for the first time. The above rectangular trajectory is a trajectory formed by sequentially connecting multiple discrete positions, and these discrete positions are respectively the positions of the electronic device when each picture is taken, and they are also the viewfinder positions of the electronic device when the two identifiers coincide multiple times. Take the grid stitching of a nine-grid as an example as Figures 7A - 7F shown. The nine-grid can be numbered in sequence from left to right and from top to bottom. The first row is serial numbers 1-3, the second row is serial numbers 4-6, and the third row is serial numbers 7-9. Then the above rectangular trajectory can start from the center of the nine-grid (serial number 5) and sequentially pass through the grids of serial numbers 4, 1, 2, 3, 6, 9, 8, and 7.

[0100] Three-frame stitching mode

[0101] Figures 8A - 8HIt is a schematic diagram of user interaction for panoramic shooting in a three-frame stitching mode provided by an embodiment of the present application.

[0102] After the user selects the three-frame stitching mode, the panoramic preview interface is as shown in Figure 8A . After detecting a user operation on the shooting control, the electronic device switches from the panoramic preview interface shown in Figure 8A to the panoramic shooting interface shown in Figure 8B . Similar to the grid-frame stitching mode, if the electronic device is stable and the picture is in normal focus, the electronic device can switch from Figure 8B to Figure 8C shown in the panoramic shooting interface. In Figure 8C , if the circular mark in the above guiding mark coincides with the annular mark (in this article, the case where the circular mark is located at the center of the annular mark is regarded as the coincidence of the circular mark and the annular mark), the electronic device can also highlight the above guiding mark 801. In the stitching preview areas 802 shown in Figure 8B and Figure 8C , the electronic device can display a positioning block diagram in the center to prompt the user that the next picture for panoramic stitching in this three-frame stitching mode is located at the center of the panoramic image. Subsequently, the electronic device can shoot the first picture of the panoramic stitching. After shooting the first picture of the panoramic stitching, the electronic device switches from Figure 8C to Figure 8D . As shown in Figure 8D , the electronic device can display the first picture of the panoramic stitching as the first picture in the stitching preview area 802. Since the three-frame stitching mode allows the user to freely choose to stitch to the left or to the right first after shooting the first picture of the panoramic stitching, the electronic device can switch the position of the circular mark from the center of the annular mark shown in Figure 8C to both sides of the annular mark shown in Figure 8D to indicate that the first preview picture is on the left or right side of the first picture of the panoramic stitching. Similarly, the electronic device can display the above shooting auxiliary lines and arrows between the annular mark and the circular mark. The electronic device can display the following prompt words in the prompt area of the panoramic shooting interface: "Turn to the right and shoot the 2nd picture".

[0103] When it is detected that the user rotates the electronic device based on the prompt in the guiding area 803 in Figure 8D , in the guiding area 803, the above circular mark can move along the horizontal shooting auxiliary line towards the annular mark. Taking the user turning to the left first as an example, as shown in Figure 8E , when the electronic device detects that the above circular mark coincides with the above annular mark, the electronic device can shoot the second picture. At this time, the following prompt words are displayed in the prompt area of the shooting interface: "Shooting, please keep stable".

[0104] After the second screen is captured, the electronic device can switch from Figure 8E to Figure 8F . As Figure 8F shown, the electronic device can splice the second screen and the first screen to obtain a third screen, and update the preview image of the first screen to the preview image of the third screen in the splicing preview area 802. At this time, if the electronic device also determines that the next screen for panoramic splicing is located at the far right of the panoramic image, the electronic device can calculate the composition center of the next screen and display a guiding identifier and prompt words as Figure 8F shown. It can be understood that since the composition center of the next screen is not within the viewfinder range, the electronic device can, as Figure 8F shown, not display the circular identifier. In some embodiments, when the composition center of the next screen is not within the panoramic shooting interface range, the electronic device can also display the above-mentioned circular identifier at the edge of the viewfinder. The embodiments of the present application do not impose special restrictions on this.

[0105] Similarly, when it is detected that the user rotates the electronic device based on the prompt in the guiding area 803 of Figure 8F , for example, as Figure 8G shown, the above-mentioned circular identifier can move along the horizontal shooting auxiliary line towards the annular identifier. It can be understood that when it is detected that the electronic device moves in the first shooting direction until the composition center of the second screen appears in the viewfinder, the electronic device displays the circular identifier in the viewfinder. Among them, the above-mentioned first shooting direction is the direction towards the second screen, that is, the direction in which the electronic device guides the user to move the electronic device. When the electronic device detects that the above-mentioned circular identifier coincides with the center of the above-mentioned annular identifier (as Figure 8H shown), the electronic device can continue shooting. It can be understood that when it is detected that the circular identifier coincides with the center of the above-mentioned annular identifier, the electronic device automatically shoots without the user manually shooting. After all the screens for splicing are captured, the electronic device can end the current panoramic shooting and store the spliced panoramic image in the gallery. The screens of the above-mentioned panoramic image include the screen of the above-mentioned third screen.

[0106] It can be understood that in the three-frame splicing mode, during the panoramic splicing process, the viewfinder movement trajectory is a trajectory that first moves in a first direction in the horizontal or vertical direction and then moves in a second direction; the second direction is the opposite direction of the first direction. That is, in the three-frame splicing mode, the viewfinder movement trajectory is a first trajectory, which is formed by sequentially connecting multiple positions. These multiple positions can be in the same horizontal / vertical direction. The first few positions experienced are arranged in the first direction, and the last few positions experienced are arranged in the second direction, and the second direction is opposite to the first direction.

[0107] Multi-frame splicing mode

[0108] Figures 9A - 9H It is a schematic diagram of user interaction for panoramic shooting in a multi-frame stitching mode provided by an embodiment of the present application.

[0109] After the user selects the multi-frame stitching mode, the panoramic preview interface is as shown in Figure 9A . After detecting a user operation on the shooting control, the electronic device switches from the panoramic preview interface shown in Figure 9A to the panoramic shooting interface shown in Figure 9B . Similar to the grid-frame stitching mode, if the electronic device is stable and the picture is in normal focus, the electronic device can switch from Figure 9B to Figure 9C shown in the full shooting interface. In Figure 9C , if the circular logo in the above guiding logo coincides with the annular logo, the electronic device can highlight the above guiding logo 901. In the stitching preview area 902 shown in Figure 9B and Figure 9C , the electronic device can display a positioning block diagram on the far left to prompt the user that the first picture for panoramic stitching in this multi-frame stitching mode is located at the boundary position of the panoramic image. Subsequently, the electronic device can shoot the first picture of the panoramic stitching. At this time, the electronic device displays Figure 9D , that is, the above annular logo can be presented as a loading animation during shooting. After shooting the first picture of the panoramic stitching, the electronic device switches from Figure 9D to Figure 9E . As shown in Figure 9E , the electronic device can display the first picture of the panoramic stitching as the first picture in the stitching preview area 902, and switch the position of the circular logo from the center of the annular logo as shown in Figure 9D to the right side of the annular logo shown in Figure 9E for display, thereby indicating that the first preview image is located on the right side of the first picture of the panoramic stitching. At the same time, the electronic device can display the above shooting auxiliary line and direction logo (such as an arrow) between the annular logo and the circular logo. The electronic device can display the following prompt word in the prompt area of the shooting interface: "Turn right and shoot the 2nd picture".

[0110] When it is detected that the user rotates the electronic device based on the prompt in the guiding area 903 in Figure 9E , in the guiding area 903, the above circular logo can move along the horizontal shooting auxiliary line towards the annular logo. As shown in Figure 9F , when the electronic device detects that the above circular logo coincides with the above annular logo, the electronic device can shoot the second picture. At this time, the following prompt word is displayed in the prompt area of the shooting interface: "Shooting, please keep stable".

[0111] After the second picture is shot, the electronic device can switch from Figure 9F toFigure 9G As shown Figure 9F in the splicing preview area 902 as shown in Figure 9F , the electronic device can splice the second screen and the first screen to obtain a third screen, and update the preview image of the first screen to the preview image of the third screen in the splicing preview area 902. In addition Figure 9G For each control, identifier, etc. in Figure 9G , reference can be made to Figure 9E , Figure 9G and Figure 9H The process of Figure 9H can refer to Figures 9E - 9F and will not be elaborated here

[0112] It can be understood that in some embodiments, at the beginning of panoramic shooting (i.e., at the stage as shown in Figure 9B or Figure 9C ), the above positioning block diagram can also be displayed on the far right of the splicing preview area 902. Then, in these embodiments, the electronic device can capture subsequent images during the process of turning left and splice them one by one to the far left of the first screen Figure 9B or Figure 9C shown).

[0113] In the above multi - frame splicing mode, the electronic device can store the spliced panoramic image in the gallery after the specification of the panoramic image reaches the preset panoramic image specification (i.e., reaches the maximum splicing limit) or the user ends panoramic shooting (such as detecting a user operation on the stop shooting control). The picture of the panoramic image includes the picture of the third screen

[0114] It can be understood that in the multi - frame splicing mode, during the panoramic splicing process, the view - finding movement trajectory is a trajectory that moves in the horizontal or vertical direction towards the first direction. Among them, the horizontal or vertical direction in this article is in the context of the Earth coordinate system. The above first direction can be a direction in the horizontal or vertical direction, such as towards the left

[0115] Based on the above interaction process, it can be known that the panoramic image obtained by shooting in the above panoramic splicing mode is composed of multiple images spliced together. These multiple images are shot at multiple times successively. The above multiple times are the multiple times when the first identifier coincides with the second identifier before saving the panoramic image. Specifically, the above multiple times can include the shooting time of the first picture and the multiple times when the first identifier coincides with the second identifier after adjusting the view - finding. It can be understood that as time passes, the completion degree of the splicing of the panoramic image is higher, and the third image obtained by splicing is also more complete. It can be understood that if during the above panoramic shooting process, the user terminates the shooting of the panoramic image in advance, for example, the electronic device detects a user operation on the stop shooting control before the spliced image reaches the preset panoramic image specification, then the specification of the panoramic image stored in the gallery of the electronic device can also be less than the above - mentioned preset panoramic image specification

[0116] When performing panoramic shooting using the above three panoramic stitching modes, in some embodiments, after detecting a user operation on the shooting control, if the electronic device is still in a shaking state or the picture is out of focus, the electronic device can display as shown in Figure 7B , Figure 8B or Figure 9B the interface shown, where the guiding identifier in the above shooting interface is not highlighted (i.e., the guiding identifier in the figure is black); otherwise, the electronic device may not display the above Figure 7B , Figure 8B or Figure 9B the interface shown. In some embodiments, if the shutter speed of the electronic device is greater than the first shutter threshold, the electronic device can display as shown in Figure 7C , Figure 8C or Figure 9C the interface shown, where the electronic device can highlight the guiding identifier in the above panoramic shooting interface (i.e., the guiding identifier in the figure is gray); otherwise, the electronic device may not display as shown in Figure 7C , Figure 8C or Figure 9C the interface shown. In some embodiments, if the shutter speed of the electronic device is greater than the second shutter threshold, and the second shutter threshold is greater than the first shutter threshold, the electronic device can display as shown in Figure 9D the interface shown, where the circular identifier in the guiding identifier shows a loading state. In other words, in some embodiments, after detecting a user operation on the shooting control, the electronic device can directly jump to as shown in Figure 7D , Figure 8D or Figure 9E the page shown.

[0117] It can be understood that compared with the prior art method of using a single arrow and horizontal line to prompt the user, the guiding method in the above panoramic shooting method gives the user a more intuitive visual guidance, which can help the user more accurately adjust the shooting angle and position, thereby improving the shooting accuracy and image quality and reducing the operation difficulty of panoramic shooting.

[0118] Moreover, in the grid format stitching mode and the three - frame stitching mode implemented through the above guiding method, the user can first determine the center of the composition and then perform subsequent panoramic image shooting. On the one hand, the panoramic stitching mode of first determining the center of the composition can result in a smaller deviation between each of the remaining images and the center image, and thus a smaller overall distortion degree of the stitched panoramic image, improving the imaging effect of the panoramic image; on the other hand, the panoramic stitching mode of first determining the center of the composition is conducive to improving the accuracy of the composition, thereby reducing the operation threshold for the user to perform panoramic shooting, reducing the failure rate caused by improper composition, and increasing the success rate of panoramic shooting.

[0119] It can be understood that in the existing panoramic stitching technology, users cannot view the relative position of the captured content in the preview result image during panoramic shooting. Exemplarily, in the existing technology, an electronic device can perform multi-frame stitching from left to right to form a panoramic image. During panoramic shooting, if the electronic device detects that the user rotates the electronic device to the left, the electronic device can end the current panoramic shooting and output the panoramic image, or the electronic device can capture a new image during the left rotation to overwrite the existing stitched image. This also increases the operation difficulty of users during panoramic shooting.

[0120] Therefore, in some embodiments, the electronic device can also view the relative position of the captured content in the stitched image during panoramic shooting, so as to help the user judge the current shooting position. Specifically, the electronic device can reflect the relative position of the captured content in the stitched image by dynamically adjusting the relative position of the positioning frame diagram on the stitching preview area.

[0121] Figure 10 It is a schematic diagram of a user interface when viewing panoramic shooting content provided by an embodiment of the present application.

[0122] As Figure 10 shown in (a) of, at this time, during the panoramic shooting process of the electronic device using the multi-frame stitching mode, the first picture in the current stitching preview area is obtained by stitching 2 images, and the positioning frame diagram 1001 is used to indicate the relative position of the next picture for panoramic stitching in the panoramic image. If according to Figure 10 shown in (a) of, the electronic device instructs the user to rotate the electronic device to the right, so as to continue shooting images for stitching the panoramic image. However, if the user does not rotate the electronic device to the right, but rotates the electronic device to the left, after detecting that the user rotates the electronic device to the left, the electronic device can calculate the relative position of the current preview picture in the stitched image based on the rotation angle, and as Figure 10 shown in (b) of, mark it through the positioning frame diagram. Among them, the electronic device can measure the above rotation angle through an inertial measurement unit (IMU).

[0123] In the user interface as Figure 10 shown, the above stitching preview area is fixed, and the electronic device reflects different positions of the preview picture in the viewfinder in the stitched image by adjusting the position of the positioning frame diagram. Not limited to the method as Figure 10 shown, when detecting that the user views the captured content, the electronic device can also jump from Figure 10 (a) of to the interface as Figure 11 shown, that is, by fixing the positioning frame diagram and moving the stitching preview area to achieve the same effect as Figure 10 shown.

[0124] Not limited to calculating the above relative position based on the rotation angle, the electronic device can also determine the relative position of the image content corresponding to the current preview image in the preview result image by means of image recognition, etc. The embodiments of the present application do not make special limitations on how the electronic device specifically determines the relative position of the image content corresponding to the current preview image in the preview result image.

[0125] It can be understood that by viewing the content of the panoramic shot, the user can flexibly adjust the splicing length of the panoramic image, thereby indirectly adjusting the overall composition of the panoramic image, and thus reducing the operation difficulty of the user for panoramic shooting.

[0126] Not limited to the horizontal splicing in the vertical screen situation in the above example, the above panoramic shooting method can also be applied to the panoramic shooting of vertical splicing in the vertical screen situation. Taking the three-frame splicing mode as an example, Figures 12A - 12B FIG. is a schematic diagram of panoramic shooting of vertical screen vertical splicing provided by an embodiment of the present application. It can be understood that after detecting the user operation on the panoramic splicing direction control 207, the electronic device can Figure 2 switch the display to Figure 12A . As Figure 12A shown, after the electronic device switches to vertical splicing, the electronic device switches the panoramic splicing direction control 207 to the panoramic splicing direction control 1201, and also switches the panoramic splicing mode identifier 202 to the panoramic splicing mode identifier 1202. The above panoramic splicing mode identifier 1202 indicates that when vertically splicing, the first picture of the panoramic splicing is still located at the composition center of the panoramic image, and the other two images are respectively located above and below the first picture of the panoramic splicing. After detecting the user operation on the shooting control, the electronic device can Figure 12A switch to Figure 12B . As Figure 12B shown, the electronic device vertically displays the above guiding identifier in the guiding area 1203. The above vertical display means that the circular identifier and the annular identifier are vertically arranged, and the shooting auxiliary line identifier for indicating whether the device is tilted is vertically displayed, and the arrow points upward and / or downward. The electronic device can arrange the splicing preview area 1204 vertically, and display the first picture of the panoramic splicing taken as the first picture in the splicing preview area 1204. The interfaces for vertical screen vertical splicing in other panoramic splicing modes can be obtained by analogy with the relevant descriptions of the three-frame splicing mode above, and will not be elaborated here.

[0127] Not limited to the vertical screen situation, the above panoramic shooting method can also be applied to the panoramic shooting of horizontal and / or vertical splicing in the horizontal screen situation. Still taking the three-frame splicing mode as an example, Figures 13A - 13B FIG. is a schematic diagram of panoramic shooting of horizontal screen horizontal splicing provided by an embodiment of the present application. The electronic device can automatically trigger from Figure 2The interface shown is switched to as Figure 13A the interface shown. In the interface shown in Fig. 13A, if a user operation on the panoramic stitching direction control 1301 is detected, the electronic device can be switched to as Figure 14A the interface shown. Figures 14A - 14B Fig. Figures 14A - 14B is a schematic diagram of panoramic shooting with horizontal screen and vertical stitching provided by an embodiment of the present application. Figures 13A - 13B the interface of Figures 14A - 14B and the interface of

[0128] Figure 15 Fig. Figure 15 is a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application.

[0129] As Figure 15 shown, the electronic device may include a processor 1501, a memory 1502, a wireless communication processing module 1503, a power switch 1504, a display screen 1505, an audio module 1506, a speaker 1507, a camera 1508 and other components. Components in the electronic device are connected through a bus and communicate based on the bus.

[0130] The processor 1501 may include one or more processing units. For example, the processor 1501 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. The controller can generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching instructions and executing instructions.

[0131] The memory 1502 is coupled to the processor 1501 and is used to store various software programs and / or multiple groups of instructions. The memory 1502 may be used to store computer-executable program code, and the executable program code includes instructions. The processor 1501 executes various functional applications and data processing of the electronic device by running the instructions stored in the memory 1502. A memory may also be provided in the processor 1501 to store instructions and data.

[0132] Among them, the memory 1502 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). The random access memory can be directly read and written by the processor 1501. The random access memory can be used to store the operating system or executable programs of other running programs (such as machine instructions), and can also be used to store user and application data, etc. The non-volatile memory can also store executable programs and store user and application data, etc. The executable programs and user data stored in the non-volatile memory can be pre-loaded into the random access memory for the processor 1501 to directly read and write.

[0133] The executable program code and data (including but not limited to various voice data) for implementing the panoramic shooting method provided by the embodiments of the present application can be stored in the non-volatile memory. During the process of implementing the above panoramic shooting method, the electronic device can load the executable program code and user data of the non-volatile memory into the random access memory to provide an intuitive and easy-to-use panoramic shooting guidance method for the user, thereby greatly reducing the operation threshold of panoramic shooting and enhancing the user experience.

[0134] The wireless communication processing module 1503 can provide wireless communication solutions including WLAN, such as Wi-Fi, Bluetooth communication, ZigBee communication, NFC communication, infrared communication, and UWB communication.

[0135] The power switch 1504 can be used to control the power supply to the electronic device, and thus supply power to the processor 1501, the memory 1502, the wireless communication processing module 1503, the display screen 1505, the audio module 1506, the speaker 1507, the camera 1508, etc.

[0136] The display screen 1505 can be used for display. The display screen 1505 includes a display panel. A touch sensor can be set in the display screen 1505. The touch sensor is used to detect touch operations acting on or near it. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event. Furthermore, the electronic device can provide visual output related to the touch operation through the display screen 1505. The electronic device can implement the display function through the GPU, the display screen 1505, the touch sensor, and the application processor, etc. In the embodiments of the present application, the electronic device can display the user interface of the camera application program to the user through the display function provided by the GPU, the display screen 1505, the touch sensor, and the application processor, etc., and perform user interaction with the electronic device through the above display screen 1505, such as selecting the panoramic stitching mode, etc.

[0137] The audio module 1506 can be used to convert a digital audio signal into an analog audio signal output, and can also be used to convert an analog audio input into a digital audio signal. The speaker 1507 can be used to convert the audio signal sent by the audio module 1506 into a sound signal. The electronic device can implement an audio playback function through the audio module 1506, the speaker 1507, etc. In some embodiments, the audio module 1506 can also include a microphone for converting the sound signal into an electrical signal. In an embodiment of the present application, especially in a multi-mode input scenario, the electronic device can call a microphone to pick up the user's voice, generate a voice packet, and then use the voice packet to perform voice input.

[0138] The electronic device can realize the shooting function through ISP, camera 1508, video codec, GPU, display screen 1505 and application processor. Camera 1508 is used to capture static images or videos. In the embodiment of the present application, camera 1508 can be used to capture images, so as to stitch them together to obtain a panoramic image.

[0139] It is to be understood that the structures illustrated in the embodiments of the present application do not constitute specific limitations on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components may be implemented in hardware, software, or a combination of software and hardware.

[0140] Figure 16 Schematic diagram of the software architecture of a panoramic shooting method provided by an embodiment of the present application. Figure 16 How does the software architecture in support the above hardware to achieve the above user interaction.

[0141] like Figure 16 As shown, the panoramic shooting method is implemented in a camera application, and its software architecture consists of multiple modules. Among them, the above-mentioned camera application can be installed on an electronic device through an Android application package (android package, APK) installation package, and the above-mentioned installation package contains the environment and resources required for the entire camera application to run. Inside the APK, the user interface (user experience, UX) module is responsible for starting the panoramic shooting function. The user selects the desired stitching mode through this interface and obtains a real-time stitching preview during the shooting process. In addition, the camera application can display a guide logo through the UX module to guide the user on how to move the device to optimize the shooting effect, and automatically trigger the photo function at the appropriate time. That is, the electronic device can implement user interaction in the above-mentioned multiple panoramic stitching modes through the above-mentioned UX module.

[0142] The panoramic shooting function in the camera application mainly involves two main functions: preview and photo taking.

[0143] The electronic device can start the preview function through the UX (User Interface) module. Subsequently, the camera application can send a call request to the hardware abstraction layer (HAL) to obtain the preview screen and the stitched image for display in the stitched preview area. The HAL layer is responsible for interacting with the underlying hardware and receiving the preview image from the camera and the data from the IMU sensor. These data are then transmitted to the image processing system (IPS) module, which is used to call the preview stitching algorithm to perform image processing on the preview image, generate the stitched image and the coordinates of the above guiding identifier on the screen for the user to view in the preview interface, thereby implementing the above preview function. Among them, the coordinates of the above guiding identifier on the screen are also called button coordinates, and the electronic device can determine the above button coordinates based on the progress of panoramic stitching and the pixel distance between the annular identifier and the circular identifier. The embodiments of the present application do not impose special restrictions on the above preview stitching algorithm and the calculation method of the above button coordinates.

[0144] The electronic device can start the photographing function through the UX (User Interface) module. Subsequently, the camera application can send a call request to the HAL layer to obtain the image data required for photographing. The HAL module is also responsible for collecting gyroscope information and 3A (auto-exposure, autofocus, auto white balance) data, and the above information and data are crucial for determining the focus area of autofocus and adjusting the brightness-related parameters. The IPS module receives these data and uses them for the photographing stitching algorithm, which is responsible for stitching multiple captured images into a complete panoramic image and storing it in the gallery module of the electronic device. In addition, the electronic device can also upload the multiple captured images to the cloud through the cloud stitching data module to implement the cloud camera stitching function, and the finally generated panoramic image is stored in the gallery module. The embodiments of the present application do not make special limitations on the above photographing stitching algorithm and the image stitching algorithm used in the cloud camera stitching function.

[0145] It can be understood that the software architecture illustrated in the embodiments of the present application does not constitute a specific limitation on the above panoramic shooting method. In other embodiments of the present application, the software architecture involved in the above panoramic shooting method may further include more or fewer modules.

[0146] Those skilled in the art should be able to realize that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented by hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. The computer-readable medium includes computer storage media and communication media, where the communication media includes any medium that facilitates the transfer of a computer program from one place to another. The storage media can be any available medium accessible by a general-purpose or special-purpose computer. The embodiments of the present application also provide a computer program product, including a computer program, which can implement the steps in each of the above method embodiments when the computer program runs on a processor.

[0147] The above specific implementation manners further elaborate on the purpose, technical solutions, and beneficial effects of the embodiments of the present application. It should be understood that the above is only the specific implementation manners of the embodiments of the present application and is not used to limit the protection scope of the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the protection scope of the embodiments of the present application.

Claims

1. A panoramic shooting method, characterized in that, The method includes: The electronic device displays a first shooting interface, which includes: a viewfinder, a first identifier, and a stitching preview area; wherein, a preview image of a first picture is displayed in the stitching preview area; the viewfinder is used to display the preview picture; the first identifier is displayed in the viewfinder and is used to mark the center of the viewfinder; the first identifier does not coincide with a second identifier, and the second identifier is used to prompt the user to move the electronic device to obtain a second picture, and the second picture is the next picture to be stitched to the first picture in panoramic stitching. The electronic device detects the movement of the electronic device, and in response to the movement, refreshes the preview picture in the viewfinder until the second picture appears in the viewfinder and the first identifier coincides with the second identifier, and then shoots the second picture. The electronic device updates the preview image of the first picture in the stitching preview area to a preview image of a third picture, and the third picture is composed of the first picture and the second picture stitched together.

2. The method according to claim 1, wherein A direction identifier is further displayed in the first shooting interface, and the direction identifier points from the first identifier to the second identifier and is used to indicate the viewfinder movement direction.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Saving a panoramic image, and the picture of the panoramic image is the third picture or includes the third picture.

4. The method according to claim 3, characterized in that, The conditions for saving the panoramic image include: the specification of the panoramic image reaches a preset panoramic image specification, or the user ends panoramic shooting.

5. The method according to any one of claims 1-4, characterized in that The method further includes: the electronic device shoots the first picture of the panoramic stitching; before shooting the first picture, the first identifier and the second identifier coincide.

6. The method according to any one of claims 1-5, characterized in that, When the preview image of the first picture is displayed in the stitching preview area, the method further includes: the electronic device further displays a third identifier in the stitching preview area, and the third identifier is used to indicate the position of the second picture relative to the first picture.

7. The method according to claim 6, wherein After shooting the second picture, the method further includes: The electronic device changes the position of the third identifier in the stitching preview area, and the third identifier is used to indicate the position of a fourth picture relative to the second picture in the stitching preview area, and the fourth picture is the next picture to be stitched to the third picture in the panoramic stitching.

8. The method according to any one of claims 1-7, characterized in that, During the panoramic stitching process, the viewfinder movement trajectory is a trajectory that moves in a horizontal or vertical direction towards a first direction.

9. The method according to any one of claims 1 to 7, characterized in that During the panoramic stitching process, the viewfinder movement trajectory is a trajectory that first moves in a horizontal or vertical direction towards a first direction and then moves in a second direction; the second direction is the opposite direction of the first direction.

10. The method according to any one of claims 1-7, characterized in that, During the panoramic stitching process, the viewfinder movement trajectory is a rectangular trajectory that starts from a viewfinder starting position and moves around the viewfinder starting position.

11. The method according to any one of claims 8 - 10, characterized in that, The method further includes: Before displaying the first shooting interface, a second preview interface is further displayed, and the second preview interface includes a stitching mode switching control, and the stitching mode switching control is used for the user to select the following stitching modes: a first stitching mode, a second stitching mode, and a third stitching mode. Wherein, In the first stitching mode, during the panoramic stitching process, the viewfinder movement trajectory is a trajectory that first moves in the horizontal or vertical direction towards the first direction and then moves towards the second direction, where the second direction is the opposite direction of the first direction; In the second stitching mode, during the panoramic stitching process, the viewfinder movement trajectory is a rectangular trajectory that starts from the viewfinder starting position and moves around the viewfinder starting position; In the third stitching mode, during the panoramic stitching process, the viewfinder movement trajectory is a trajectory that moves in the horizontal or vertical direction towards the first direction.

12. The method according to any one of claims 1-11, characterized in that, The method further includes: If the distance between the center of the second picture and the center of the first picture is greater than half of the viewfinder frame, the electronic device does not display the second identifier in the viewfinder frame; If the distance between the center of the second picture and the center of the first picture is less than or equal to half of the viewfinder frame, the electronic device displays the second identifier in the viewfinder frame.

13. The method according to claim 12, wherein If the distance between the center of the second picture and the center of the first picture is greater than half of the viewfinder frame, the method further includes: When it is detected that the electronic device moves in the first viewfinder direction until the composition center of the second picture appears in the viewfinder frame, the electronic device displays the second identifier in the viewfinder frame.

14. The method according to any one of claims 1 to 13, characterized in that The panoramic image is composed of multiple images stitched together, and the multiple images are taken at multiple times successively, where the multiple times are multiple times when the first identifier and the second identifier coincide before saving the panoramic image.

15. The method according to any one of claims 1-14, characterized in that, Before shooting the second picture, the method further includes: When the first identifier moves to coincide with the second identifier, the preview picture in the viewfinder is in an out-of-focus state, and the electronic device does not highlight the first identifier and the second identifier; After successful focusing, the electronic device highlights the first identifier and the second identifier.

16. An electronic device, characterized in that, It includes one or more processors and one or more memories; wherein, the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the method according to any one of claims 1-15 is performed.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program runs on an electronic device, the method according to any one of claims 1-15 is performed.

18. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1-15 is implemented.

Images