A photographing method and related device

Abstract

The application provides a photographing method and related equipment. According to the photographing method, the electronic device can change the aspect ratio of the window for shooting a close-up video without changing the shooting direction of the electronic device, thereby obtaining close-up videos with different proportions. In this process, the electronic device can change the cropping direction of the close-up image and perform rotation processing on the cropped close-up image, so that the size of the close-up image transmitted to the Buffer meets the aspect ratio of the Buffer. The camera application in the electronic device can also perform rotation processing on the close-up image transmitted by the Buffer again, and send the processed image to the rotated window for display. The above-mentioned method not only meets the demand of users for obtaining close-up images with multiple proportions, but also avoids the interruption of the camera path caused by the fact that the image transmitted to the Buffer does not meet the aspect ratio of the Buffer, that is, the phenomenon of freezing or blurring can be avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminals, and in particular to a photographing method and related equipment. BACKGROUND

[0002] With the development of terminal technology, video shooting has more possibilities. For example, people can realize automatic focus on a face, a person, a pet or other subjects through a terminal device such as a mobile phone, so as to solve the problem of defocus or focus shift when the subject moves and cannot be automatically focused on the subject. SUMMARY

[0003] The present application provides a photographing method and related equipment. According to the photographing method, in a photographing scene in which multiple photographing objects are in the same frame, an electronic device can focus on one of the photographing objects and record a close-up video of the photographing object, and generate an original video and a close-up video. In the case that the photographing direction of the electronic device does not change, the electronic device can change the aspect ratio of the window for photographing the close-up video, so as to obtain close-up videos of different proportions. In this process, the electronic device can change the cropping direction of the close-up image through the camera HAL, and rotate the cropped close-up image, so that the size of the close-up image transmitted to the Buffer meets the aspect ratio of the Buffer. The electronic device can also transmit the close-up image transmitted to the Buffer to the camera application, rotate the close-up image again through the camera application, and display the processed close-up image in the rotated window. The above-mentioned method not only meets the user's demand for obtaining close-up images of multiple proportions, but also avoids the interruption of the camera path caused by the fact that the image data transmitted to the Buffer does not meet the aspect ratio of the Buffer, that is, it can avoid the phenomenon of lag or blur.

[0004] In a first aspect, this application provides a shooting method. This shooting method can be applied to an electronic device. According to the method, the electronic device can display a first interface. The first interface includes a preview window. The preview window displays a first image. The first image displays one or more markers. The markers are used to mark the subject being shot. The electronic device can detect a first operation applied to the first marker, and in response to the first operation, a first window can be displayed on the preview window. It is understood that the first marker is any one of the one or more markers. The first window displays a close-up image of a first subject being shot, the first subject being the subject corresponding to the first marker. The first window may include a first control. The electronic device can detect a second operation applied to the first control, and in response to the second operation, adjust the aspect ratio of the first window from a first aspect ratio to a second aspect ratio. In response to a third operation to start recording, the electronic device can start recording video. In response to a fourth operation to end recording, the electronic device can save the original video and the close-up video. The original video is a video obtained based on the image displayed in the preview window, and the close-up video is a video obtained based on the image displayed in the first window.

[0005] In the solution provided in this application, after the electronic device enters the focus tracking mode, it can display a corresponding video preview interface. This video preview interface can include a preview window. The preview window displays a first image. The first image includes one or more subjects. Furthermore, one or more markers can be displayed on the first image to mark the subjects. The user can trigger the markers displayed on the first image, and the electronic device can then perform focus tracking on the marked subject, displaying a preview window upon successful focus tracking. The user can trigger corresponding controls on the preview window, allowing the electronic device to adjust the aspect ratio of the preview window. Based on this, the electronic device can record both raw video and close-up video. It should be noted that when the aspect ratio of the preview window changes, the aspect ratio of the close-up video acquired by the electronic device will also change. The above method can meet the user's need to acquire close-up videos in multiple aspect ratios, improving the user experience.

[0006] Understandably, the first interface can be the video preview interface after entering focus tracking mode (e.g., Figure 2D The user interface 400 shown is shown. The preview window can be the large preview window described later (e.g., Figure 2D The preview window 201 shown. The first window can be the preview window described later (e.g., Figure 2E The preview window shown is 501.

[0007] In some embodiments of this application, the subject of the photograph can be a person, and the marker can be a human body frame. Of course, the subject of the photograph can also be other objects, and the marker can have other forms of representation (e.g., symbols, text, etc.), and this application does not limit this.

[0008] In some embodiments of the present application, before the one or more markers are displayed on the first image, the electronic device can detect a photographed object in the first image according to a detection algorithm, and mark the detected photographed object. It can be understood that the detection algorithm can include a face detection algorithm, a human body detection algorithm, etc. The detection algorithm can include but is not limited to: inter-frame difference method, background modeling method, point detection method, image segmentation method, clustering analysis method and motion vector field method. Exemplarily, the detection algorithm can include a neural network model. For example, a neural network model based on deep learning. The related description of the detection algorithm can refer to the related technical documents, which will not be described here.

[0009] It can be understood that the first operation, the second operation, the third operation and the fourth operation can be user operations such as clicks, gestures, sounds, etc., which are not limited in the present application.

[0010] It can be understood that the first control is used to adjust the aspect ratio of the first window. Exemplarily, the first control can be the control 5012.

[0011] It can be understood that the aspect ratio of the first window refers to the ratio of the width and height of the first window. For example, the aspect ratio of the first window can be 1920:1080.

[0012] In some embodiments of the present application, the aspect ratio of the first window is the same as the aspect ratio of the image displayed by the first window. It can be understood that the aspect ratio of the image refers to the ratio of the width and height of the image. For example, the aspect ratio of the first window can be 1920:1080, the size of the image displayed by the first window is 1920px*1080px, and the aspect ratio of the image displayed by the first window is 1920:1080.

[0013] In some embodiments of the present application, the aspect ratio of the first window is different from the aspect ratio of the image displayed by the first window. For example, the aspect ratio of the first window can be 2000:1100, the size of the image displayed by the first window is 1920px*1080px, and the aspect ratio of the image displayed by the first window is 1920:1080.

[0014] It can be understood that the first aspect ratio and the second aspect ratio can be pre-set. For example, the first aspect ratio can be 1920:1080, and the second aspect ratio can be 1080:1920.

[0015] In some embodiments of the present application, the specific way in which the electronic device adjusts the aspect ratio of the first window from the first aspect ratio to the second aspect ratio is that the electronic device rotates the first window. For example, the display management module in the electronic device rotates the first window counterclockwise or clockwise by 90 degrees. In this case, the ratio of the first aspect ratio and the ratio of the second aspect ratio are reciprocal of each other.

[0016] It can be understood that the electronic device starts recording a video, which can specifically include that the electronic device records an original video based on the image displayed in the preview window and records a close-up video based on the image displayed in the first window.

[0017] It can be understood that the original video is a video obtained based on a panoramic image displayed in the preview window, and the close-up video is a video obtained based on a close-up image of the focus target displayed in the first window. The panoramic image mentioned in this application refers to an image captured by the electronic device through the camera based on the field of view thereof, and is not an image that embodies a 360-degree shooting scene in the general sense. The field of view of the camera is related to the focal length and the field of view angle thereof. It is worth noting that the panoramic image and the close-up image are not the original image data captured by the camera. The electronic device needs to perform corresponding processing on the original image data to obtain the panoramic image and the close-up image.

[0018] In some embodiments of the present application, the image displayed in the preview window has a corresponding relationship with the image constituting the original video, and the image displayed in the preview window can be an image obtained after compression of the corresponding image constituting the original video. Similarly, the image displayed in the first window has a corresponding relationship with the image constituting the close-up video, and the image displayed in the first window can be an image obtained after compression of the corresponding image constituting the close-up video.

[0019] In some embodiments of the present application, the original video is a video constituted by the image displayed in the preview window, and the close-up video is constituted by the image displayed in the first window.

[0020] With reference to the first aspect, in a possible implementation manner, the first window displays a first close-up image before the second operation acting on the first control is detected. The first window displays a second close-up image after the second operation acting on the first control is detected. The first close-up image has a first aspect ratio, and the second close-up image has a second aspect ratio. In the scheme provided in the present application, the aspect ratio of the first window and the close-up image displayed therein can be the same. When the aspect ratio of the first window changes, the aspect ratio of the close-up image displayed therein also changes accordingly. It can be understood that the proportion of the close-up video obtained by the electronic device also changes accordingly. The above-mentioned manner can meet the demand of the user for obtaining close-up videos of multiple proportions, and improves the user experience.

[0021] It can be understood that the first close-up image and the second close-up image can both be close-up images of the first shooting object. The first close-up image and the second close-up image can also be close-up images of other shooting objects.

[0022] With reference to the first aspect, in a possible implementation manner, the ratio of the first aspect ratio and the ratio of the second aspect ratio are reciprocal to each other.

[0023] In the scheme provided in the present application, the ratio of the first width-height ratio and the ratio of the second width-height ratio are reciprocal. Illustratively, the electronic device can rotate the first window by 90 degrees, thereby realizing the adjustment of the width-height ratio of the first window from the first width-height ratio to the second width-height ratio.

[0024] In combination with the first aspect, in a possible implementation, the width-height ratio of the buffer area Buffer in the camera FWK of the electronic device is the first width-height ratio. The Buffer is used to buffer image data sent to the camera FWK.

[0025] In the scheme provided in the present application, the width-height ratio of the buffer area Buffer in the camera FWK of the electronic device is consistent with the default width-height ratio of the first window. The default width-height ratio of the first window is the width-height ratio of the first window displayed after the electronic device enters the focus tracking mode and successfully focuses. In common terms, the default width-height ratio of the first window refers to the width-height ratio of the first window displayed by the electronic device when the first control is not triggered. The default width-height ratio of the first window can be pre-set. In some embodiments of the present application, the default width-height ratio of the first window is the first width-height ratio. The width-height ratio of the Buffer refers to the width-height ratio of the image that can be buffered by the Buffer.

[0026] It can be understood that the default width-height ratio of the first window can be related to the state of the electronic device when recording. For example, when the user records horizontally with the electronic device, the default width-height ratio of the first window is the first width-height ratio, and when the user records vertically with the electronic device, the default width-height ratio of the first window is the second width-height ratio. In some embodiments of the present application, when the width-height ratio of the first window is the default width-height ratio, the first window is parallel to the preview window. In some embodiments of the present application, when the width-height ratio of the first window is the default width-height ratio, the first window is perpendicular to the preview window. In some embodiments of the present application, the width-height ratio of the first window can be the same as the width-height ratio of the preview window. In some embodiments of the present application, the width-height ratio of the first window can be different from the width-height ratio of the preview window. For example, the width-height ratio of the first window can be 1000:500, and the width-height ratio of the preview window can be 1920:1080. In some embodiments of the present application, the ratio of the width-height ratio of the first window and the ratio of the width-height ratio of the preview window can be reciprocal.

[0027] Illustratively, the default width-height ratio of the first window is 1920:1080, and the width-height ratio of the Buffer is also 1920:1080.

[0028] It can be understood that the related description of the Buffer can be referred to later and will not be expanded here.

[0029] With reference to the first aspect, in a possible implementation manner, after detecting the second operation acting on the first control, the method further includes: in response to the second operation, the direction management module in the electronic device sends rotation information to the display management module and the camera HAL in the electronic device; the display management module determines a first rotation direction and a first rotation angle based on the rotation information; and the camera HAL updates the second image effect corresponding to the second data stream based on the rotation information, obtains a first image effect, and determines a second rotation direction and a second rotation angle. The first image effect includes a first cropping direction, and the first cropping direction corresponds to a first cropping size. The camera HAL processes the original image data based on the first image effect to obtain a first image. The original image data is image data collected by the camera of the electronic device. The camera HAL performs rotation processing on the first image based on the second rotation direction and the second rotation angle to obtain a second image. The display management module performs rotation processing on the second image based on the first rotation direction and the first rotation angle to obtain a second close-up image, and displays the second close-up image in the first window.

[0030] In the scheme provided in the present application, in the case where the aspect ratio of the preview window changes, the electronic device updates the cropping direction and then processes the original image data, thereby obtaining an image adapted to the preview window after the aspect ratio changes. It can be understood that the camera HAL can perform rotation processing on the image, and the aspect ratio of the rotated image is the same as the Buffer aspect ratio. This means that the rotated image can be cached in the Buffer, and the caching of the image will not be affected because the aspect ratios are consistent. The Buffer can upload the rotated image to the display management module in the camera application. The display management module can perform rotation on the rotated image again, and the obtained image is adapted to the preview window after the aspect ratio changes, and can be successfully displayed in the preview window. The above-mentioned manner not only meets the user's demand for obtaining close-up images of multiple proportions, but also avoids the interruption of the camera path caused by the image transmitted to the Buffer being different from the Buffer aspect ratio, that is, it can avoid the phenomenon of lag or blur.

[0031] In some embodiments of the present application, the first data stream is a large window data stream, and the second data stream is a small window data stream. The first image effect is an image effect updated after the aspect ratio of the preview window changes and matched with the small window data stream.

[0032] In some embodiments of the present application, the cropping direction can include horizontal cropping and vertical cropping. The cropping direction can correspond to the cropping size. For example, when the cropping direction is horizontal cropping, the corresponding cropping size is 1920px*1080px, and when the cropping direction is vertical cropping, the corresponding cropping size is 1080px*1920px. The cropping direction can also be represented in the form of numbers, characters, etc. For example, 0 represents horizontal cropping, and 1 represents vertical cropping. In some embodiments of the present application, the cropping size can be pre-set. For example, the cropping size is 1920px*1080px or 1080px*1920px. For example, the first cropping direction can be horizontal cropping, and the first cropping size can be 1920px*1080px. For example, the first cropping direction can be vertical cropping, and the first cropping size can be 1080px*1920px. Of course, the first cropping direction and the first cropping size can also be other contents, which are not limited in the present application. It can be understood that the first cropping direction and the first cropping size can be pre-set.

[0033] In some embodiments of the present application, the first image can be the small window image data mentioned in step S714. The second image can be the small window image data after rotation mentioned in step S715. The second close-up image can be the small window image data after processing mentioned in step S717.

[0034] In some embodiments of the present application, the first rotation direction can be rotation direction F1, and the first rotation angle can be J1. The second rotation direction can be rotation direction F2, and the first rotation angle can be J2. The related description of the rotation angle and the rotation direction can be referred to later, which is not expanded here.

[0035] It can be understood that the related description of the rotation information can be referred to later, which is not expanded here.

[0036] In some embodiments of the present application, after obtaining the second image, the method further includes: the camera HAL sends the second image to the Buffer; and the Buffer sends the second image to the display management module.

[0037] In combination with the first aspect, in a possible implementation manner, before displaying the first interface, the method further includes: a stream management module in the electronic device configures a first data stream and a second data stream, and sends stream configuration information to a camera FWK and a camera HAL in the electronic device. The camera FWK can determine the aspect ratio of the Buffer according to the stream configuration information. The camera HAL determines that the image effect matched with the second data stream is the second image effect according to the stream configuration information. The second image effect includes a second cropping direction, and the second cropping direction corresponds to a second cropping size.

[0038] In the scheme provided in the application, the electronic device can determine the aspect ratio of the Buffer according to the streaming information. In some embodiments of the application, the aspect ratio of the Buffer is the same as the default aspect ratio of the first window.

[0039] It can be understood that the streaming information can include Buffersize. Buffersize can be understood as the size of the image that the Buffer can accommodate. The description of the streaming information and Buffersize can be referred to later, and will not be expanded here.

[0040] It can be understood that the second image effect is the image effect matched with the small window data stream before the change of the aspect ratio of the preview small window.

[0041] It can be understood that the related content of the second cropping direction and the second cropping size can be referred to the description of the first cropping direction and the first cropping size in the foregoing. It should be noted that the first cropping direction is inconsistent with the second cropping direction, and the first cropping size is inconsistent with the second cropping size. For example, the first cropping direction can be horizontal cropping, and the first cropping size can be 1920px*1080px. The second cropping direction can be vertical cropping, and the second cropping size can be 1080px*1920px. Of course, the second cropping direction and the second cropping size can also be other content, which is not limited by the application. It can be understood that the second cropping direction and the second cropping size can be pre-set.

[0042] In combination with the first aspect, in a possible implementation manner, before detecting the second operation acting on the first control, the method further includes: the camera HAL processes the original image data based on the second image effect to obtain a first close-up image; and the display management module displays the first close-up image in the first window.

[0043] In the scheme provided in the application, the image effect matched with the small window data stream before and after the change of the aspect ratio of the preview small window is also different, for example, the cropping direction and the cropping size in the image effect are different. The electronic device processes the original image data based on the different image effects, thereby obtaining close-up images of different sizes. Further, the electronic device can record close-up videos of different proportions, which can meet the demand of the user for close-up videos of multiple proportions, and improves the user experience.

[0044] In combination with the first aspect, in a possible implementation manner, the first rotation direction and the second rotation direction are the same, the second rotation angle is 90 degrees, and the first rotation angle is 270 degrees.

[0045] In the scheme provided in the application, the first rotation direction can be the same as the second rotation direction. For example, the first rotation direction and the second rotation direction can both be the clockwise direction or the counterclockwise direction. The second rotation angle is 90 degrees, and the first rotation angle is 270 degrees.

[0046] In some embodiments of the present application, the electronic device can determine the corresponding rotation direction and rotation angle according to the positional relationship between the first window and the preview window. For example, when the aspect ratio of the first window is the first aspect ratio, the first window is parallel to the preview window. When the positional relationship between the first window and the preview window changes to vertical, the electronic device can determine that the first rotation direction is counterclockwise and the rotation angle is 270 degrees. It can be understood that the related description of the positional relationship can be referred to later, and will not be expanded here.

[0047] In some embodiments of the present application, the first rotation direction is opposite to the second rotation direction, and the first rotation angle is equal to the second rotation angle. For example, the first rotation direction is counterclockwise, the second rotation direction is clockwise, and the first rotation angle and the second rotation angle are both 90 degrees.

[0048] In some embodiments of the present application, in response to the user operation acting on the first control, the electronic device can adjust the aspect ratio of the first window from the second aspect ratio to the first aspect ratio. In this case, the camera mode module sends a message to the direction management module and the camera HAL. After the camera HAL receives the message, it updates the cropping direction. Correspondingly, the image effect corresponding to the small window data stream is also updated. The camera HAL processes the small window data stream based on the updated image effect, and uploads the processed image to the camera FWK. The image can be cached in the Buffer in the camera FWK. After receiving the message, the direction management module does not need to rotate the image transmitted by the Buffer, but directly displays it to the corresponding window.

[0049] In a second aspect, the present application provides an electronic device. The electronic device can include a camera, a display, a memory, and one or more processors. The memory is configured to store a computer program. The processor is configured to invoke the computer program to implement corresponding functions. The display can be configured to display a first interface. The first interface includes a preview window. The preview window displays a first image. The first image displays one or more markers. The markers are configured to mark a shooting object. The processor can be configured to detect a first operation acting on a first marker. The first marker is any one of the one or more markers. The display can be further configured to display, in response to the first operation, a first window on the preview window. The first window displays a close-up image of a first shooting object. The first shooting object corresponds to the first marker. The first window includes a first control. The processor can be further configured to detect a second operation acting on the first control, adjust a width-to-height ratio of the first window from a first width-to-height ratio to a second width-to-height ratio in response to the second operation, start recording a video in response to a third operation of starting recording, and save an original video and a close-up video in response to a fourth operation of ending recording. The original video is a video obtained based on the image displayed in the preview window, and the close-up video is a video obtained based on the image displayed in the first window.

[0050] With reference to the second aspect, in a possible implementation, the first window displays a first close-up image before the second operation acting on the first control is detected. The first window displays a second close-up image after the second operation acting on the first control is detected. The first close-up image has the first width-to-height ratio, and the second close-up image has the second width-to-height ratio.

[0051] With reference to the second aspect, in a possible implementation, the ratio of the first width-to-height ratio and the ratio of the second width-to-height ratio are reciprocals of each other.

[0052] With reference to the second aspect, in a possible implementation, a buffer area Buffer in a camera FWK of the electronic device has the first width-to-height ratio. The Buffer is configured to buffer image data sent to the camera FWK.

[0053] With reference to the second aspect, in a possible implementation manner, the electronic device further includes a direction management module, a display management module, and a camera HAL. After detecting the second operation acting on the first control, the direction management module can be configured to: in response to the second operation, send rotation information to the display management module and the camera HAL. The display management module can be configured to: determine a first rotation direction and a first rotation angle based on the rotation information; and the camera HAL can be configured to: update an image effect corresponding to the second data stream based on the rotation information to obtain a first image effect, and determine a second rotation direction and a second rotation angle; process the original image data based on the first image effect to obtain a first image; and perform rotation processing on the first image based on the second rotation direction and the second rotation angle to obtain a second image. The first image effect includes a first cropping direction. The first cropping direction corresponds to a first cropping size. The original image data is image data collected by the camera of the electronic device. The display management module can be configured to: perform rotation processing on the second image based on the first rotation direction and the first rotation angle to obtain a second close-up image, and display the second close-up image in the first window.

[0054] With reference to the second aspect, in a possible implementation manner, the electronic device further includes a stream management module and a camera FWK. Before displaying the first interface, the stream management module can be configured to: configure the first data stream and the second data stream, and send stream configuration information to the camera FWK and the camera HAL in the electronic device. The camera FWK can be configured to: determine the width-to-height ratio of the Buffer according to the stream configuration information. The camera HAL can be further configured to: determine, according to the stream configuration information, that the image effect matched with the second data stream is a second image effect. The second image effect includes a second cropping direction. The second cropping direction corresponds to a second cropping size.

[0055] With reference to the second aspect, in a possible implementation manner, before detecting the second operation acting on the first control, the camera HAL can be further configured to: process the original image data based on the second image effect to obtain a first close-up image. The display management module can be further configured to: display the first close-up image in the first window.

[0056] With reference to the second aspect, in a possible implementation manner, the first rotation direction and the second rotation direction are the same, the second rotation angle is 90 degrees, and the first rotation angle is 270 degrees.

[0057] In a third aspect, the present application provides a computer storage medium, including computer instructions, when the computer instructions run on an electronic device, the electronic device executes any one of the possible implementation manners of the first aspect.

[0058] In a fourth aspect, an embodiment of the present application provides a chip, which can be applied to an electronic device, and the chip comprises one or more processors configured to invoke computer instructions to cause the electronic device to perform any possible implementation manner of the first aspect.

[0059] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on an electronic device, cause the electronic device to perform any possible implementation manner of the first aspect.

[0060] It can be understood that the electronic device provided in the second aspect, the computer storage medium provided in the third aspect, the chip provided in the fourth aspect, and the computer program product provided in the fifth aspect are all used to perform any possible implementation manner of the first aspect. Therefore, the beneficial effects that can be achieved are referable to the beneficial effects of any possible implementation manner of the first aspect, which will not be described here again. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 A schematic diagram of a shooting scene is provided for an embodiment of the present application.

[0062] Figures 2A-2I A set of user interface schematic diagrams is provided for an embodiment of the present application.

[0063] Figure 3 A hardware structure schematic diagram of an electronic device is provided for an embodiment of the present application.

[0064] Figure 4 A software structure schematic diagram of an electronic device is provided for an embodiment of the present application.

[0065] Figure 5 A module interaction schematic diagram is provided for an embodiment of the present application.

[0066] Figure 6 A schematic diagram of image data cropping and rotation is provided for an embodiment of the present application.

[0067] Figure 7 A flowchart of a shooting method is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; the "and / or" in the text only represents a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0069] It should be understood that the terms "first", "second" and the like in the description of the present application and claims and drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0070] In the present application, referring to "embodiments" means that the specific features, structures or characteristics described in conjunction with the embodiments can be included in at least one embodiment of the present application. The phrase appears at various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in the present application can be combined with other embodiments.

[0071] The application provides a photographing method. The photographing method can be applied to an electronic device provided with a camera. According to the photographing method, in a photographing scene in which multiple photographing objects are in the same frame, the electronic device can focus on one photographing object and record a close-up video of the photographing object, and generate an original video and a close-up video respectively. In the case where the photographing direction of the electronic device does not change, the electronic device can change the aspect ratio of a window for photographing the close-up video, so as to obtain close-up videos of different proportions. In this process, the electronic device can change the cropping direction of the close-up image through a camera HAL, and rotate the cropped close-up image, so that the size of the close-up image transmitted to a Buffer meets the aspect ratio of the Buffer. The electronic device can also transmit the close-up image transmitted to the Buffer to a camera application, rotate the close-up image again through the camera application, and display the processed close-up image in the rotated window. The above method can not only meet the demand of a user for obtaining close-up images of multiple proportions, but also avoid the interruption of the camera path caused by the fact that the close-up image transmitted to the Buffer does not meet the aspect ratio of the Buffer, that is, the phenomenon of freezing or blurring can be avoided.

[0072] Please refer to Figure 1 , Figure 1 for a schematic diagram of a photographing scene provided by an embodiment of the application.

[0073] As shown in Figure 1 , there are multiple persons in the photographing environment, i.e., person R1, person R2 and person R3. The user can determine that the photographing subject is person R2, and use the electronic device to photograph. Since the photographing subject is in a motion state, the problem of out-of-focus may occur. Moreover, there are other persons (e.g., person R1 and person R3) in the photographing environment, which may cause the problem of focus shift. That is, the focus shifts from person R2 to other persons. In this case, the user can start the focus tracking mode and select person R2 as the focus tracking target. After the user selects person R2 as the focus tracking target, the user can click the video recording start control on the electronic device to record. As shown in Figure 1 , during the recording process, the preview large window of the electronic device displays the image captured by the camera based on the entire viewfinder range (containing the images of person R1, person R2 and person R3). The preview small window can also be displayed on the preview large window. The preview small window displays the close-up image of the focus tracking target (i.e., person R2). It can be understood that after the recording is completed, the electronic device can generate two videos, i.e., an original video and a close-up video. The original video is obtained based on the image displayed on the preview large window. The close-up video is obtained based on the close-up image of the focus tracking target displayed on the preview small window.

[0074] It is understood that the subject of the photograph mentioned in this application refers to the main subject being photographed when the user uses an electronic device. The framing is related to the camera parameters of the electronic device (e.g., field of view, focal length, etc.).

[0075] It is understood that electronic devices can specifically be mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), or dedicated cameras (e.g., SLR cameras, point-and-shoot cameras), etc. This application embodiment does not impose any restrictions on the specific type of electronic device.

[0076] The following section uses a set of user interface diagrams to illustrate the shooting scenarios provided in this application.

[0077] It is understood that the terms "interface" and "user interface" in the specification, claims, and drawings of this application refer to the medium interface through which an application or operating system interacts and exchanges information with the user, realizing the conversion between the internal form of information and a form acceptable to the user. A commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0078] 1. Enter focus tracking mode ( Figures 2A-2D )

[0079] like Figure 2A As shown, the user interface 100 displays a page with application icons, which may include multiple application icons (e.g., weather app icon, calendar app icon, photo album app icon, notepad app icon, email app icon, app store app icon, settings app icon, etc.). Below these application icons, a page indicator may also be displayed to indicate the positional relationship between the currently displayed page and other pages. Below the page indicator are multiple application icons (e.g., camera app icon 101, browser app icon, messaging app icon, dialer app icon). The application icons remain displayed when switching pages.

[0080] It is appreciated that the camera application icon 101 is an icon of a camera application (i.e., a camera application program). The camera application icon 101 can be used to trigger the launching of the camera application program.

[0081] The electronic device can detect a user operation acting on the camera application icon 101, and in response to the operation, the electronic device can display a user interface 200 as shown in Figure 2B The user interface 200 is a shooting interface of a default shooting mode of the camera application program, on which the user can preview an image and complete shooting.

[0082] It is appreciated that the user operations mentioned in the present application can include, but are not limited to, touch (e.g., click, etc.), voice control, gesture, etc., and the present application does not limit the same.

[0083] As shown in Figure 2B The user interface 200 can include a preview window 201, a camera mode option 202, an album shortcut control 203, a shutter control, and a camera flip control.

[0084] The preview window 201 can be used to display a preview image. The preview image displayed by the preview window 201 is an image captured by the camera of the electronic device based on the entire field of view. The preview window 201 can be the preview large window mentioned above.

[0085] One or more shooting mode options can be displayed in the camera mode option 202. The one or more shooting mode options can include a night scene mode option, a smart portrait mode option, a shooting mode option, a video recording mode option 2021, and more options. It is appreciated that more or fewer shooting mode options can also be included in the camera mode option 202.

[0086] The album shortcut control 203 can be used to open the album application program. After the user triggers the electronic device to open the album application program through the album shortcut control 203, the user can view the images and videos obtained by shooting. In addition, a thumbnail of the image or video obtained by shooting can also be displayed on the album shortcut control 203.

[0087] The electronic device can detect a user operation acting on the video recording mode option 2021. In response to the user operation, the electronic device can display a user interface 300 as shown in Figure 2C The user interface 300 is a video recording preview interface of the camera application program.

[0088] The controls included in the user interface 300 are substantially the same as those included in the user interface 200, except that the user interface 300 can further include a follow focus mode control 301 and a video recording start control 302. The follow focus mode control 301 can be used to trigger the electronic device to enter the follow focus mode. The video recording start control 302 is used to trigger the electronic device to start video recording.

[0089] The electronic device can detect a user operation acting on the follow focus mode control 301. In response to the user operation, the electronic device can display a user interface 400 as shown in FIG. 4A. The user interface 400 is a recording preview interface when the follow focus mode is entered but the follow focus has not started yet. Figure 2D

[0090] The user interface 400 can include a display area 401, a preview window 201, a human body frame 402, a human body frame 403, and a human body frame 404. The display area 401 can include a prompt information to enter the follow focus mode, and an operation prompt information to record a close-up video. As shown in FIG. 4A, the display area 401 displays "Tap a human body frame to additionally generate a close-up video", and "The follow focus mode has started". As shown in FIG. 4A, the preview window 201 can display an image including a person R1, a person R2, and a person R3. The human body frame 402 can be used to frame the person R1. The human body frame 403 can be used to frame the person R2. The human body frame 404 can be used to frame the person R3. Figure 2D Figure 2D

[0091] 2、Preview and select the follow focus target, and adjust the size of the preview window Figures 2E-2F

[0092] The electronic device can detect a user operation acting on the human body frame 403. In response to the user operation, the electronic device can display a user interface 500 as shown in FIG. 5A. The user interface 500 is a recording preview interface when the electronic device is in the follow focus mode and the follow focus is successful. Figure 2E

[0093] It can be understood that the electronic device can determine that the follow focus target is the person R2 after detecting the user operation acting on the human body frame 403. After the follow focus is successful, the electronic device can lock the focus on the person R2.

[0094] The user interface 500 can include the preview window 201, a preview window 501, a follow focus human body frame 502, and a recording start control 302.

[0095] The preview window 501 is used to display a close-up image of the follow focus target. That is, compared with the image displayed by the preview window 201, the image displayed by the preview window 501 is mainly the follow focus target. The preview window 501 can include a control 5011 and a control 5012. The control 5011 is used to exit the follow focus. The control 5012 is used to adjust the size of the preview window 501. Specifically, the control 5012 can be used to adjust the aspect ratio of the preview window 501. The aspect ratio of the preview window 501 refers to the ratio of the width and the height of the preview window 501.

[0096] ​​​​​In some embodiments of the present application, the aspect ratio of the preview window 501 refers to the size of the close-up image displayed by the preview window 501. For example, 1920px*1080px. In the case where the aspect ratio of the preview window 501 is 1920:1080, the width of the close-up image displayed by the preview window 501 is 1920px and the height is 1080px. The size of the close-up image is 1920px*1080px. It can be understood that the full name of px is "Pixel", which means "pixel" in Chinese, which is the smallest unit to represent pictures or graphics.

[0097] The focus human body frame 502 can be used to frame the focus target. As shown in Figure 2E The focus human body frame is used to frame the character R2.

[0098] It can be understood that the preview window 501 can be displayed floating on the preview window 201. In some embodiments of the present application, the user can drag the preview window 501 within the range of the preview window 201.

[0099] The electronic device can detect the user operation acting on the control 5012. In response to the user operation, the electronic device can display the user interface 600 as shown in Figure 2F The controls included in the user interface 600 are substantially the same as those included in the user interface 500, except that the aspect ratio of the preview window 501 in the user interface 600 and the user interface 500 changes.

[0100] In some embodiments of the present application, the size of the preview window 501 in the user interface 600 and the user interface 500 is the same, but the width of the preview window 501 in the user interface 600 is the same as the height of the preview window 501 in the user interface 500, and the height of the preview window 501 in the user interface 600 is the same as the width of the preview window 501 in the user interface 500.

[0101] It should be noted that the default size (including the default aspect ratio) of the preview window 501 can be set according to actual needs, which is not limited in the present application.

[0102] It can be understood that after starting the focus and successfully focusing, the electronic device can display the preview window 501 as shown in Figure 2E by default, or can display the preview window 501 as shown in Figure 2F by default.

[0103] Of course, the user interface 500 and the user interface 600 can also include a human body frame for framing a person who is not selected as a focus target.

[0104] In some embodiments of the present application, after the user selects the focus target and clicks the corresponding human body frame, if the user does not switch the focus target within a period of time, the electronic device can hide other human body frames that are not selected.

[0105] It should be noted that the electronic device can also perform focus preview in the horizontal screen state. In the case that the electronic device is in the horizontal screen state, once the focus on the person R2 is successful, the electronic device can display a user interface 700 as shown in Figure 2G The controls included in the user interface 700 are substantially the same as those included in the user interface 600, except that the user interface 700 can further include a control 701. The control 701 is used to indicate that the current preview window 201 and the preview small window 501 are parallel.

[0106] The electronic device can detect a user operation acting on the control 5012 included in the user interface 700. In response to the user operation, the electronic device can display a user interface 800 as shown in Figure 2H The controls included in the user interface 800 are substantially the same as those included in the user interface 700, except that the aspect ratio of the preview small window 501 in the user interface 800 and the user interface 700 changes, and the user interface 800 includes a control 801. The control 801 is used to indicate that the current preview window 201 and the preview small window 501 are vertical. The related description of the change of the aspect ratio of the preview small window 501 can be referred to the above, and will not be described here.

[0107] 3. Video recording in the focus success state Figure 2I

[0108] The electronic device can detect a user operation acting on the video start control 302 included in the user interface 800 as shown in Figure 2H In response to the user operation, the electronic device can start recording a video.

[0109] As shown in Figure 2I The electronic device can display a user interface 900. The user interface 900 can include a display area 901, a preview small window 501, a video end control 904, and a video pause control 905.

[0110] The display area 901 is used to display the video recording time corresponding to the preview window 201. That is, the recording time of the original video. As shown in Figure 2I The video recording time corresponding to the preview window 201 is “00:03”, that is, the recording time of the original video is 3s.

[0111] It can be understood that after the electronic device starts recording a video in the focus mode, the preview small window 501 can hide the control 5011 and the control 5012. As Figure 2I ​As shown, the preview window 501 can include a display area 902 and a recording end control 903.

[0112] The display area 902 is used to display the video recording time corresponding to the preview window 501, i.e., the recording time of the close-up video. Consistent with the video recording time displayed by the display area 901, the video recording time displayed by the display area 902 is also “00:05”, i.e., the recording time of the original video is 5s, and the recording time of the close-up video is also 5s.

[0113] It should be noted that the related meanings of the original video and the close-up video can be referred to the above, and will not be described here.

[0114] The recording end control 903 is used to end the recording of the video corresponding to the preview window 501, i.e., the close-up video. The recording end control 904 is used to end the recording of the video corresponding to the preview window 201, i.e., the original video. The recording pause control 905 is used to pause the recording of the original video.

[0115] It should be noted that the above user interface is only some examples provided by the present application, and should not be regarded as a limitation to the present application.

[0116] The following describes an apparatus related to an embodiment of the present application.

[0117] Figure 3 A hardware structure schematic diagram of an electronic device provided by an embodiment of the present application.

[0118] The electronic device can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charge management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0119] It can be understood that the structural schematic of the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device can include more or fewer components than those shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software or a combination of software and hardware.

[0120] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units can be independent devices, or can be integrated in one or more processors. Among them, the controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to instruction operation codes and timing signals, complete the control of fetching instructions and executing instructions. The processor 110 can also be provided with a memory for storing instructions and data.

[0121] In the embodiments provided in the present application, the electronic device can execute the photographing method through the processor 110.

[0122] The internal memory 121 can be used to store computer executable program codes, the executable program codes including instructions. The processor 110 executes various functions of the electronic device and data processing by running the instructions stored in the internal memory 121.

[0123] The antenna 1 and the antenna 2 are used for transmitting and receiving electromagnetic wave signals. The mobile communication module 150 can provide a solution for wireless communication including 2G / 3G / 4G / 5G, etc. applied to the electronic device. The wireless communication module 160 can provide a solution for wireless communication including Wireless Local Area Networks (WLAN) (such as Wireless Fidelity (Wi-Fi) network), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared technology (IR), etc. applied to the electronic device.

[0124] The electronic device implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0125] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can adopt a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), an Active-Matrix Organic Light Emitting Diode (AMOLED), a Flex Light-Emitting Diode (FLED), a Mini LED, a Micro LED, a Micro-OLED, a Quantum Dot Light Emitting Diodes (QLED), etc. In some embodiments, the electronic device can include 1 or N display screens 194, and N is a positive integer greater than 1.

[0126] The electronic device can implement an acquisition function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor, etc.

[0127] ISP is used to process the data feedback from the camera 193. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and is converted into an image or video visible to the naked eye. ISP can also optimize the noise, brightness, and color of the image. ISP can also optimize the exposure, color temperature, and other parameters of the shooting scene. In some embodiments, the ISP can be provided in the camera 193.

[0128] The camera 193 is used to capture still images or videos. Objects generate optical images through lenses and project them onto photosensitive elements. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image or video signal. The ISP outputs the digital image or video signal to the DSP for processing. The DSP converts the digital image or video signal into a standard RGB, YUV, or other format image or video signal. In some embodiments, the electronic device can include multiple cameras 193.

[0129] A video codec is used to compress or decompress digital video. The electronic device can support one or more video codecs. In this way, the electronic device can play or record videos in multiple encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, and the like.

[0130] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for shooting anti-shake.

[0131] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device in various directions (generally three axes). When the electronic device is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and applied to landscape / portrait switching, pedometers, and other applications.

[0132] Touch sensor 180K, also referred to as "touch panel". The touch sensor 180K can be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also referred to as "touch panel". The touch sensor 180K is configured to detect a touch operation acting on or near the touch sensor 180K. The touch sensor 180K can transmit the detected touch operation to the application processor to determine the touch event type. The visual output related to the touch operation can be provided through the display screen 194. In some other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device, which is different from the position where the display screen 194 is located.

[0133] It should be noted that, Figure 3 The functions of other modules not mentioned in the electronic device shown can be referred to the related technical documents, and the present application does not expand the description.

[0134] Figure 4 A software structure schematic diagram of an electronic device provided by the embodiments of the present application.

[0135] As Figure 4 The software framework of the electronic device involved in the present application can include an application layer, an application framework layer (FWK), a system library, an Android runtime, a hardware abstraction layer (HAL) and a kernel layer, as shown.

[0136] The application layer can include a series of application packages, such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications (also referred to as applications). The camera application can be used to acquire images and videos.

[0137] As Figure 4As shown, the camera application can include a camera mode module, a stream management module, a direction management module, and a display management module. The camera mode module can be used to listen to user operations and determine the mode of the camera. The mode of the camera can include, but is not limited to, a photographing mode, a video preview mode, a large window preview mode, and a large window video mode, etc. Among them, the video preview mode can include a video preview mode in a follow focus mode. The stream management module is used for data stream management. For example, the issuance of data stream configuration information (which can be referred to as stream configuration information for short). The stream management module can include an image buffer area, i.e., an area for data stream buffering. In the present application, the image buffer area can include a large window image buffer area and a small window image buffer area. The large window image buffer area and the small window image buffer area can store two-way image data returned by the camera HAL. Specifically, the large window image buffer area is used to buffer the data stream corresponding to the preview large window, and the small window image buffer area is used to buffer the data stream corresponding to the preview small window image. The direction management module is used to manage the cropping direction. It can be understood that the cropping direction belongs to part of the image effect. In the present application, the camera HAL can process the original image data according to the image effect, thereby obtaining large window image data and small window image data. The display management module is used to manage the display window for displaying the panoramic image and the close-up image. The display management module can also be used to adjust the angle of the image displayed in the display window. It can be understood that the panoramic image mentioned in the present application is an image captured by the electronic device through the camera based on its field of view, and is not an image that embodies the 360-degree shooting scene in the general sense. The field of view is related to the focal length and the field of view angle of the camera. In a popular way, the panoramic image includes more content in the shooting scene than the close-up image.

[0138] The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer includes some pre-defined functions.

[0139] As Figure 4As shown, the application framework layer can include a camera Fwk. The camera Fwk can provide an API interface for an application (e.g., a camera application) to call, receive a request from the application, maintain the business logic of the request internally, and finally send the request to a camera service for processing through a cross-process interface of a camera AIDL, and then wait for the return of the result of the camera service, and finally send the final result to the camera application. The full name of AIDL is Android Interface Definition Language, which means Android interface definition language. The camera Fwk can include a buffer area (Buffer). The Buffer can be understood as an image data container for caching image data transmitted to the camera Fwk.

[0140] It can be understood that the application framework layer can also include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc. Their specific meanings can be referred to related technical documents, which are not expanded here.

[0141] The runtime is responsible for the scheduling and management of the system. The runtime includes a core library and a virtual machine. The core library includes two parts: one part is the function function required by the programming language (e.g., java language) to call, and the other part is the core library of the system.

[0142] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the programming files (e.g., java files) of the application layer and the application framework layer into binary files. The virtual machine is used to perform functions such as management of object life cycle, stack management, thread management, security and exception management, and garbage collection.

[0143] The system library can include a plurality of functional modules. For example: a surface manager (Surface Manager), a media library (Media Libraries), a three-dimensional graphics processing library (e.g., OpenGL ES), a two-dimensional graphics engine (e.g., SGL), etc.

[0144] The surface manager is used to manage the display subsystem and provides a fusion of two-dimensional (2-Dimensional, 2D) and three-dimensional (3-Dimensional, 3D) layers for multiple applications.

[0145] The media library supports playback and recording of a variety of commonly used audio, video formats, and static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0146] A three-dimensional graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0147] A 2D graphics engine is a drawing engine for 2D drawing.

[0148] A hardware abstraction layer (HAL) is an interface layer between an operating system kernel and upper-layer software, and is used to abstract hardware. The hardware abstraction layer is an abstract interface of a device kernel driver, and is used to implement an application programming interface for providing access to underlying devices to a higher-level Java API framework. The HAL includes a plurality of library modules, such as a camera HAL, a display screen, Bluetooth, audio, and the like. Each of the library modules implements an interface for a specific type of hardware component. It can be understood that the camera HAL can provide an interface for accessing hardware components such as a camera for the camera FWK. When a system framework layer API requires access to hardware of a portable device, the Android operating system will load the library module for the hardware component.

[0149] A kernel layer is a basis of the Android operating system, and final functions of the Android operating system are completed through the kernel layer. The kernel layer at least includes a display driver, a camera driver, an audio driver, a sensor driver, and a virtual card driver.

[0150] It should be noted that the photographing method provided in the present application is not limited to the Android operating system, and can also be implemented based on other operating systems. Figure 4 The software structure diagram of the electronic device shown in the figure is only an example, and does not limit the specific module division in different layers of the Android operating system. For details, refer to the introduction of the software structure of the Android operating system in the conventional technology. In addition, the photographing method provided in the present application can also be implemented based on other operating systems, which will not be enumerated one by one.

[0151] Based on the software and hardware structure of the electronic device shown in Figure 3 and Figure 4 The photographing method provided in the embodiment of the present application is introduced in combination with the software and hardware structure of the electronic device shown in Figure 5 and Figure 6 The photographing method provided in the embodiment of the present application is introduced in combination with the software and hardware structure of the electronic device shown in

[0152] Please refer to Figure 5 , Figure 5 The photographing process diagram provided in the embodiment of the present application is introduced in combination with the software and hardware structure of the electronic device shown in

[0153] The camera mode module can listen to a user operation acting on the follow focus mode control, so as to determine that the current camera mode becomes the video preview mode in the follow focus mode, and notify the stream management module. Correspondingly, the stream management module can issue stream allocation information of two data streams to the camera FWK. The two data streams are a data stream corresponding to the preview large window and a data stream corresponding to the preview small window. After receiving the stream allocation information, the camera FWK can determine the aspect ratio of the Buffer according to the stream allocation information. The aspect ratio of the Buffer refers to the aspect ratio of the image data that the Buffer as an image data container can cache. The camera FWK can also issue the stream allocation information to the camera HAL. After receiving the stream allocation information, the camera HAL can parse the stream allocation information, and match corresponding image effects to the two data streams (i.e., the large window data stream and the small window data stream). It can be understood that the image effects matched to the small window data stream can include a cropping direction. In some embodiments of the present application, the image effects can also include a cropping size. It can be understood that after the camera HAL receives the original image data collected by the camera, the original image data can be processed according to the image effects matched to the large window data stream and the small window data stream, to obtain the large window image data and the small window image data, respectively, and the large window image data and the small window image data are transmitted back to the camera application.

[0154] The camera mode module can listen to a user operation acting on the preview small window size adjustment control (such as the control 5012 mentioned above), so as to determine that the current camera mode is still in the video preview mode in the follow focus mode, but the size of the preview small window needs to be adjusted, and notify the direction management module. The direction management module can determine the updated cropping direction, and send rotation information to the display management module and the camera HAL. The rotation information can include the cropping direction and the rotation direction. After receiving the rotation information, the display management module can rotate the preview small window according to the rotation information. After receiving the rotation information, the camera HAL can update the cropping direction in the image effects corresponding to the small window data stream according to the rotation information. After the follow focus is successful, the camera HAL can process the original image data according to the updated cropping direction to obtain the small window image data. The camera HAL can also rotate the small window image data, and transmit the rotated small window image data to the Buffer. The Buffer transmits the rotated small window image data to the display management module. It should be noted that after receiving the rotation information, the display management module can also determine the rotation direction and the rotation angle of the image data. After receiving the rotated small window image data, the display management module can rotate the image data according to the rotation direction and the rotation angle, and display the processed small window image data in the preview small window.

[0155] Please refer to Figure 6 , Figure 6 A schematic diagram of image data cropping and rotation is provided for embodiments of the present application.

[0156] According to the foregoing, after the display management module receives the rotation information, the display management module can rotate the preview widget according to the rotation information. As shown in Figure 6 , the initial aspect ratio of the preview widget is b1. The display management module can rotate the preview widget by 90 degrees. The aspect ratio of the rotated preview widget is b2.

[0157] In some embodiments of the present application, b1 is 1920:1080, and b2 is 1080:1920.

[0158] According to the foregoing, after the camera HAL receives the rotation information, the camera HAL can update the cropping direction corresponding to the widget data stream. As shown in Figure 6 , the camera HAL can obtain the widget image data after processing the original image data according to the image effect (including the cropping direction) corresponding to the widget data stream. The aspect ratio of the widget image data is b1. The camera HAL can obtain the widget image data after processing the original image data according to the updated cropping direction. The aspect ratio of the obtained widget image data is b2. The camera HAL can rotate the widget image data with the aspect ratio of b2 counterclockwise by 90 degrees. The aspect ratio of the rotated widget image data becomes b1. The camera HAL can send the rotated widget image data into the Buffer. As shown in Figure 6 , the aspect ratio of the Buffer is b1. That is, the image data sent into the Buffer meets the aspect ratio requirement of the Buffer. The Buffer can send the rotated widget image data into the display management module. According to the foregoing, after the display management module receives the rotation information, the display management module can also determine the rotation direction and the rotation angle of the image data. For example, the display management module can determine that the rotation direction of the image data is counterclockwise and the rotation angle is 270 degrees. As shown in Figure 6 , the display management module can rotate the rotated widget image data counterclockwise by 270 degrees to obtain the re-rotated widget image data. It can be understood that the aspect ratio of the re-rotated widget image data is b2. The display management module can send the re-rotated widget image data into the preview widget. It can be understood that the aspect ratio of the current preview widget is b2, which is consistent with the aspect ratio of the re-rotated widget image data.

[0159] It should be noted that the meanings of the large widget and the preview large widget mentioned in the present application are the same, and the meanings of the small widget and the preview small widget are the same.

[0160] The specific implementation of the foregoing embodiments will be described below in conjunction with Figure 7 .

[0161] It can be understood that the user can trigger the start of the camera application (for example, click the camera application icon). The camera mode module can listen to the user operation and send message X1 to the camera. Message X1 is used to request the start of the camera.

[0162] I. Configure data streams (e.g., step S701-step S705)

[0163] The user can trigger the camera to enter the follow focus mode (e.g., click the follow focus mode control). It can be understood that the camera mode module can listen to the corresponding triggering operation of the user and notify other modules in the camera application.

[0164] S701: The camera mode module sends message X2 to the stream management module.

[0165] It can be understood that message X2 is used to prompt the mode change of the camera application to other modules in the camera application. The mode change mentioned here can specifically refer to the camera application about to change from the video recording mode to the follow focus mode.

[0166] Correspondingly, the stream management module can receive the message X2 sent by the camera mode module.

[0167] S702: The stream management module configures the large window data stream and the small window data stream.

[0168] It can be understood that the large window data stream refers to the data stream corresponding to the preview large window, i.e., the data stream corresponding to the image displayed by the preview large window. The small window data stream refers to the data stream corresponding to the preview small window, i.e., the data stream corresponding to the image displayed by the preview small window.

[0169] It can be understood that the configuration information of the large window data stream and the small window data stream can include format, resolution, and (buffer size), etc. It should be noted that Buffer size refers to the size of the image data that the Buffer can accommodate. In some embodiments of the present application, the configuration information of the large window data stream and the small window data stream is the same. For example, the format can be YUV. The resolution can be 1920px*1080px. The Buffer size can be 1920px*1080px.

[0170] It should be noted that the specific content of the configuration information can be set according to industry regulations and actual needs, which is not limited in the present application.

[0171] It should be noted that the stream management module can include the buffer area corresponding to the large window data stream and the small window data stream.

[0172] S703: The stream management module sends the configuration information to the camera FWK and the camera HAL.

[0173] It can be understood that according to the above, the configuration information can include format, resolution, etc., which is not limited in the present application.

[0174] Correspondingly, the camera FWK and the camera HAL can receive the streaming information sent by the streaming management module.

[0175] S704: The camera FWK determines the Buffer aspect ratio according to the streaming information.

[0176] After the camera FWK receives the streaming information sent by the streaming management module, the camera FWK can determine the Buffer aspect ratio according to the Buffer size in the streaming information. The Buffer aspect ratio refers to the aspect ratio of the image data that the Buffer as an image data container can accommodate. For example, the Buffer size in the streaming information received by the camera FWK is 1920px*1080px, and the camera FWK can determine the Buffer aspect ratio as 1920:1080 according to the Buffer size. It can be understood that the Buffer can accommodate image data with a width of 1920px and a height of 1080px, and cannot accommodate image data with other aspect ratios. For example, the Buffer cannot accommodate image data with an aspect ratio of 1080:1920 (for example, a width of 1080px and a height of 1920px).

[0177] S705: The camera HAL determines the image effects corresponding to the large window data stream and the small window data stream according to the streaming information. The image effects corresponding to the small window data stream include the cropping direction.

[0178] After the camera HAL receives the streaming information, the camera HAL can determine the image effects corresponding to the large window data stream and the small window data stream according to the streaming information. The image effects can include the cropping direction, and can also include the cropping size, etc. It can be understood that the cropping direction can include horizontal cropping and vertical cropping. In some embodiments of the present application, the cropping size is set in advance. For example, the cropping size is 1920px*1080px or 1080px*1920px. In this case, when the cropping direction is horizontal cropping, the corresponding cropping size is 1920px*1080px, and when the cropping direction is vertical cropping, the corresponding cropping size is 1080px*1920px. Of course, the above-mentioned cropping direction and cropping size are only examples provided by the present application, and should not be regarded as a limitation on the present application.

[0179] In some embodiments of the present application, the camera HAL can determine the cropping direction corresponding to the small window data stream according to the Buffer size in the streaming information. For example, in the case of Buffer size of 1920px*1080px, the camera HAL can determine that the cropping direction corresponding to the small window data stream is horizontal cropping, and correspondingly, the cropping size is 1920px*1080px.

[0180] In some embodiments of the present application, the electronic device can pre-set the cropping direction corresponding to the small window data stream. For example, the electronic device can pre-set the cropping direction corresponding to the small window data stream as horizontal cropping, and the corresponding cropping size is 1920px*1080px. It can be understood that the pre-set cropping direction corresponding to the small window data stream corresponds to the size of the preview small window after successful tracking focus. For example, the size of the preview small window after successful tracking focus is 1920px*1080px, the pre-set cropping direction corresponding to the small window data stream is horizontal cropping, and the cropping size is 1920px*1080px.

[0181] II. Displaying a large window when tracking focus is successful (e.g., steps S706-S709)

[0182] The user can trigger the camera to enter the large window preview mode (e.g., click the human body frame). It can be understood that the camera mode module can listen to the corresponding triggering operation of the user and notify other modules in the camera application.

[0183] S706: The camera mode module sends message X3 to the camera HAL.

[0184] It can be understood that message X3 is used to prompt the mode change of the camera application to other modules in the camera application. The mode change mentioned here can specifically refer to the camera application about to change from the tracking focus mode to the large window preview mode.

[0185] Correspondingly, the camera HAL can receive the message X3 sent by the camera mode module. It can be understood that after receiving the message X3, the camera HAL can track the shooting object corresponding to the human body frame selected by the user.

[0186] S707: In the case of successful tracking focus, the camera HAL processes the original image data according to the image effects corresponding to the large window data stream and the small window data stream to obtain large window image data and small window image data. The original image data is the image data sent by the camera to the camera HAL.

[0187] It can be understood that in the case of successful tracking focus, the camera HAL processes the original image data according to the image effects corresponding to the large window data stream and the small window data stream to obtain large window image data and small window image data. It can be understood that the camera HAL can perform cropping processing on the original image data according to the cropping direction corresponding to the small window data stream and the cropping size to obtain the small window image data.

[0188] For example, the size of the small window image data is 1920px*1080px, that is, the width of the small window image data is 1920px and the height is 1080px.

[0189] Of course, in addition to the cropping processing, the camera HAL can also process the original image data according to the image effect (for example, add filters, beautify, etc.), which is not limited in the present application.

[0190] S708: The camera HAL sends the large window image data and the small window image data to the display management module.

[0191] The camera HAL sends the processed large window image data and small window image data to the display management module.

[0192] Correspondingly, the display management module can receive the large window image data and the small window image data sent by the camera HAL.

[0193] S709: The display management module displays the large window image data and the small window image data in the preview large window and the preview small window, respectively.

[0194] After the display management module receives the large window image data and the small window image data sent by the camera HAL, it can display the large window image data in the preview large window and the small window image data in the preview small window.

[0195] III. Adjusting the size of the small window (e.g., steps S710-S717)

[0196] The user can trigger the camera to enter the large-small window preview mode (e.g., click on the human body frame). It can be understood that the camera mode module can listen to the corresponding trigger operation of the user and notify other modules in the camera application.

[0197] S710: The camera mode module sends message X4 to the direction management module and the camera HAL.

[0198] It can be understood that message X4 is used to prompt the mode change of the camera application to other modules in the camera application. The mode change mentioned here can specifically refer to the change in the size of the preview small window of the camera application in the large-small window preview mode.

[0199] Correspondingly, the direction management module and the camera HAL can receive the message X4 sent by the camera mode module.

[0200] S711: The direction management module sends the rotation information to the display management module.

[0201] After the direction management module receives the message X4, the direction management module can determine the rotation information and send the rotation information to the display management module. It can be understood that the rotation information can include the positional relationship between the preview small window and the preview large window. In some embodiments of the present application, the positional relationship can include 0 or 1. When the positional relationship is 0, it indicates that the preview small window is parallel to the preview large window. When the positional relationship is 1, it indicates that the preview small window is perpendicular to the preview large window. It can be understood that the preview small window being parallel to the preview large window specifically includes that the aspect ratio of the preview small window is consistent with the aspect ratio of the preview large window. The preview small window being perpendicular to the preview large window specifically includes that the aspect ratio of the preview small window is the inverse of the aspect ratio of the preview large window. For example, when the aspect ratio of the preview small window and the aspect ratio of the preview large window are both 1920:1080, the preview small window is parallel to the preview large window. When the aspect ratio of the preview small window is 1080:1920 and the aspect ratio of the preview large window is 1920:1080, the preview small window is perpendicular to the preview large window.

[0202] Of course, the positional relationship can also have other representations, which are not limited in the present application.

[0203] Correspondingly, the display management module can receive the rotation information.

[0204] S712: The display management module rotates the preview small window according to the rotation information, and determines a rotation direction F1 and a rotation angle J1. The rotation direction F1 and the rotation angle J1 are used for rotating the image data sent to the display management module.

[0205] After the display management module receives the rotation information, the display management module can rotate the preview small window according to the rotation direction in the rotation information, and determine a rotation direction F1 and a rotation angle J1. The rotation direction F1 and the rotation angle J1 are used for rotating the image data sent to the display management module.

[0206] In some embodiments of the present application, the display management module can obtain an initial positional relationship of the preview small window and the preview large window. The initial positional relationship refers to the positional relationship between the preview small window and the preview large window displayed by the electronic device when the focus tracking is successful. For example, in the case of the initial positional relationship being 0, the preview small window displayed by the electronic device is parallel to the preview large window when the focus tracking is successful. For another example, in the case of the initial positional relationship being 1, the preview small window displayed by the electronic device is perpendicular to the preview large window when the focus tracking is successful.

[0207] In some embodiments of the present application, the rotation direction F1 is pre-set. For example, the electronic device pre-sets the rotation direction F1 as the counterclockwise direction, or the electronic device pre-sets the rotation direction F1 as the clockwise direction. In some embodiments of the present application, the rotation direction F1 is fixed. That is, if the rotation direction F1 is pre-set as counterclockwise, the electronic device can only rotate the image data counterclockwise in subsequent rotation.

[0208] For example, in the case that the initial position relationship is 0, if the position relationship in the rotation information received by the display management module is 1, the display management module can determine to rotate the preview sub-window by 90 degrees, and determine that the rotation direction F1 is counterclockwise and the rotation angle J1 is 270 degrees. It can be understood that the display management module can rotate the preview sub-window by 90 degrees counterclockwise or 90 degrees clockwise.

[0209] For example, in the case that the initial position relationship is 1, if the position relationship in the rotation information received by the display management module is 0, the display management module can determine to rotate the preview sub-window by 90 degrees, and determine that the rotation direction F1 is counterclockwise and the rotation angle J1 is 270 degrees. It can be understood that the display management module can rotate the preview sub-window by 90 degrees counterclockwise or 90 degrees clockwise.

[0210] S713: The camera HAL updates the cropping direction according to the rotation information, and obtains the image effect corresponding to the updated sub-window data stream.

[0211] After the camera HAL receives the rotation information sent by the direction management module, the camera HAL can update the cropping direction according to the position relationship in the rotation information, and obtain the image effect corresponding to the updated sub-window data stream.

[0212] In some embodiments of the present application, the camera HAL can obtain the initial position relationship. The related description of the initial position relationship can be referred to the above, and will not be described here.

[0213] For example, in the case that the initial position relationship is 0, if the position relationship in the rotation information received by the camera HAL is 1, the camera HAL determines that the cropping direction changes from horizontal cropping to vertical cropping, and correspondingly, the cropping size changes from 1920px*1080px to 1080px*1920px.

[0214] For example, in the case that the initial position relationship is 1, if the position relationship in the rotation information received by the camera HAL is 0, the camera HAL determines that the cropping direction changes from vertical cropping to horizontal cropping, and correspondingly, the cropping size changes from 1080px*1920px to 1920px*1080px.

[0215] It can be understood that the image effect corresponding to the updated sub-window data stream can include the updated cropping direction and cropping size, and can also include filters, beautification, etc.

[0216] S714: The camera HAL processes the original image data according to the image effect corresponding to the large window data stream and the image effect corresponding to the updated sub-window data stream, to obtain large window image data and sub-window image data.

[0217] After the image effect corresponding to the small-window data stream is updated, the camera HAL can process the original image data according to the image effect corresponding to the large-window data stream and the image effect corresponding to the updated small-window data stream to obtain the large-window image data and the small-window image data. It can be understood that the camera HAL can perform cropping processing on the original image data according to the updated cropping direction to obtain the small-window image data.

[0218] For example, the size of the small-window image data is 1080px*1920px, that is, the width of the small-window image data is 1080px and the height is 1920px.

[0219] It is worth noting that the size of the small-window image data obtained by the camera HAL after processing the original image data in step S714 and step S707 is different. For example, in step S707, the size of the small-window image data obtained by the camera HAL is 1920px*1080px, while in step S714, the size of the small-window image data obtained by the camera HAL is 1080px*1920px.

[0220] S715: The camera HAL determines the rotation direction F2 and the rotation angle J2 according to the rotation information, and rotates the small-window image data according to the rotation direction F2 and the rotation angle J2 to obtain the rotated small-window image data.

[0221] The camera HAL can determine the rotation direction F2 and the rotation angle J2 according to the position relationship in the rotation information, and rotate the small-window image data according to the rotation direction F2 and the rotation angle J2 to obtain the rotated small-window image data.

[0222] In some embodiments of the present application, the rotation direction F2 is pre-set. For example, the electronic device pre-sets the rotation direction F2 as counterclockwise, or the electronic device pre-sets the rotation direction F2 as clockwise. In this case, the camera HAL can determine the rotation angle of the small-window image data according to the initial position relationship and the position relationship in the rotation information, and rotate the small-window image data according to the angle and the pre-set rotation direction F2. In some embodiments of the present application, the rotation direction F2 is fixed. That is, if the rotation direction F2 is pre-set as counterclockwise, the electronic device can only rotate counterclockwise when rotating the image data subsequently. It can be understood that the pre-set rotation direction F1 and the rotation direction F2 of the electronic device can be the same.

[0223] For example, in the case of the initial position relationship being 0, if the position relationship in the rotation information received by the camera HAL is 1, the camera HAL can determine that the rotation angle J2 is 90 degrees and the pre-set rotation direction F2 is counterclockwise. It can be understood that the camera HAL can rotate the small-window image data counterclockwise by 90 degrees.

[0224] Exemplarily, in the case that the initial position relationship is 1, if the position relationship in the rotation information received by the camera HAL is 0, the camera HAL can determine that the rotation angle J2 is 90 degrees, and the rotation direction F2 set in advance is counterclockwise. It can be understood that the camera HAL can rotate the small window image data counterclockwise by 90 degrees.

[0225] Exemplarily, the size of the small window image data before rotation is 1080px*1920px, and the size of the small window image data after rotation is 1920px*1080px.

[0226] S716: The camera HAL sends the large window image data and the small window image data after rotation to the display management module.

[0227] The camera HAL sends the large window image data and the small window image data after rotation to the display management module.

[0228] Correspondingly, the display management module can receive the large window image data and the small window image data after rotation sent by the camera HAL.

[0229] S717: The display management module rotates the small window image data after rotation according to the rotation direction and the rotation angle of the image data, and displays the small window image data after rotation in the preview small window and displays the large window image data in the preview large window.

[0230] The display management module can rotate the small window image data after rotation again according to the rotation direction F1 and the rotation angle J1, and display the small window image data after rotation again in the preview small window and display the large window image data in the preview large window.

[0231] Exemplarily, the size of the small window image data before rotation is 1080px*1920px. The camera HAL can rotate the small window image data counterclockwise by 90 degrees to obtain the small window image data after rotation. The size of the small window image data after rotation is 1920px*1080px. It can be understood that according to the above, the display management module can determine that the rotation direction F1 is counterclockwise and the rotation angle J1 is 270 degrees. The display management module can rotate the small window image data after rotation counterclockwise by 270 degrees to obtain the small window image data after rotation again. The size of the small window image data after rotation again is 1080px*1920px.

[0232] The above examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A photographing method characterized by comprising: The method is applied to an electronic device, and the method comprises: displaying a first interface; the first interface comprises a preview window; the preview window displays a first image; the first image displays one or more markers; the markers are used for marking a shooting object; detecting a first operation acting on a first marker; the first marker is any one of the one or more markers; in response to the first operation, a first window is displayed on the preview window; the first window displays a close-up image of a first shooting object; the first shooting object is a shooting object corresponding to the first marker; the first window comprises a first control; detecting a second operation acting on the first control; in response to the second operation, the aspect ratio of the first window is adjusted from a first aspect ratio to a second aspect ratio; based on the collected original image data, two rotation processes are performed; the aspect ratio of an image obtained after a first rotation process in the two rotation processes is consistent with the aspect ratio of an image cached in a buffer area Buffer; the aspect ratio of the image obtained after the two rotation processes is the second aspect ratio; in response to a third operation of starting video recording, video recording is started; in response to a fourth operation of ending video recording, an original video and a close-up video are saved; the original video is a video obtained based on an image displayed in the preview window; the close-up video is a video obtained based on an image displayed in the first window; wherein, when the aspect ratio of the first window changes, the cropping method for the original image data changes accordingly.

2. The method of claim 1, wherein, Before the second operation acting on the first control is detected, the first window displays a first close-up image; after the second operation acting on the first control is detected, the first window displays a second close-up image; the aspect ratio of the first close-up image is the first aspect ratio; the aspect ratio of the second close-up image is the second aspect ratio.

3. The method of claim 1 or 2, wherein, The ratio of the first aspect ratio and the ratio of the second aspect ratio are reciprocals of each other.

4. The method of claim 1 or 2, wherein, The aspect ratio of a buffer area Buffer in a camera FWK of the electronic device is the first aspect ratio; the Buffer is used for caching image data sent to the camera FWK.

5. The method of claim 4, wherein, After the second operation acting on the first control is detected, the method further comprises: in response to the second operation, a direction management module in the electronic device sends rotation information to a display management module and a camera HAL in the electronic device; the display management module determines a first rotation direction and a first rotation angle based on the rotation information; the camera HAL updates an image effect corresponding to a second data stream based on the rotation information, obtains a first image effect, and determines a second rotation direction and a second rotation angle; the first image effect comprises a first cropping direction; the first cropping direction corresponds to a first cropping size; the camera HAL processes original image data based on the first image effect to obtain a first image; the original image data is image data collected by a camera of the electronic device; The camera HAL performs rotation processing on the first image based on the second rotation direction and the second rotation angle, to obtain a second image; The display management module performs rotation processing on the second image based on the first rotation direction and the first rotation angle, to obtain a second close-up image, and displays the second close-up image in the first window.

6. The method of claim 5, wherein, Before the first interface is displayed, the method further includes: A flow management module in the electronic device configures a first data stream and the second data stream, and sends stream configuration information to a camera FWK and a camera HAL in the electronic device; The camera FWK determines an aspect ratio of the Buffer according to the stream configuration information; The camera HAL determines, according to the stream configuration information, that an image effect matched with the second data stream is a second image effect; the second image effect includes a second cropping direction; the second cropping direction corresponds to a second cropping size.

7. The method of claim 6, wherein, Before the second operation acting on the first control is detected, the method further includes: The camera HAL performs processing on original image data based on the second image effect, to obtain a first close-up image; The display management module displays the first close-up image in the first window.

8. The method according to any one of claims 5 to 7, wherein, The first rotation direction and the second rotation direction are the same, the second rotation angle is 90 degrees, and the first rotation angle is 270 degrees. 9.An electronic device comprising a camera, a display screen, a memory, and one or more processors, wherein, The memory is configured to store computer programs; and the processor is configured to invoke the computer programs, so that the electronic device executes the method in any one of claims 1-8.

10. A computer storage medium, characterized in that comprise: Computer instructions; when the computer instructions run on an electronic device, the electronic device executes the method in any one of claims 1-8.

Images