Photographing method, electronic device, and readable storage medium

Abstract

The present application relates to the technical field of smart terminals, and particularly relates to a photographing method, an electronic device, and a readable storage medium. The photographing method is applied to an electronic device. When a first image, which is obtained by a user performing a first instance of photographing by means of the electronic device, does not meet the photographing requirements, the electronic device may prompt the user to perform a second instance of photographing. When the user determines to perform the second instance of photographing, prompt information related to an area to be optimized, which area does not meet the photographing requirements, in the first image is displayed, so as to instruct the user on whether it is necessary to perform adjustment. Moreover, when the electronic device determines that the area to be optimized meets a photographing condition, the prompt information is displayed to assist the user in completing the second instance of photographing. In this way, the electronic device may guide the user to perform multiple instances of photographing to make up for the defects of a single instance of photographing, and assist the user in capturing satisfactory images, thereby improving the quality of a finally captured image.

Description

Shooting method, electronic device and readable storage medium

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on January 12, 2024, with application number 2024100525200 and application name “A shooting method, electronic device and readable storage medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the technical field of smart terminals, and in particular to a shooting method, an electronic device, and a readable storage medium. Background Art

[0003] When users use electronic devices to take photos, it is difficult to take a satisfactory photo in one go. For example, in scenes such as busy scenic spots or group photos, multiple shots are usually required. Alternatively, users need to use image editing software to perform detailed post-processing on the captured image, such as cropping the image or removing irrelevant elements in the image, in order to obtain a satisfactory photo. However, photos obtained through post-processing using image editing software may suffer from problems such as loss of effective information and poor image quality. These problems often affect the quality of the image, resulting in users being unable to obtain high-quality images and affecting the user's shooting experience.

[0004] Summary of the Invention

[0005] To solve the above problems, embodiments of the present application provide a shooting method, an electronic device, and a readable storage medium.

[0006] In a first aspect, an embodiment of the present application provides a shooting method, which is applied to an electronic device, comprising: in response to a first shooting instruction, obtaining a first image, and displaying a first prompt message prompting a user to shoot again; in response to a re-shooting instruction, displaying a shooting interface, wherein the shooting interface includes a preview screen and second prompt message related to the area to be optimized of the first image; wherein the preview screen is used to display a preview image that changes in real time; corresponding to the area to be optimized meeting the shooting conditions, displaying a third prompt message to the user to shoot; in response to a second shooting instruction, obtaining a second image, the second image including a fused image after image synthesis processing of the preview image and the first image.

[0007] In the above solution, if the first image captured by the user using the electronic device does not meet the shooting requirements, the electronic device can prompt the user to take a second shot. When the user decides to take the second shot, prompt information related to the area to be optimized in the first image that does not meet the shooting requirements is displayed to guide the user whether adjustments are necessary. If the electronic device determines that the area to be optimized meets the shooting conditions, it will display prompt information to assist the user in completing the second shot. The image captured by the user in the second shot is a fusion image of the first image captured in the first shot and the preview image during the second shot.

[0008] This allows the user to take multiple shots to compensate for the shortcomings of a single shot, guiding them to shoot at the right moment and helping them capture satisfactory images. Furthermore, fused images can incorporate real-world information from different shooting environments at different times and spaces, preserving more effective information and ultimately improving the quality of the final image.

[0009] In some embodiments of the first aspect above, a fused picture is displayed in the shooting interface, and the second prompt information is displayed in the fused picture or the preview picture.

[0010] The fused screen is used to display the fused image, so that the electronic device can provide real-time shooting feedback to the user.

[0011] In addition, the fused image can be displayed in different ways in the shooting interface according to the different forms of the electronic device. For example, for a foldable electronic device, the shooting interface may include a main screen interface and a secondary screen interface. Then, the preview image can be displayed in the main screen interface and the fused image can be displayed in the secondary screen interface. For another example, for a flat-screen electronic device, the fused image can be displayed in the shooting interface in a picture-in-picture format.

[0012] In some embodiments of the first aspect, the fused image does not overlap with the preview image; or the fused image at least partially overlaps with the preview image.

[0013] It is understood that for foldable electronic devices, the fused screen and the preview screen do not overlap. For flat-screen electronic devices, the fused screen and the preview screen at least partially overlap. The overlapping area can be set according to the actual application, and the fused screen should not block the preview screen.

[0014] In some embodiments of the first aspect above, the area to be optimized includes a first area obtained by performing image detection on a first image; the shooting method also includes: determining first prompt information and second prompt information based on the image detection result of the first image; wherein the second prompt information includes shooting adjustment suggestions related to the first area.

[0015] It will be understood that the electronic device performs image detection on the first image to obtain an image detection result for the first image. Here, the primary purpose of the image detection is to determine whether a defective area, i.e., the first area to be optimized, exists. Therefore, when the image detection result indicates the presence of the first area, a first prompt message prompting the user to retake the image is generated and displayed. Furthermore, based on the defect in the first area, a second prompt message containing shooting adjustment suggestions related to the first area is generated and displayed.

[0016] In some embodiments of the first aspect above, the shooting method also includes: determining the scene type of the first image; performing image detection on the first image according to a quality scoring model corresponding to the scene type to obtain an image detection result of the first image, the image detection result of the first image including a first score; corresponding to the first score being less than or equal to a first threshold, the image detection result of the first image includes location information of the first area.

[0017] It is understandable that images captured in different shooting scenes may have different defects. Therefore, the electronic device identifies the scene type of the first image and uses different quality scoring models to perform image detection based on different scene types. Different quality scoring models correspond to different detection principles or detection algorithms.

[0018] For example, when the electronic device identifies that the scene type of the first image is a pedestrian removal scene, the quality scoring model may use a pedestrian detection algorithm, a subject area detection algorithm, an image registration algorithm, and an image difference algorithm, etc. For another example, when the electronic device identifies that the scene type of the first image is an uneven lighting scene, the quality scoring model may evaluate each pixel based on image parameters such as contrast, saturation, and brightness. For another example, when the electronic device identifies that the scene type of the first image is a group photo scene, the quality scoring model may use a face detection algorithm, an expression recognition algorithm, etc. For another example, when the electronic device identifies that the scene type of the first image is a telephoto scene, the quality scoring model may use a subject truncation detection algorithm to determine whether there is subject truncation in the shooting target.

[0019] The quality scoring model takes an image as input and outputs an aesthetic quality score of the image. It is understood that if the aesthetic quality score is low, indicating the presence of a first region, the output of the quality scoring model may also include location information of the first region, such as the pixel coordinates of the first region in the image.

[0020] In some embodiments of the first aspect described above, displaying first prompt information prompting the user to take another photo includes: displaying the first area with a first display element in a first image based on position information of the first area, thereby forming the first prompt information prompting the user to take another photo.

[0021] It can be understood that the first area of ​​the present application can be represented by a first display element, and the first display element has attributes such as shape and color.

[0022] In some embodiments of the first aspect above, the shooting method further includes: corresponding to the first area satisfying the shooting condition, changing the attribute of the first display element, where the attribute includes at least one of shape and color.

[0023] It is understandable that the electronic device can eliminate or transform the first display element, for example, when the first area meets the shooting condition, change the shape and / or color of the first display element so that the user can promptly understand whether the first area has met the shooting condition.

[0024] In some embodiments of the first aspect above, the shooting method further includes: in response to a first operation of the user adding an area to be optimized, displaying a second area corresponding to the first operation using a second display element in the first image.

[0025] It is understood that, in addition to the first area automatically detected by the electronic device, the user can manually add a second area to the first image. The first operation can be the manual adding operation mentioned in this application.

[0026] The second area is represented by a second display element, which has attributes such as shape and color. The shape and color of the first display element and the second display element may be the same or different.

[0027] In some embodiments of the first aspect above, the shooting method further includes: determining whether the area to be optimized meets the shooting conditions based on the image detection result of the fused image; the image detection result of the fused image includes a second score of the fused image; corresponding to the second score being greater than or equal to a second threshold, determining that the area to be optimized meets the shooting conditions.

[0028] It is understood that the electronic device may input the fused image into a quality scoring model to determine a second score for the fused image. The electronic device may determine whether the area to be optimized meets the shooting conditions based on the second score. If the second score is greater than or equal to a second threshold, the electronic device may determine that the area to be optimized meets the shooting conditions.

[0029] In some embodiments of the first aspect above, the shooting method further includes: determining whether the area to be optimized meets the shooting conditions based on the display changes of the preview image in the preview screen; and determining that the area to be optimized meets the shooting conditions when there is a display change in the area corresponding to the area to be optimized in the preview image.

[0030] The display change can be directly observed by the user. For example, when a pedestrian in the area to be optimized leaves, the user can observe the display change and determine that the area to be optimized meets the shooting conditions.

[0031] Alternatively, the electronic device can determine whether there is a display change in the preview image based on the area to be optimized in the fused image and the first image. For example, if there are pedestrians in the area to be optimized in the first image, but no pedestrians in the area to be optimized in the fused image, it means that there is a display change, that is, it is determined that the area to be optimized meets the shooting conditions.

[0032] In a second aspect, an embodiment of the present application provides an electronic device comprising: one or more processors, one or more memories, wherein the one or more memories store one or more programs, and when the one or more programs are executed by one or more processors, the electronic device executes the shooting method of the first aspect described above.

[0033] In a third aspect, an embodiment of the present application provides a readable storage medium, on which instructions are stored. When the instructions are executed on an electronic device, the electronic device executes the shooting method of the first aspect mentioned above. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is a schematic diagram showing a user performing fine post-processing on a photo using image editing software in an electronic device;

[0035] Figure 2a shows a schematic diagram of a pedestrian removal scenario;

[0036] FIG2 b shows a schematic diagram of a scene with uneven illumination;

[0037] FIG2c shows a schematic diagram of a group photo scene;

[0038] FIG2 d shows a schematic diagram of a telephoto scene;

[0039] FIG3 shows a schematic diagram of functional modules of an electronic device according to some embodiments of the present application;

[0040] FIG4 shows a schematic diagram of the hardware structure of a mobile phone 10 according to some embodiments of the present application;

[0041] FIG5 is a schematic diagram showing a flow chart of a shooting method according to some embodiments of the present application;

[0042] FIG6 shows a data flow diagram of a shooting method according to some embodiments of the present application.

[0043] FIG7 shows a schematic diagram of the interaction process between a user and a mobile phone 10 in a pedestrian removal scenario according to some embodiments of the present application;

[0044] FIG8 is a schematic diagram showing a process of guiding a mobile phone 10 to shoot in a pedestrian removal scene according to some embodiments of the present application;

[0045] FIG9 shows a schematic diagram of a shooting interface 401 of a foldable screen mobile phone 10 according to some embodiments of the present application;

[0046] FIG10 is a schematic diagram showing a process of guiding shooting by a mobile phone 10 in a scene with uneven illumination according to some embodiments of the present application;

[0047] FIG11 is a schematic diagram showing a process of guiding a mobile phone 10 to take a photo in a group photo scene according to some embodiments of the present application;

[0048] FIG12 is a schematic diagram showing a process of guiding shooting by the mobile phone 10 in a telephoto scene according to some embodiments of the present application;

[0049] FIG13 shows a schematic diagram of images captured by different cameras in a telephoto scene in some embodiments of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0051] It should also be stated that the steps in the methods and processes in the embodiments of the present application are numbered for ease of reference, rather than to limit the order of precedence. If there is a sequence between the steps, the written description shall prevail.

[0052] As previously mentioned, to capture high-quality images, users can use image editing software in electronic devices to perform detailed post-processing on their photos. For example, as shown in FIG1 , in an image P1 captured by a user, including a target person P11 and a pedestrian P12 , the user can use the image editing software to post-process the image P1 using the software's artificial intelligence (AI) post-processing algorithm to remove pedestrian P12 . The software can then present the processed image P2 to the user as the final captured image.

[0053] In some embodiments, the AI ​​post-processing algorithm of the image editing software can automatically identify areas in the image that contain defects and affect image quality, and then optimize these areas. For example, for the above-mentioned image P1, the AI ​​post-processing algorithm of the image editing software can automatically identify the area where pedestrian P12 is located, and then eliminate pedestrian P12 from this area to achieve optimization of this area. However, the AI ​​post-processing algorithm may identify the wrong area, such as incorrectly identifying the area where target person P11 is located and eliminating target person P11. Moreover, when optimizing the area where pedestrian P12 is located, the AI ​​post-processing algorithm first erases the area where pedestrian P12 is located and then completes the area by referencing other parts of the background. Because the background in the area where pedestrian P12 is located is obscured by pedestrian P12, the true background of this area cannot be obtained from the single image P1. Supplementing this area by referencing other parts of the background may result in loss of effective information, making it difficult to obtain a high-quality image. Therefore, the effect of the AI ​​post-processing algorithm is uncontrollable. Before the post-processing results are available, the user cannot know the image effect, which affects the user's shooting experience.

[0054] In addition, the AI ​​post-processing algorithm model can be an end-side model or a large cloud model. The end-side model is a neural network model based on the mobile terminal device, and its performance and power consumption are limited by the performance of the mobile terminal on which it is located; the large cloud model is a deep neural network model in the cloud with millions or billions of parameters. It takes a long time to process images and often requires users to pay for its use.

[0055] In other embodiments, the electronic device may also employ a camera guidance scheme, guiding the user to adjust the framing to increase the probability of capturing a satisfactory image in a single shot. However, this scheme may not be effective in some scenarios where it's difficult to capture a perfect image in a single shot. Examples include the pedestrian removal scenario shown in Figure 2a, the uneven lighting scenario shown in Figure 2b, the group photo scenario shown in Figure 2c, and the telephoto scenario shown in Figure 2d. In pedestrian removal scenarios, such as a crowded scenic area, the large number of pedestrians and their uncontrollable movement make it difficult to capture a high-quality image in a single shot simply by guiding the user to adjust the framing. For example, adjusting the framing still cannot guarantee that no unrelated pedestrians are in the frame. If unrelated pedestrians re-enter the frame after the user has adjusted the framing, the image captured in a single shot will contain unrelated pedestrians, affecting image quality. In group photos, since it's difficult to simultaneously capture the required facial expressions and movements of each person, some people will likely close their eyes or smile awkwardly. Therefore, guiding the user to adjust the framing cannot guarantee a high-quality image in a single shot. In telephoto scenarios, due to limited field of view due to location constraints, it's impossible to capture the entire subject in a single shot simply by adjusting the framing, impacting the user's shooting experience. For example, if the user is far from the subject, a telephoto lens is required. However, the field of view of a telephoto lens is limited, and no matter how the framing is adjusted, the subject cannot be fully captured within the frame. Therefore, simply adjusting the framing alone will not capture the entire subject, resulting in a substandard image in a single shot.

[0056] In order to solve the above problems, the present application provides a shooting method, which can guide the user to take multiple shots to make up for the shortcomings of a single shot. For example, when the first image taken by the user for the first time through the electronic device does not meet the shooting requirements, such as when there are passers-by, uneven lighting, poor facial expressions of the characters in the first image, etc., the electronic device can prompt the user to take a second shot. For another example, the electronic device can guide the user to take a second shot based on the first image selected by the user from the gallery to make up for the shortcomings of the first image. In addition, the electronic device can determine the area to be optimized in the first image that does not meet the shooting requirements, prompt information containing shooting suggestions for the user, etc. based on the first image. Then, when the user takes the second shot, one or more of the area to be optimized, prompt information, fusion picture, etc. can be displayed in the shooting interface. In this way, the electronic device can provide the user with real-time shooting feedback and guide the user to shoot at the right time to improve the success rate of shooting.

[0057] It can be understood that during the second shooting, a preview screen and a fused screen can be displayed on the shooting interface, wherein the preview screen is used to display the preview image that changes in real time, and the fused screen is used to display the fused image of the preview image and the first image. The area to be optimized can be displayed in the preview screen or in the fused screen. The user can determine whether to adjust the shooting angle (such as avoiding pedestrians) and shooting parameters (such as magnification, brightness, etc.) based on the display changes of the area to be optimized. When the fused screen or the preview screen meets the shooting requirements, the electronic device can prompt the user to shoot.

[0058] In addition, it is understood that the prompt information may include suggestions for adjusting shooting parameters (such as magnification, brightness, etc.) and shooting angles, and the user can adjust the shooting parameters, shooting angles, etc. according to the prompt information. After the user adjusts the shooting parameters, shooting angles, etc. so that the fused image or preview image meets the shooting requirements, the electronic device may prompt the user to shoot.

[0059] Furthermore, it is understood that when the shooting environment changes, such as when a pedestrian leaves the optimized area, the electronic device can also determine in real time whether the shooting conditions have been met and prompt the user to shoot. Therefore, the above shooting method can help the user capture a satisfactory second image.

[0060] In some embodiments, the image captured by the user after the second shot can be the preview image displayed on the preview screen, or a fused image of the preview image displayed on the fused screen and the first image. The fused image can incorporate real information about the shooting environment at different times and spaces, retaining more effective information and improving image quality.

[0061] It is understood that if the second shot fails to produce an image that meets the aesthetic quality requirements, the electronic device may guide the user to take a third shot, a fourth shot, and so on. This application does not impose a limit on the number of shots. For example, if the second score is less than or equal to the second threshold, the electronic device may first guide the user to take a second shot to obtain a second image, and then, based on a fusion image of the second image and the preview image, guide the user to take a third shot to obtain a third image, until the image meets the aesthetic quality requirements or the user's requirements.

[0062] The following describes in detail the shooting method of the embodiment of the present application. The shooting method of the embodiment of the present application can be applied to electronic devices. It is understood that the electronic devices to which the present application is applicable can be mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, netbooks, augmented reality (AR) / virtual reality (VR) devices, smart TVs, smart watches, and other electronic devices, without limitation.

[0063] FIG3 is a schematic diagram of the functional modules of an electronic device provided in an embodiment of the present application. As shown in FIG3 , the electronic device in an embodiment of the present application may include an image acquisition module 101, a scene recognition module 102, a quality scoring module 103, an image fusion module 104, and an image display module 105. Among them, the image acquisition module 101 is used to acquire a first image, a preview image, a second image, etc. containing a shooting target. The scene recognition module 102 is used to perform scene recognition on the first image and determine the scene type of the first image. The quality scoring module 103 performs aesthetic quality scoring on the first image and other images based on the quality scoring model. The image fusion module 104 is used to fuse the first image and the preview image according to the image fusion algorithm to obtain a fused image. The image display module 105 is used to display the above-mentioned first image, preview image, and fused image in the shooting interface provided by the electronic device.

[0064] Taking the electronic device as a mobile phone 10 as an example, the following introduces a hardware structure diagram of the mobile phone 10 that can implement the shooting method of the embodiment of the present application.

[0065] As shown in Figure 4, the mobile phone 10 may include a processor 110, a power module 140, a memory 180, a mobile communication module 130, a wireless communication module 120, a sensor module 190, an audio module 150, a camera 170, an interface module 160, a button 101 and a display screen 102, etc.

[0066] It should be understood that the illustrated structure of the embodiment of the present invention does not constitute a specific limitation on the mobile phone 10. In other embodiments of the present application, the mobile phone 10 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0067] The processor 110 may include one or more processing units, for example, a processing module or processing circuit such as a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a microprocessor (micro-programmed control unit, MCU), an artificial intelligence (AI) processor, or a programmable logic device (field programmable gate array, FPGA). Among them, different processing units may be independent devices or integrated into one or more processors. A storage unit may be provided in the processor 110 for storing instructions and data. In some embodiments, the storage unit in the processor 110 is a cache memory 180. In some embodiments of the present application, the processor 110 may be used to execute the shooting method mentioned in the present application. The memory 180 may store relevant instructions for executing the shooting method mentioned in the present application.

[0068] The display screen 102 is used to display human-computer interaction interfaces, images, videos, etc. The display screen 102 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In the embodiment of the present application, the display screen 102 can be used to display the first image, preview image, second image, and fused image mentioned in the present application, etc. The above-mentioned image display module 105 can include the display screen 102.

[0069] Camera 170 is used to capture the first image, preview image, second image, etc. The lens generates an optical image of the object and projects it onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, which is then passed to the image signal processing (ISP) to be converted into a digital image signal. Mobile phone 10 implements the camera function through the ISP, camera 170, video codec, GPU, display 102, and application processor.

[0070] In some embodiments, the mobile phone 10 further includes buttons 101, a motor, and an indicator. The buttons 101 may include a volume button, an on / off button, and the like. The motor is used to vibrate the mobile phone 10. For example, in the embodiments of the present application, vibration is generated when the fused image meets aesthetic quality requirements to prompt the user to take a photo. The indicator may include a laser pointer, a radio frequency indicator, an LED indicator, and the like.

[0071] It should be noted that the hardware functional components of the above-mentioned mobile phone 10 can be changed according to user needs. It can be understood that the specific embodiment introduced above is only a specific implementation method of the electronic device. Other ways to implement the embodiments of this application are also within the scope of protection of this application and will not be repeated here.

[0072] The following describes a shooting method provided by an embodiment of the present application, which can be performed by an electronic device. Figure 5 is a flow chart of a shooting method provided by an embodiment of the present application, and Figure 6 is a data flow chart of a shooting method provided by an embodiment of the present application. Combined with Figures 5 and 6, the shooting method may include:

[0073] S1: The electronic device acquires a first image.

[0074] The first image may be an image captured for the first time.

[0075] In some embodiments, the electronic device detects a capture instruction and may use a camera to capture an image and generate a first image. The capture instruction is generated based on a user's capture operation. The capture instruction may be at least one of a touch command, a voice command, and a gesture command. For example, if the capture instruction is a touch command, the capture operation may include a user clicking a capture control on a capture interface.

[0076] In actual applications, the user can aim the camera at the shooting target, and when the shooting target is placed in the viewfinder of the shooting interface, directly click the shooting control to shoot, and the electronic device captures the first image. Alternatively, the user can first select a template image, and then the electronic device processes the image containing the shooting target taken by the user according to the style of the template image to obtain a first image similar in style to the template image. Here, similar style can refer to similar shooting targets, similar composition, similar colors, etc. Therefore, in some embodiments, the electronic device can use the camera to capture an image based on the template image selected by the user while detecting the shooting instruction, and obtain a first image similar in style to the template image.

[0077] In other embodiments, the electronic device detects a user's selection operation and uses the image selected by the user as the first image. For example, the user's selection operation may be selecting an image taken in the past from a gallery as the first image, and the image may be an image taken at a certain time in the past at the current location.

[0078] S2: The electronic device performs scene recognition on the first image to determine the scene type of the first image.

[0079] The electronic device may perform scene recognition on the first image by using a scene recognition algorithm, an image recognition model or other related recognition methods to determine the scene type of the first image.

[0080] In some embodiments, the electronic device may use a trained image recognition model to identify the type of the first image, thereby quickly determining the scene type of the first image.

[0081] In some embodiments, the electronic device pre-trains the image recognition model. According to actual applications and needs, during the model training process, a large number of training images are labeled with different scene types, such as portraits, landscapes, and other scene types. The model is then trained using the labeled training images to obtain a trained image recognition model.

[0082] It is understood that by optimizing the model structure and annotated data, more fine-grained scene recognition can be achieved. For example, the electronic device can directly output the pedestrian removal scene shown in Figure 2a, the uneven lighting scene shown in Figure 2b, the multi-person photo scene shown in Figure 2c, or the telephoto scene shown in Figure 2d.

[0083] It should be noted that the scene types are classified in combination with actual application scenarios, and this application does not limit the scene types and scene identification methods.

[0084] S3: The electronic device performs an aesthetic quality score on the first image according to the quality score model.

[0085] In an embodiment of the present application, the electronic device may input a first image into a quality scoring model, use the quality scoring model to perform an aesthetic quality score on the first image, and determine a first score for the first image and whether a first region exists in the first image. The first score serves as the basis for the electronic device to determine whether to guide the user to take a second shot. The first region is an area in the first image that does not meet the shooting requirements and is an area whose changes require special attention during the second shot. It is understood that the presence of the first region in the first image will result in a lower first score for the first image.

[0086] In some embodiments, when the first score is less than or equal to the first threshold, it indicates that the first area exists in the first image, that is, the first image does not meet the shooting requirement.

[0087] In some embodiments, the electronic device may employ different quality scoring models to perform aesthetic quality scoring based on different scene types. Specifically, different quality scoring models employ different algorithmic principles. For example, when the scene type is the aforementioned pedestrian removal scene, the quality scoring model typically utilizes detection algorithms such as pedestrian detection algorithms, subject region detection algorithms, image registration, and image difference algorithms to determine the first score by detecting whether pedestrians obstruct the subject region. For example, when the scene type is the aforementioned uneven illumination scene, the quality scoring model typically evaluates each pixel based on image parameters such as contrast, saturation, and brightness to determine the first score by detecting whether the image exposure is appropriate. For example, when the scene type is the aforementioned group photo scene, the quality scoring model typically utilizes face detection algorithms and expression recognition algorithms to determine the first score by detecting whether the expressions of the subjects are appropriate. For example, when the scene type is the aforementioned telephoto scene, the quality scoring model typically detects whether the subject is truncated to determine the first score. It will be appreciated that, depending on the scene type of the first image, the first region determined by the electronic device may include areas with pedestrians, areas with exposure issues, areas with poor expressions, or areas with truncated subjects.

[0088] In some embodiments, a first image may be identified as belonging to multiple scene types, and the electronic device may sequentially perform aesthetic quality scoring on the first image using multiple quality scoring models corresponding to the multiple scene types, then fuse the multiple scoring results to obtain a first score, and thereby determine one or more first regions. For example, if the first image is simultaneously identified as a pedestrian removal scene and a low-lighting scene, the electronic device may perform two aesthetic quality scorings on the first image using two quality scoring models, fuse the two scoring results to obtain a first score, and determine the areas where pedestrians are located and areas with exposure issues.

[0089] S4: The electronic device determines the area to be optimized.

[0090] The area to be optimized may include a first area determined by a quality scoring model, and may also include a second area manually added by a user.

[0091] In some embodiments, the electronic device may display the first image after the user takes the first shot, and after scene recognition and quality scoring, display the first area in the first image based on the first score being less than or equal to a first threshold.

[0092] Furthermore, the electronic device may determine, based on the first score being less than or equal to a first threshold, a prompt message containing a shooting suggestion for the user, and display the prompt message simultaneously with the first area. Here, the prompt message is used to remind the user to pay attention to the first area, and the prompt message may include displayable elements such as text and images. In other embodiments, the prompt message may also be in any form perceptible to the user, such as voice or vibration.

[0093] In some embodiments, the prompt information may provide shooting suggestions, including suggestions for adjusting shooting parameters and shooting angles determined based on the first image. Shooting parameters include, but are not limited to, magnification and brightness, and shooting angles include, but are not limited to, the height and tilt of the user holding the electronic device. The prompt information is described in detail below and is not detailed here.

[0094] In some embodiments, the user can adjust the size and position of the first area displayed in the first image. For example, the first area can be represented by a first display element, and the user can adjust the size and position of the first area by dragging, enlarging, or reducing the first display element in the first image. The shape of the first display element can be rectangular, circular, or other shapes, and this application does not limit the shape and color of the first display element.

[0095] Exemplarily, when the electronic device detects a user clicking operation on a first display element, it can change the display form of the first display element, where the display form includes shape and color, for example, from a first color to a second color, or from a first shape to a second shape.

[0096] In some embodiments, the user can manually add a second area in the first image. The electronic device can respond to the user's manual addition operation and display the second area corresponding to the operation. For example, the user can copy the first display element representing the first area to form the second display element, and the user can move the second display element to any area in the first image to form the second area. For another example, the electronic device provides a control for adding a second area in the display interface of the first image. The user can click on the control, and the electronic device can display the second display element in the first image. The user drags, zooms in or out of the first display element to form a second area in any area of ​​the first image. For another example, the user can draw the second display element directly on the first image to form the second area.

[0097] S5: The electronic device takes a second shot and obtains a preview image of the second shot.

[0098] In some embodiments, the electronic device guides the user to take a second shot based on the first score being less than or equal to a first threshold. In response to the user's reshoot instruction, the electronic device may display a second-shooting interface. Here, the reshoot instruction instructs the electronic device to enter a second-shooting state, not to execute a second shot and generate a captured image. The format and triggering method of the reshoot instruction can refer to the embodiment of the shooting instruction in S1 above and are not further described here.

[0099] The electronic device displays a preview screen in the shooting interface for the second shot. The preview screen may refer to the screen within the viewfinder, which is used to display the preview image that changes in real time. The electronic device may display the area to be optimized in the preview screen and may also display prompt information for the user to provide shooting suggestions, guiding the user to adjust the shooting angle, shooting parameters, etc.

[0100] S6: The electronic device performs image synthesis processing on the preview image and the first image.

[0101] Generally, when shooting for the second time, the electronic device directly uses the preview image displayed in the preview screen when the user presses the shooting control as the second image obtained for the second shooting, which may cause the loss of valid information in the first image. In the embodiment of the present application, the electronic device uses the preview image displayed in the preview screen when the user presses the shooting control and the fused image of the first image as the second image obtained for the second shooting, which can retain the valid information of the image during each shooting.

[0102] In some embodiments, the electronic device may adopt different image synthesis algorithms for different scene types. Specifically, different image synthesis algorithms correspond to different synthesis principles and purposes. The electronic device may adopt the image synthesis algorithm corresponding to the scene type according to the scene type identified in S2. For example, when the scene type is the above-mentioned pedestrian removal scene, the image synthesis algorithm mainly uses the preview image to replace the area where the pedestrians exist in the first image by cropping, replacing, etc. When the scene type is the above-mentioned uneven lighting scene, the image synthesis algorithm mainly uses the preview image to optimize the area with exposure problems in the first image by pixel fusion, etc.

[0103] In some embodiments, when the first image is identified as multiple scene types, the electronic device can determine the corresponding image synthesis algorithm based on the weight of the scene type. For example, when the weight of the first image belonging to the pedestrian removal scene is greater than the weight of the uneven lighting scene, it can be determined to use the image synthesis algorithm corresponding to the pedestrian removal scene. The weight of the scene type can be customized in combination with the actual application. For example, the weights of the multiple scene types provided in the above embodiment are, from large to small, telephoto scene, pedestrian removal scene, uneven lighting scene, and multi-person photo scene.

[0104] It is understood that the preview image in the preview screen changes in real time. In some embodiments, the electronic device can synthesize each frame of the preview image that changes in real time with the first image, or synthesize the preview image with the first image every preset number of frames or every preset duration. The preset number of frames and the preset duration can be determined in conjunction with the acquisition frequency of the electronic device's camera. For example, the preset number of frames can be 3 frames, 5 frames, etc., and the preset duration can be milliseconds, microseconds, etc. In this way, the frequency of image synthesis processing is reduced, and the hardware requirements for the electronic device are reduced.

[0105] S7: The electronic device displays the fused image in the shooting interface of the second shooting.

[0106] In addition to displaying the preview screen described above, the electronic device can also display a fused screen in the second capture interface. The fused screen is used to display the fused image resulting from the synthesis of the preview image and the first image. Accordingly, in addition to displaying areas to be optimized and user-recommended shooting tips in the preview screen, the electronic device can also display areas to be optimized and user-recommended shooting tips in the fused screen.

[0107] As described above, the area to be optimized may include a first area and a second area. In some embodiments, the user can manually add the second area to the first image, so that the first area and the second area can be directly displayed in the preview screen or the fused screen in the shooting interface of the second shot.

[0108] In other embodiments, the user can manually add a second area in the preview screen or the fused screen in the shooting interface of the second shot. In one application scenario, the electronic device may not display the first image when the first image is acquired. After scene recognition and quality scoring, based on the first score being less than or equal to a first threshold, the electronic device may first display the first area in the preview screen or the fused screen, and then respond to the user's manual addition operation by determining and displaying the second area corresponding to the operation. The manual addition operation can refer to the previous embodiment and will not be repeated here.

[0109] In some embodiments, in the second shooting interface, the electronic device displays the first area with a first display element and displays the second area with a second display element. The shapes and colors of the first and second display elements can be the same or different.

[0110] It can be understood that electronic devices have a variety of different forms, such as folding, straight screen, etc. Electronic devices can determine how to display the fused screen and preview screen in the shooting interface according to different forms. For example, for a foldable electronic device, the shooting interface may include a main screen interface and a sub-screen interface. The foldable electronic device can display the preview screen in the main screen interface and the fused screen in the sub-screen interface. For a straight-screen electronic device, only one shooting interface can be displayed, and most of the shooting interface is used to display the preview screen. The straight-screen electronic device can display the fused screen in the form of picture-in-picture in the shooting interface. Here, the straight-screen electronic device can set the ratio of the preview screen to the fused screen to avoid the fusion screen blocking the preview screen.

[0111] S8: The electronic device performs aesthetic quality scoring on the fused image based on the quality scoring model.

[0112] In an embodiment of the present application, the electronic device may input the fused image into a quality scoring model, perform an aesthetic quality score on the fused image using the quality scoring model, and determine a second score of the fused image.

[0113] In some embodiments, the electronic device may perform an aesthetic quality rating on each frame of the fused image, or may perform an aesthetic quality rating on the fused image at a preset number of frames or at a preset duration. The preset number of frames and the preset duration may be determined based on actual application requirements and are not limited in this application. For example, the preset number of frames may be 3 frames, 5 frames, etc., and the preset duration may be in milliseconds, microseconds, etc.

[0114] In some embodiments, the electronic device may determine whether the area to be optimized meets the shooting condition based on the second score. If the second score is greater than or equal to a second threshold, the electronic device may determine that the area to be optimized meets the shooting condition.

[0115] Furthermore, if the second score is greater than or equal to the second threshold, the electronic device may display a prompt message to take a photo. If the second score is less than the second threshold, the electronic device may display a prompt message including a photo suggestion. Here, the photo suggestion may include a photo suggestion determined based on the first image or a photo suggestion determined based on the fused image.

[0116] Furthermore, if the second score is greater than or equal to the second threshold, the electronic device may eliminate or change the first display element and the second display element displayed on the preview screen or the fused screen, so that the user can promptly understand that the first area and the second area meet the shooting conditions. If the second score is less than the second threshold, the first display element and the second display element are neither eliminated nor changed.

[0117] In some embodiments, the electronic device may further determine whether the area to be optimized meets the shooting conditions based on the display changes of the area to be optimized. Specifically, the electronic device may determine the display changes of the area to be optimized based on the fused image and the first image. For example, if the area to be optimized includes an area where a pedestrian is located in the first image, the electronic device may determine whether there are still pedestrians in the area in the fused image. If there are no pedestrians, it indicates that the area meets the shooting conditions. For another example, if the area to be optimized includes an area with insufficient lighting, the electronic device may determine whether the lighting in the area is sufficient in the fused image. If there is sufficient lighting, it indicates that the area meets the shooting conditions.

[0118] S9: The electronic device acquires a second image.

[0119] The second image is an image generated during the second shooting.

[0120] In some embodiments, the electronic device detects a shooting instruction indicating shooting and uses a fusion image of the preview image and the first image as the second image. The shooting instruction is generated based on the user's shooting operation, and the embodiment of the shooting instruction in S1 above can be referred to.

[0121] Specifically, the user executes a shooting operation based on a prompt displayed on the electronic device to shoot, triggering the generation of a shooting instruction, completing a second shooting, and obtaining a second image. In other embodiments, the user can actively execute a shooting operation based on a display change in the area to be optimized, triggering the generation of a shooting instruction, completing a second shooting, and obtaining a second image.

[0122] As previously described, the second image is a fusion of the preview image and the first image. In some embodiments, the electronic device may use the fusion image generated before the capture instruction is generated as the second image. Alternatively, upon detecting the capture instruction, the electronic device may synthesize the preview image displayed on the preview screen with the first image to generate a new fusion image as the second image.

[0123] It can be understood that if the second shooting cannot make the area to be optimized meet the shooting conditions, for example, the second score of the fused image of the preview image and the first image cannot reach the second threshold, or there are many areas to be optimized and the shooting conditions cannot be met at the same time, the electronic device can guide the user to take a third shot, and determine whether the area to be optimized meets the shooting conditions based on the preview image in the preview screen during the third shooting and the fused image of the second image. If it still does not meet the conditions, the fourth and fifth shots can be taken until the area to be optimized meets the shooting conditions to obtain the final shot image.

[0124] The following takes the electronic device as a mobile phone 10 and performs two shots as an example, and introduces in detail the shooting method proposed in the embodiment of the present application based on the pedestrian removal scene shown in Figure 2a, the uneven lighting scene shown in Figure 2b, the multi-person photo scene shown in Figure 2c and the telephoto scene shown in Figure 2d.

[0125] The following describes a shooting method provided by an embodiment of the present application based on the pedestrian removal scenario shown in Figure 2a, in conjunction with Figures 7 and 8. Figure 7 shows a schematic diagram of the interaction process between a user and mobile phone 10 in the pedestrian removal scenario shown in Figure 2a, and Figure 8 shows a schematic diagram of the process of mobile phone 10 guiding shooting in the pedestrian removal scenario shown in Figure 2a according to some embodiments of the present application.

[0126] As shown in Figure 7, the interaction process includes the following steps:

[0127] S101: A terminal user shoots a target and performs a shooting operation to form a first image.

[0128] In this step, the user uses the mobile phone 10 to shoot the target, performs a shooting operation, and forms a first image. The image acquisition module 101 in the mobile phone 10 can acquire the first image to execute subsequent steps.

[0129] As shown in FIG8 , the user's shooting operation may be that the user clicks a shooting control 2011 in the shooting interface 201 of the mobile phone 10 .

[0130] In this embodiment, the shooting interface 201 may include a shooting control 2011 , a zoom control 2012 , a preview screen 2013 and other controls. The user can click the shooting control 2011 to form a first image A1 .

[0131] S102: The mobile phone 10 performs scene recognition on the first image.

[0132] In this step, after the image acquisition module 101 of the mobile phone 10 acquires the first image, it sends the first image to the scene recognition module 102. The scene recognition module 102 performs scene recognition on the first image and sends the recognized scene type of the first image and the first image to the quality scoring module 103 and the image fusion module 104.

[0133] As shown in FIG8 , the first image A1 includes a photographed object P11 and a pedestrian P12 , and the scene type of the first image may be a pedestrian-removed scene.

[0134] S103: The mobile phone 10 performs aesthetic quality scoring on the first image, and determines a region to be optimized and corresponding prompt information.

[0135] In this step, the quality scoring module 103 of the mobile phone 10 determines a quality scoring model corresponding to the scene type of the first image according to the scene type of the first image, and uses the quality scoring model to perform an aesthetic quality score on the first image to obtain a first score and an area to be optimized in the first image.

[0136] As shown in Figure 8, the quality scoring module 103 can select a quality scoring model corresponding to the pedestrian removal scenario to perform an aesthetic quality score on the first image A1, focusing on whether there are irrelevant pedestrians that affect the quality of the first image A1. Exemplarily, the quality scoring model can include pedestrian detection and subject area detection algorithms. The quality scoring module 103 can perform subject area detection on the first image A1 and determine that the photographed object P11 is the subject and the pedestrian P12 is an irrelevant person. Pedestrian P12 will affect the score of the first image A1, causing the first image A1 to fail to meet the aesthetic quality requirements and the score of the first image A1 to fail to reach the first threshold. Therefore, the quality scoring module 103 can determine that the area where pedestrian P12 is located is the area to be optimized U11. Furthermore, based on the scoring results, the mobile phone 10 generates corresponding prompt information T11 and sends the first image A1, the location information of the area to be optimized U11, and the prompt information T11 to the image display module 105.

[0137] Specifically, the quality scoring module 103 may set the full score of the quality score to 100 and the first threshold to 80. When the first score is greater than or equal to 80, the image is considered to meet the aesthetic quality requirements and be a high-quality image. The quality scoring module 103 may also set the full score of the quality score to 1 and the threshold to 0.8. When the image score is greater than or equal to 0.8, the image is considered to meet the aesthetic quality requirements and be a high-quality image. It should be understood that the full score and threshold of the quality score may also be set to other values, and this application does not limit this.

[0138] For example, if the full score of the quality score is set to 100 and the first threshold is 80, since there is an area U11 to be optimized in the first image A1, the score of the first image may be 75 points, which does not reach the first threshold, and the first image A1 needs to be optimized.

[0139] For example, as shown in FIG8 , the prompt information T11 may be a text prompt, such as “Please select or adjust the area to be optimized and shoot again.” The text prompt may also be other content. This application does not limit the specific text content of the prompt information.

[0140] In some embodiments, the first image is an image of similar style taken based on a template image selected by the user from a gallery, and the quality scoring model can use image registration and image difference algorithms to perform aesthetic quality scoring on the first image based on the template image.

[0141] S104: The mobile phone 10 visually displays the area to be optimized in the first image.

[0142] In this step, after the image display module 105 obtains the first image, the location information of the area to be optimized and the prompt information from the quality scoring module 103, it visually displays at least one of the first image, the area to be optimized and the prompt information to the user.

[0143] As shown in FIG8 , the shooting interface 301 may include a first image A1, an area to be optimized U11, prompt information T11, an “OK” control 3011, and a “Cancel” control 3012. The user may interact with the mobile phone 10 based on the prompt information T11 in the shooting interface 301 and perform subsequent operations by clicking the “OK” control 3011 or the “Cancel” control 3012.

[0144] It can be understood that the area to be optimized U11 can be displayed as a first display element, and the first display element can be a rectangular frame as shown in FIG. 8 .

[0145] It is understood that the shooting interface 301 may also include other controls. The first image A1, prompt information T11, "OK" control 3011 and "Cancel" control 3012 may also be displayed in other locations in the shooting interface 301, and this application does not limit this.

[0146] S105: The terminal user adjusts the area to be optimized or adds a new area to be optimized, and performs a second shot.

[0147] In this step, the user can adjust the area to be optimized or add a new area to be optimized according to the prompt information, and enter the shooting interface for the second shooting by clicking the "OK" control 3011.

[0148] In the embodiment of the present application, as shown in Figure 8, the user can perform corresponding operations based on the prompt T11 in the shooting interface 301. For example, the user can adjust the area to be optimized U11 by clicking, dragging, zooming in or out, etc., to avoid the situation where people who need to be retained are removed or people who need to be removed are retained. The user can also manually add new areas to be optimized in the first image A1. After the operation is completed, the user can click the "OK" control 3011 to enter the shooting interface 401 for the second shot.

[0149] In some embodiments, the user may not perform corresponding operations according to the prompt information. For example, the user may not adjust the area to be optimized and directly click the "OK" control 3011 to enter the shooting interface 401. For another example, the user may return to the shooting interface 201 by clicking the "Cancel" control 3012.

[0150] S106: The mobile phone 10 obtains a preview stream.

[0151] In this step, the preview stream includes the preview images collected continuously. After the mobile phone 10 enters the second shooting, the image acquisition module 101 will acquire the preview image containing the shooting target in real time through the camera and send the preview stream to the image fusion module 104 and the image display module 105.

[0152] As shown in FIG8 , after entering the second shooting mode, the mobile phone 10 may display a shooting interface 401 , which includes a shooting control 2011 , a zoom control 2012 , a preview screen 4013 and other controls. The preview screen 4013 displays a preview image A2 .

[0153] S107: The mobile phone 10 fuses the preview image and the first image to obtain a fused image.

[0154] In this step, image fusion module 104 selects a corresponding image synthesis algorithm based on the scene type of the first image. After obtaining a preview image, it fuses the preview image with the first image using the selected image synthesis algorithm to generate a fused image. Image fusion module 104 sends the fused image to image display module 105 for visual display to the user and sends the fused image to quality scoring module 103 for aesthetic quality scoring.

[0155] As shown in FIG. 8 , the shooting interface 401 may further include a fused screen 4014 . The fused screen 4014 is displayed in a picture-in-picture format in the shooting interface 401 , and the area to be optimized U11 is displayed in the fused screen 4014 .

[0156] S108: The mobile phone 10 performs aesthetic quality scoring on the fused image of the preview image and the first image in real time.

[0157] In this step, the quality scoring module 103 performs aesthetic quality scoring on the fused image of the preview image and the first image in real time to determine whether the area to be optimized in the fused image is optimized.

[0158] It is understood that the quality scoring module 103 performs aesthetic quality scoring on the fused image to determine the second score. The main purpose is to determine whether the area to be optimized in the fused image is optimized, such as whether the area to be optimized U11 in FIG8 is optimized.

[0159] S109: The mobile phone 10 displays the preview image and the fused image of the first image, the prompt information and the preview image in real time.

[0160] In this step, the image display module 105 displays the preview image, the fused image of the preview image and the first image, and prompt information in the shooting interface of the second shooting. The prompt information can be used to guide the user to perform the second shooting.

[0161] As shown in FIG8 , since there is an irrelevant pedestrian P12 in the area to be optimized U11, the quality scoring module 103 can perform pedestrian motion estimation on the fused image A3 and determine whether the irrelevant pedestrian P12 has left the area. When it is detected that the pedestrian P12 has left the area, it indicates that the area to be optimized U11 has been optimized and the fused image A3 meets the shooting conditions. At this time, the corresponding prompt information T12 can be generated and displayed.

[0162] Specifically, when the quality scoring module 103 detects that the irrelevant pedestrian P12 is moving slowly and the irrelevant pedestrian P12 or part of the irrelevant pedestrian P12 still exists in the area to be optimized U11 in the fused image A3, a corresponding prompt information T12 is generated, which can be a text prompt "Please wait"; when the quality scoring module 103 detects that the irrelevant pedestrian P12 is in a stationary state and the irrelevant pedestrian P12 or part of the irrelevant pedestrian P12 still exists in the area to be optimized U11 in the fused image A3, a corresponding prompt information T12 is generated, which can be a text prompt "Adjust the shooting angle"; when the quality scoring module 103 detects that there is no irrelevant pedestrian in the area to be optimized U11 in the fused image A3, at this time, the second score of the fused image A3 reaches the second threshold, and the corresponding prompt information T12 generated can be a text prompt "Please shoot".

[0163] In some embodiments, when it is determined that the area to be optimized U11 is not optimized, the mobile phone 10 can display the color of the rectangular box corresponding to the area to be optimized U11 as red; when it is determined that the area to be optimized U11 is optimized, the color of the rectangular box corresponding to the area to be optimized U11 can be displayed as green.

[0164] In some embodiments, the prompt information T12 may also be a specific numerical value of the second score. For example, the second score of the fused image may be directly displayed in the preview screen 4013.

[0165] In some embodiments, as shown in Figure 9, when the mobile phone 10 is a foldable screen mobile phone, the shooting interface 401 includes a main screen shooting interface 401a and a sub-screen display interface 401b, and the fused image can also be displayed at a specific position (for example, the lower half) of the sub-screen display interface 401b of the mobile phone 10. At this time, the first image can also be simultaneously displayed at a specific position (for example, the upper half) of the sub-screen display interface 401b of the mobile phone 10, and the text description corresponding to the image can be displayed in the sub-screen display interface 401b, for example, the words "final effect preview" are displayed above the fused image.

[0166] It should be understood that the fused image can also be displayed in the upper half of the secondary screen display interface 401b, and accordingly, the first image can also be displayed in the lower half of the secondary screen display interface 401b. The proportions of the fused image and the first image in the secondary screen display interface 401b can be the same or different, and the fused image can occupy more or less of the screen, which is not limited in this application. The text description corresponding to the image can also be displayed below the image or in other locations, which is not limited in this application.

[0167] S110: The terminal user performs related operations according to the prompt information or remains unchanged.

[0168] In this step, the user performs related operations or remains unchanged according to the prompt information in the shooting interface of the second shooting.

[0169] As shown in FIG8 , when the prompt message T12 is a text prompt of “Please wait”, the user can keep the current posture unchanged; when the prompt message T12 is a text prompt of “Adjust shooting angle”, the user can adjust the shooting angle appropriately.

[0170] In some embodiments, when the rectangular frame of the area to be optimized U11 in FIG8 is displayed in red, the user can also adjust the shooting angle appropriately.

[0171] It should be understood that the user may adjust the shooting angle or perform other operations on his own without following the prompt information.

[0172] S111: The mobile phone 10 scores the aesthetic quality of the fused image of the preview image and the first image in real time, and adjusts the prompt information; when the fused image meets the shooting conditions, the user is prompted to shoot.

[0173] In this step, the quality scoring module 103 performs aesthetic quality scoring on the fused image of the preview image and the first image that changes in real time, adjusts the prompt information, and prompts the user to shoot when it is determined that the fused image meets the shooting conditions.

[0174] Specifically, the prompt information corresponding to the judgment result of the quality scoring module 103 in step S108 may be referred to, and the prompt information may be adjusted accordingly according to the real-time judgment result of the fused image.

[0175] As shown in FIG8 , when the user adjusts the shooting angle and the unrelated pedestrian P12 leaves the area to be optimized U11, the quality scoring module 103 determines that the fused image A3 meets the shooting conditions and prompts the user to shoot. For example, the prompt T12 can be adjusted to a text prompt "Please shoot," prompting the user to click the shooting control 211 to execute the shooting operation; the rectangular frame of the area to be optimized U11 can be changed from red to green, prompting the user to click the shooting control 211 to execute the shooting operation; or a "√" symbol can be displayed on the shooting control 211, prompting the user to click the shooting control 211 to execute the shooting operation.

[0176] In some scenarios, as shown in FIG8 , even if other unrelated pedestrians P13 appear in the preview image A2 , since the pedestrian P13 does not exist in the area of ​​the first image A1 , the pedestrian P13 will not exist in the fused image A3 either.

[0177] S112: The terminal user performs a shooting operation to form a second image.

[0178] In this step, when the user sees the prompt information indicating that the shooting is possible in the shooting interface of the second shooting, the user can click the shooting control to perform the shooting operation to form the second image of the second shooting.

[0179] S113: The mobile phone 10 merges the preview image and the first image to obtain a second image.

[0180] In this step, the image acquisition module 101 acquires the preview image and sends the preview image to the image fusion module 104 . The image fusion module 104 fuses the preview image with the first image to obtain a second image, and sends the second image to the image display module 105 .

[0181] In some embodiments, as shown in FIG8 , the mobile phone 10 can directly use the fused image A3 displayed in the fused screen 4014 as the second image. Alternatively, the image fusion module 104 fuses the preview image displayed in the preview screen when the user clicks the capture control with the first image to obtain a new fused image, and uses the new fused image as the second image.

[0182] S114: The mobile phone 10 presents the second image as the final captured image to the user.

[0183] In this step, the image display module 105 presents the second image as the final captured image to the user, and saves the second image in the gallery of the mobile phone 10 .

[0184] It should be understood that during the second shooting process, before the user clicks the shooting control, steps S106 to S111 are executed in a real-time loop.

[0185] It is understood that the embodiments of the present application can display the area to be optimized in the first image obtained by the first shot, and guide the user to take multiple shots through prompt information during the subsequent shooting process, thereby optimizing the image obtained by the first shot and fusing the images obtained by the multiple shots to compensate for the defects in the image obtained by the first shot, thereby obtaining a high-quality image. Moreover, during the shooting process, the user can see the final shooting effect in real time, which can make it easier to obtain an image that the user is satisfied with and avoid missing the best time to shoot.

[0186] The following describes a shooting method proposed in an embodiment of the present application based on the uneven illumination scene shown in Figure 2b and in conjunction with Figure 10. Figure 10 shows a schematic diagram of a shooting guidance process of the mobile phone 10 in the uneven illumination scene shown in Figure 2b according to some embodiments of the present application.

[0187] As shown in Figure 10, the scene has very strong lighting in some areas and very weak lighting in others. The difference is so great that it is impossible to obtain satisfactory imaging results using a single set of exposure parameters. When a user takes their first photo, mobile phone 10 displays shooting interface 201, with target P31 displayed in preview screen 2013. When the user presses shooting control 2011, first image A4 is formed. After acquiring first image A4, image acquisition module 101 sends first image A4 to scene recognition module 102. Scene recognition module 102 performs scene recognition on first image A4, determines that the scene type of first image A3 is an uneven lighting scene, and sends the scene recognition result to quality scoring module 103 and image fusion module 104.

[0188] In the embodiment of the present application, the quality scoring module 103 selects a corresponding quality scoring model based on the scene recognition results of the first image A4 and performs an aesthetic quality score on the first image A4. The focus is on whether the first image A4 has underexposed or overexposed areas that affect the score of the first image A4. Specifically, the quality scoring module 103 can determine that area U31 in the first image A4 is underexposed, resulting in a low score for the first image A4 that fails to meet the first threshold. The image display module 105 can display area U31 in the shooting interface 301 and display the corresponding prompt information T31. It will be understood that the acquisition of the first image and the related description of the shooting interface can be referred to the previous steps S101 to S104 and will not be repeated here.

[0189] As shown in FIG10 , after the user clicks the “OK” control 3011 in the shooting interface 301 to enter the second shooting interface 401 , the user can perform corresponding operations according to the prompt information T32 . When the area U31 in the fused image is optimized, the user is prompted to shoot.

[0190] For example, if the area to be optimized U31 is underexposed, the prompt message T32 may read "Increase exposure gain." If the area to be optimized U31 is overexposed, the prompt message T32 may read "Lower exposure gain." The user can adjust the exposure parameters based on the prompt message T32. This application does not limit the specific content of the prompt message.

[0191] In some embodiments, the mobile phone 10 may also automatically calculate and adjust exposure parameters without the need for manual adjustment by the user.

[0192] It can be understood that the description of the second shooting process can refer to the description of steps S105 to S114 above, which will not be repeated here.

[0193] In the embodiment of the present application, by guiding the user to take multiple shots and fusing the images taken multiple times, an imaging result with appropriate exposure height of each part can be obtained, so that the user can obtain high-quality images.

[0194] The following describes a shooting method proposed in an embodiment of the present application based on the group photo scene shown in FIG2c and in combination with FIG11.

[0195] FIG11 is a schematic diagram showing a process of guiding shooting by the mobile phone 10 in the group photo scene shown in FIG2c according to some embodiments of the present application.

[0196] As shown in Figure 11, the scene in which the subject P21 is photographed consists of multiple people, making it easy for some people to have their eyes closed, smile poorly, or be obscured. After the first image A5 is captured for the first time, the scene recognition module 102 first performs scene recognition on the first image A5, determining that the scene is a group photo. The focus is on whether the expressions of each face in the subject P21 meet the requirements. Based on the scene recognition results, the quality scoring module 103 first performs face detection on the first image A5 to obtain the location of each face, and then performs aesthetic quality scoring on the obtained faces. Based on the scoring results, it determines whether there are any facial areas with low scores, and identifies the facial areas with low scores as areas to be optimized, obtaining the location information of these areas. The image display module 105 displays the first image A5, prompt information T21, and area to be optimized U21 in the shooting interface 301 for visual display to the user.

[0197] In the embodiment of the present application, the user can also manually select the face to be optimized as the area to be optimized, and then click the "OK" control 3011 to enter the second shooting process.

[0198] It can be understood that the related description of obtaining the first image and visual display of the shooting interface can be referred to the above steps S101 to S104, which will not be repeated here.

[0199] In the embodiment of the present application, after the user clicks the "OK" control 3011 in the shooting interface 301 to enter the shooting interface 401 with prompt information, the user can perform corresponding operations according to the prompt information T22. When the area to be optimized U21 in the fused image is optimized, the user is prompted to shoot.

[0200] For example, during the second shooting process, the prompt information T22 may also include a rating score. For example, the rating score of the face in the area to be optimized U21 in the fused image may be directly displayed above the area to be optimized U21 in the fused image 4014. When the rating of the face in the area to be optimized U21 does not reach the second threshold, a text prompt such as "Please adjust your expression" may be used to remind the user that adjustments are still needed. When the rating of the face in the area to be optimized U21 reaches the second threshold, a text prompt such as "Please shoot" may be used to remind the user that the photo can be taken. Alternatively, the prompt information T22 may be a pattern color prompt. For example, when the rating of the face in the area to be optimized U21 does not reach the threshold, the outer frame color of the area to be optimized U21 is displayed in red. When the threshold is reached, the outer frame color of the area to be optimized U21 is displayed in green.

[0201] In some embodiments, each face in the photographic target P21 may be marked with a regularly shaped frame, and the color of the frame may be used to indicate to the user whether the face in the photographic target P21 meets the scoring threshold. For example, a green frame indicates that the face meets the scoring threshold, while a red frame indicates that the face does not meet the scoring threshold.

[0202] It is understandable that during the second shooting process, the image fusion module 104 may fuse only the face area that needs to be optimized, and other areas that do not need to be optimized may remain unchanged.

[0203] It can be understood that other relevant descriptions of the second shooting process can refer to the relevant descriptions of steps S105 to S114 above, which will not be repeated here.

[0204] In the embodiment of the present application, by guiding the user to take multiple shots and fusing the images obtained from the multiple shots, it can be ensured that each face in the shooting target meets the aesthetic quality requirements, thereby obtaining a high-quality image.

[0205] The following describes a shooting method proposed in an embodiment of the present application based on the telephoto scene shown in FIG2 d and in combination with FIG12 .

[0206] FIG12 is a schematic diagram showing a process of guiding shooting by the mobile phone 10 in the telephoto scene shown in FIG2 d according to some embodiments of the present application.

[0207] As shown in Figure 12, in this scene, due to the limited framing range of the 10x telephoto lens, it is difficult to obtain a complete image of the target. After capturing the first image A6, the scene recognition module 102 first performs scene recognition on the first image A6, determining that it is a telephoto scene, focusing on whether the subject P41 is truncated. Based on the scene recognition results, the quality scoring module 103 generates corresponding prompt information T41. The image display module 105 displays the first image A6 and prompt information T41 on the shooting interface 301 for visual presentation to the user. This scene does not require annotation of the areas to be optimized.

[0208] It can be understood that the related description of obtaining the first image and visual display of the shooting interface can be referred to the above steps S101 to S104, which will not be repeated here.

[0209] In the embodiment of the present application, after the user clicks the "OK" control 3011 in the shooting interface 301 to enter the shooting interface 401 with prompt information, the user can perform corresponding operations according to the prompt information T42. When the fused image meets the aesthetic quality requirements, the user is prompted to shoot.

[0210] Specifically, during the second capture process, the image acquisition module 101 uses the main camera lens to capture a panoramic image that includes the entire target P41, determines the position of the first image A6 within the panoramic image, and then generates one or more candidate framing frames near the position of the first image A6 in the panoramic image. Referring to FIG13 , based on the first image A6 captured by the telephoto lens and the panoramic image captured by the main camera lens, it can be seen that the main camera lens has a wider framing range than the telephoto lens. Therefore, a panoramic image of the target P41 can be captured by the main camera lens. Based on this panoramic image, candidate framing frames (the area indicated by the dashed box) near the position of the first image A6 (the area indicated by the solid box) can be determined. It can be understood that based on the actual position of the first image in the panoramic image, one or more candidate framing frames can be determined, and then the framing frame with the optimal composition can be selected from the one or more candidate framing frames. The framing frame with the optimal composition is the candidate framing frame that can completely accommodate the target P41. The direction in which the user needs to move the telephoto lens is then calculated, and a corresponding prompt message T42 is generated. This prompt message T42 can be at least one of "move up," "move down," "move left," or "move right." The image fusion module 104 fuses the preview image with the first image A6 and displays the fusion result to the user in real time. The quality scoring module 103 performs an aesthetic quality score on the current fused image in real time. Based on the score, it determines whether the current fused image has subject truncation issues or whether it has achieved an optimal composition. If the fused image meets the aesthetic quality requirements, the user is prompted to shoot.

[0211] For example, the panoramic image captured by the main camera lens may be a 1000×1000 image, and the first image captured by the telephoto lens may be a 100×100 image. In the panoramic image, the first image may be divided into 100 areas of 100×100 in size, and the position or coordinates of the first image in the panoramic image may be determined. Candidate framing frames may be generated near the first image, and the framing frame with the best composition may be selected from them. Based on the position of the candidate framing frames, the direction in which the user needs to move may be determined.

[0212] It can be understood that in this embodiment, the image fusion module 104 fuses the preview image and the first image A6, which can be to splice the preview image and the first image A6. For example, the first image A6 and the preview image are both square. The same parts of the first image A6 and the preview image can be overlapped, and the different parts of the two can be spliced ​​together to form a fused image. Since the fused image may be a polygon, the fused image also needs to be filled with a regular rectangle. For example, the area not covered by the framing can be filled with black or filled with the corresponding framing area in the main camera result.

[0213] In the embodiment of the present application, after the user performs a second capture operation and obtains a current preview image, the image fusion module 104 fuses the preview image with the first image to obtain a second image. The second image may be obtained by splicing the current preview image and the first image. Therefore, it is necessary to perform maximum inscribed rectangle cropping on the second image or crop it using an intelligent cropping algorithm to obtain a square image of a regular size as the final fused image presented to the user.

[0214] It can be understood that the fused image presented in the fusion screen 4014 of the shooting interface 401 with prompt information is only for showing the final fusion effect to the user in real time. The image obtained by filling the area not covered by the framing with black cannot be used as the final captured image due to the existence of black parts. The image obtained by filling it with the corresponding framing area in the main camera result cannot be used as the final captured image because the clarity of the image in the main camera result and the image obtained by the telephoto lens are quite different, and the clarity of each area in the image is inconsistent.

[0215] It can be understood that other relevant descriptions of the second shooting process can refer to the relevant descriptions of steps S105 to S114 above, which will not be repeated here.

[0216] In an embodiment of the present application, by guiding the user to take multiple shots and fusing and stitching the images obtained from the multiple shots, the effect of expanding the field of view (FOV) of telephoto photography can be achieved, thereby obtaining high-quality images.

[0217] References in the specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one exemplary implementation or technique disclosed according to the embodiment of the present application. The appearances of the phrase "in one embodiment" in various places in the specification do not necessarily all refer to the same embodiment.

[0218] The disclosure of the embodiment of the present application also relates to an operating device for executing the text. The device can be constructed specifically for the required purpose or it can include a general-purpose computer that is selectively activated or reconfigured by a computer program stored in the computer. Such a computer program can be stored in a computer-readable storage medium, such as, but not limited to any type of disk, including a floppy disk, an optical disk, a CD-ROM, a magneto-optical disk, a read-only memory (ROM), a random access memory (RAM), an EPROM, an EEPROM, a magnetic or optical card, an application-specific integrated circuit (ASIC) or any type of medium suitable for storing electronic instructions, and each can be coupled to a computer system bus. In addition, the computer mentioned in the specification can include a single processor or can be an architecture involving multiple processors for increased computing power.

[0219] In addition, the language used in this specification has been primarily selected for readability and instructional purposes and may not be selected to describe or limit the disclosed subject matter. Therefore, the present disclosure of embodiments is intended to illustrate, not to limit, the scope of the concepts discussed herein.

Claims

1. A shooting method, applied to an electronic device, characterized in that including: in response to a first shooting instruction, obtaining a first image and displaying a first prompt message prompting the user to shoot again; in response to a reshooting instruction, displaying a shooting interface, wherein the shooting interface includes a preview screen and a second prompt message related to an area to be optimized of the first image; wherein the preview screen is used to display a preview image that changes in real time; corresponding to the area to be optimized meeting the shooting conditions, displaying a third prompt message for the user to shoot; in response to a second shooting instruction, obtaining a second image, where the second image includes a fusion image after image synthesis processing of the preview image and the first image.

2. The shooting method according to claim 1, wherein further including: displaying a fusion screen in the shooting interface, and the second prompt message is displayed in the fusion screen or the preview screen; wherein the fusion screen is used to display the fusion image.

3. The photographing method according to claim 2, wherein the fusion screen does not overlap with the preview screen; or at least part of the fusion screen overlaps with the preview screen.

4. The photographing method according to claim 1, wherein the area to be optimized includes a first area obtained by performing image detection on the first image; the method further includes: determining the first prompt message and the second prompt message based on the image detection result of the first image; wherein the second prompt message includes shooting adjustment suggestions related to the first area.

5. The shooting method according to claim 4, characterized in that, further including: determining the scene type of the first image; performing image detection on the first image according to the quality scoring model corresponding to the scene type, obtaining the image detection result of the first image, and the image detection result of the first image includes a first score; corresponding to the first score being less than or equal to a first threshold, the image detection result of the first image includes the position information of the first area.

6. The photographing method according to claim 5, wherein the displaying the first prompt message prompting the user to shoot again includes: based on the position information of the first area, displaying the first area in the first image with a first display element to form a first prompt message prompting the user to shoot again.

7. The shooting method according to claim 6, wherein further including: corresponding to the first area meeting the shooting conditions, changing the attribute of the first display element, and the attribute includes at least one of shape and color.

8. The shooting method according to any one of claims 1-6, characterized in that further including: in response to a first operation by the user to add an area to be optimized, displaying a second area corresponding to the first operation in the first image with a second display element.

9. The photographing method according to claim 2, wherein further including: determining whether the area to be optimized meets the shooting conditions based on the image detection result of the fusion image; the image detection result of the fusion image includes a second score of the fusion image; corresponding to the second score being greater than or equal to a second threshold, determining that the area to be optimized meets the shooting conditions.

10. The photographing method according to claim 1, wherein further including: determining whether the area to be optimized meets the shooting conditions based on the display change of the preview image in the preview screen; corresponding to a display change in the area corresponding to the area to be optimized in the preview image, determining that the area to be optimized meets the shooting conditions.

11. An electronic device, characterized in that, including: One or more processors, one or more memories storing one or more programs, which when executed by the one or more processors, cause the electronic device to perform the shooting method according to any one of claims 1 to 10.

12. A readable storage medium, characterized in that, Instructions are stored on the readable storage medium, which when executed on the electronic device cause the electronic device to perform the shooting method according to any one of claims 1 to 10.

Images