Shooting method, electronic equipment and readable storage medium

Abstract

The invention relates to the technical field of intelligent terminals, in particular to a shooting method, electronic equipment and a readable storage medium. The shooting method is applied to the electronic equipment, and when a first image shot by a user through the electronic equipment for the first time does not meet the shooting requirement, the electronic equipment can prompt the user to shoot for the second time. And when the user determines to carry out the second shooting, displaying prompt information related to the to-be-optimized area which does not meet the shooting requirement in the first image so as to guide the user to adjust or not. And when the electronic equipment determines that the to-be-optimized area meets the shooting condition, the electronic equipment displays prompt information to assist the user in completing the second shooting. Therefore, the electronic equipment can guide the user to shoot for multiple times to make up for the deficiency of single shooting, assist the user in shooting satisfactory images, and improve the quality of the finally shot images.

Description

Technical Field

[0001] This application relates to the technical field of intelligent terminals, and particularly relates to a shooting method, an electronic device, and a readable storage medium. Background Art

[0002] During the process of a user using an electronic device to take a photo, it is very difficult to take a satisfactory photo in one go. For example, in a scenic area with "people coming and going" or a group photo of multiple people, etc., usually multiple shots are required. Or, the user needs to use image editing software to perform fine post-processing on the captured image, such as cropping the image or eliminating irrelevant elements in the image, etc., to obtain a satisfactory photo. However, the photo obtained by performing post-processing using image editing software may have problems such as loss of effective information and insufficient image effect, etc. These problems often affect the quality of the image, resulting in the user being unable to obtain a high-quality image and affecting the user's shooting experience. Summary of the Invention

[0003] To solve the above problems, embodiments of this application provide a shooting method, an electronic device, and a readable storage medium.

[0004] In a first aspect, an embodiment of this application provides a shooting method, which is applied to an electronic device and includes: in response to a first shooting instruction, obtaining a first image and displaying a first prompt message for prompting the user to take a second shot; in response to a second shooting instruction, displaying a shooting interface, where the shooting interface includes a preview screen and a second prompt message related to an area to be optimized of the first image; where the preview screen is used to display a preview image that changes in real time; when the area to be optimized meets the shooting conditions, displaying a third prompt message for the user to take a shot; in response to a second shooting instruction, obtaining a second image, where the second image includes a fusion image of the preview image and the first image after image synthesis processing.

[0005] In the above solution, when the first image taken by the user through the electronic device for the first time does not meet the shooting requirements, the electronic device can prompt the user to take a second shot. When the user determines to take a second shot, a prompt message related to the area to be optimized in the first image that does not meet the shooting requirements is displayed to guide the user whether adjustment is needed. And when the electronic device determines that the area to be optimized meets the shooting conditions, a prompt message is displayed to assist the user to complete the second shot. And, the image obtained by the user's second shot is a fusion image of the first image obtained by the first shot and the preview image during the second shot.

[0006] Thus, on the one hand, the electronic device can guide the user to take multiple shots to make up for the deficiencies of a single shot, instruct the user to take a shot at an appropriate time, and assist the user in taking a satisfactory image. On the other hand, the fused image can fuse the real information of the shooting environment at different times and spaces, and can retain more effective information, thereby improving the quality of the finally captured image.

[0007] In some embodiments of the first aspect above, a fused screen is displayed in the shooting interface, and the second prompt information is displayed in the fused screen or the preview screen.

[0008] Among them, the fused screen is used to display the fused image. In this way, the electronic device can provide real-time shooting feedback to the user.

[0009] In addition, the fused screen can be displayed in the shooting interface in different ways according to different forms of the electronic device. For example, for a foldable electronic device, the shooting interface may include a main screen interface and a secondary screen interface. Then, the preview screen can be displayed in the main screen interface, and the fused screen can be displayed in the secondary screen interface. Another example is that for a straight-screen electronic device, the fused screen can be displayed in the shooting interface in a picture-in-picture form.

[0010] In some embodiments of the first aspect above, the fused screen does not overlap with the preview screen; or, at least a part of the fused screen overlaps with the preview screen.

[0011] It can be understood that for a foldable electronic device, the fused screen does not overlap with the preview screen. For a straight-screen electronic device, at least a part of the fused screen overlaps with the preview screen, and the overlapping area can be set according to actual applications, and it should be avoided that the fused screen blocks the preview screen.

[0012] In some embodiments of the first aspect above, the area to be optimized includes a first area obtained by performing image detection on the first image; the shooting method further includes: determining first prompt information and second prompt information based on the image detection result of the first image; wherein, the second prompt information includes shooting adjustment suggestions related to the first area.

[0013] It can be understood that the electronic device performs image detection on the first image to obtain the image detection result of the first image. Here, the main purpose of image detection is to determine whether there is a defective area, that is, the first area to be optimized. Thus, when the image detection result indicates the existence of the first area, the first prompt information for prompting the user to take a re-shot is generated and displayed, and the second prompt information including shooting adjustment suggestions related to the first area is generated and displayed based on the defect of the first area.

[0014] In some embodiments of the first aspect described above, the photographing method further includes: determining a scene type of the first image; performing image detection on the first image according to a quality scoring model corresponding to the scene type to obtain an image detection result of the first image, where the image detection result of the first image includes a first score; corresponding to the first score being less than or equal to a first threshold, the image detection result of the first image includes position information of a first region.

[0015] It can be understood that the defects of images taken in different shooting scenes are different. Therefore, the electronic device identifies the scene type of the first image and uses different quality scoring models for image detection according to different scene types. Different quality scoring models correspond to different detection principles or detection algorithms.

[0016] For example, when the electronic device identifies the scene type of the first image as a pedestrian removal scene, the quality scoring model can adopt pedestrian detection algorithms, main region detection algorithms, image registration and image difference algorithms, etc. For another example, when the electronic device identifies the scene type of the first image as an uneven illumination scene, the quality scoring model can evaluate each pixel based on image parameters such as contrast, saturation, and brightness. For another example, when the electronic device identifies the scene type of the first image as a group photo scene, the quality scoring model can utilize face detection algorithms, expression recognition algorithms, etc. For another example, when the electronic device identifies the scene type of the first image as a telephoto scene, the quality scoring model can use a main body truncation detection algorithm to determine whether there is a main body truncation in the shooting target.

[0017] Among them, the input of the quality scoring model is an image, and the output includes the aesthetic quality score of the image. It can be understood that if the aesthetic quality score is low and there is a first region, the output of the quality scoring model can also include the position information of the first region. For example, the pixel coordinates of the first region in the image.

[0018] In some embodiments of the first aspect described above, displaying a first prompt message for prompting the user to take another photo includes: based on the position information of the first region, displaying the first region in the first image with a first display element to form a first prompt message for prompting the user to take another photo.

[0019] It can be understood that the first region in the present application can be characterized by a first display element, and the first display element has attributes such as shape and color.

[0020] In some embodiments of the first aspect described above, the photographing method further includes: corresponding to the first region satisfying the shooting condition, changing an attribute of the first display element, where the attribute includes at least one of shape and color.

[0021] It can be understood that the electronic device can eliminate or transform the first display element. For example, when the shooting condition is satisfied in the first area, the shape and / or color of the first display element can be changed so that the user can timely know whether the shooting condition has been satisfied in the first area.

[0022] In some embodiments of the above first aspect, the shooting method further includes: in response to a first operation by the user to add a region to be optimized, displaying, in a first image, a second region corresponding to the first operation with a second display element.

[0023] It can be understood that in addition to the first area automatically detected by the electronic device, the user can manually add a second area in the first image. The first operation can be the manual addition operation mentioned in this application.

[0024] Among them, the second region is characterized by a second display element, and the second display element has attributes such as shape and color. The shapes and colors of the first display element and the second display element can be the same or different.

[0025] In some embodiments of the above first aspect, the shooting method further includes: determining whether the region to be optimized satisfies the shooting condition based on the image detection result of the fused image; the image detection result of the fused image includes a second score of the fused image; corresponding to the second score being greater than or equal to a second threshold, determining that the region to be optimized satisfies the shooting condition.

[0026] It can be understood that the electronic device can input the fused image into a quality scoring model to determine the second score of the fused image. The electronic device can determine whether the region to be optimized satisfies the shooting condition based on the second score. If the second score is greater than or equal to the second threshold, the electronic device can determine that the region to be optimized satisfies the shooting condition.

[0027] In some embodiments of the above first aspect, the shooting method further includes: determining whether the region to be optimized satisfies the shooting condition based on the display change of the preview image in the preview screen; corresponding to the region corresponding to the region to be optimized in the preview image having a display change, determining that the region to be optimized satisfies the shooting condition.

[0028] Among them, the display change can be directly observed by the user. For example, when the pedestrian in the region to be optimized leaves, the user can observe the display change and determine that the region to be optimized satisfies the shooting condition.

[0029] Alternatively, the electronic device can determine whether there is a display change in the preview image based on the fused image and the first image to determine the region to be optimized. For example, if there is a pedestrian in the region to be optimized in the first image and there is no pedestrian in the region to be optimized in the fused image, it means that there is a display change, that is, it is determined that the region to be optimized satisfies the shooting condition.

[0030] In a second aspect, embodiments of the present application provide an electronic device, including: one or more processors, one or more memories, and one or more programs stored in the one or more memories. When the one or more programs are executed by the one or more processors, the electronic device is caused to execute the photographing method of the first aspect described above.

[0031] In a third aspect, embodiments of the present application provide a readable storage medium, on which instructions are stored. When the instructions are executed on an electronic device, the electronic device is caused to execute the photographing method of the first aspect described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 FIG. shows a schematic diagram of a user using image editing software in an electronic device to perform fine post-processing on a photo;

[0033] Figure 2a FIG. shows a schematic diagram of a pedestrian removal scenario;

[0034] Figure 2b FIG. shows a schematic diagram of an uneven illumination scenario;

[0035] Figure 2c FIG. shows a schematic diagram of a group photo scenario;

[0036] Figure 2d FIG. shows a schematic diagram of a telephoto scenario;

[0037] Figure 3 According to some embodiments of the present application, FIG. shows a schematic diagram of functional modules of an electronic device;

[0038] Figure 4 According to some embodiments of the present application, FIG. shows a schematic diagram of the hardware structure of a mobile phone 10;

[0039] Figure 5 According to some embodiments of the present application, FIG. shows a schematic diagram of the flow of a photographing method;

[0040] Figure 6 According to some embodiments of the present application, FIG. shows a schematic diagram of data flow of a photographing method

[0041] Figure 7 According to some embodiments of the present application, FIG. shows a schematic diagram of the interaction process between a user and the mobile phone 10 in a pedestrian removal scenario;

[0042] Figure 8 According to some embodiments of the present application, FIG. shows a schematic diagram of the process of the mobile phone 10 guiding photographing in a pedestrian removal scenario;

[0043] Figure 9According to some embodiments of the present application, a schematic diagram of a shooting interface 401 of a foldable screen mobile phone 10 is shown;

[0044] Figure 10 According to some embodiments of the present application, a schematic diagram of the process of the mobile phone 10 guiding shooting in a scene with uneven illumination is shown;

[0045] Figure 11 According to some embodiments of the present application, a schematic diagram of the process of the mobile phone 10 guiding shooting in a group photo scene is shown;

[0046] Figure 12 According to some embodiments of the present application, a schematic diagram of the process of the mobile phone 10 guiding shooting in a telephoto scene is shown;

[0047] Figure 13 Some embodiments of the present application show a schematic diagram of images captured by different cameras in a telephoto scene. Detailed implementation manners

[0048] In order to make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described in detail below in conjunction with the accompanying drawings of the specification and specific implementation manners.

[0049] It should also be stated that the numbering of the steps in the methods and processes in the embodiments of the present application is for the convenience of reference and does not limit the sequence. If there is a sequence between the steps, it shall be subject to the written description.

[0050] As described above, in order to capture high-quality images, users can use image editing software in electronic devices to perform fine post-processing on photos. Exemplarily, as Figure 1 shown, in the image P1 obtained by the user's shooting, it includes the target person P11 and the pedestrian P12. The user can use the image editing software and utilize the artificial intelligence (AI) post-processing algorithm of the image editing software to perform post-processing on the image P1, remove the pedestrian P12, and the image editing software can present the processed image P2 to the user as the final captured image.

[0051] In some embodiments, the AI post - processing algorithm of the image - editing software can automatically identify the areas in the image that have defects and affect the image quality, and then optimize these areas. For example, for the above - mentioned image P1, the AI post - processing algorithm of the image - editing software can automatically identify the area where the pedestrian P12 is located, and then eliminate the pedestrian P12 in this area to achieve the optimization of this area. However, the AI post - processing algorithm may identify the wrong area. For example, it may wrongly identify the area where the target person P11 is located and eliminate the target person P11. Moreover, when the AI post - processing algorithm optimizes the area where the pedestrian P12 is located, it first erases the area where the pedestrian P12 is located and then complements this area with other parts of the background. Since the background of the area where the pedestrian P12 is located is blocked by the pedestrian P12, the true background of this area cannot be obtained from the single image P1. Complementing this area by referring to other parts of the background may result in the loss of effective information, and it is difficult to obtain a high - quality image. Therefore, the effect of the AI post - processing algorithm is uncontrollable. Before the post - processing result comes out, the user cannot know the image effect, which affects the user's shooting experience.

[0052] In addition, the AI post - processing algorithm model can be an edge - side model or a cloud - based large model. The edge - side model is a neural network model based on a mobile terminal device, and its performance and power consumption are limited by the performance of the mobile terminal where it is located. The cloud - based large model is a deep neural network model with millions or billions of parameters in the cloud. The time for image processing is long, and users often need to pay to use it.

[0053] In some other embodiments, the electronic device can also use a photo - taking guidance scheme to guide the user to adjust the viewfinder to increase the probability that the user can take a satisfactory image in a single shot. However, this scheme may fail in some scenarios where a perfect image cannot be taken at once. For example Figure 2a the pedestrian removal scenario shown, Figure 2b the uneven lighting scenario shown, Figure 2c the group photo scenario shown with multiple people, Figure 2dThe long - focal - length scenes shown, etc. In the pedestrian removal scene, taking a scenic spot with a huge crowd as an example, since there are many pedestrians and their movement is uncontrollable, it is very difficult to obtain a high - quality image in a single shot just by guiding the user to adjust the viewfinder. For example, even if the user adjusts the viewfinder, it is still impossible to ensure that there are no irrelevant pedestrians in the viewfinder. After the user adjusts the viewfinder, if an irrelevant pedestrian breaks into the viewfinder again, it will result in the presence of irrelevant pedestrians in the image obtained in a single shot, affecting the image quality. In the group photo scene, since it is difficult to meet the requirements of everyone's expressions, actions, etc. simultaneously, there are always some people with problems such as closing their eyes or having an unpleasant smile. Therefore, it is also impossible to obtain a high - quality image in a single shot by guiding the user to adjust the viewfinder. In the long - focal - length scene, due to the limited field of view caused by site restrictions, it is impossible to obtain a complete shooting target in a single shot just by adjusting the viewfinder, affecting the user's shooting experience. For example, when the user is far away from the shooting target and needs to use a long - focal - length lens for shooting, and the field of view of the long - focal - length lens is limited, no matter how the viewfinder is adjusted, it is impossible to completely present the shooting target in the viewfinder. Therefore, it is impossible to shoot a complete shooting target just by adjusting the viewfinder, resulting in the image obtained in a single shot not meeting the requirements.

[0054] To solve the above problems, the present application provides a shooting method. This shooting method can guide the user to take multiple shots to make up for the deficiencies of a single shot. For example, when the first image taken by the user through the electronic device for the first time does not meet the shooting requirements, such as there are passing pedestrians, uneven lighting, or poor expressions of people in the first image, the electronic device can prompt the user to take a second shot. Another example is that the electronic device can guide the user to take a second shot based on the first image selected by the user from the photo library to make up for the deficiencies of the first image. Moreover, the electronic device can determine the area to be optimized where the first image does not meet the shooting requirements, the prompt information including shooting suggestions for the user, etc. based on the first image. Then, when the user takes the second shot, one or more of the area to be optimized, the prompt information, the fused image, etc. can be displayed on the shooting interface. In this way, the electronic device can provide real - time shooting feedback for the user and guide the user to take a shot at the appropriate time, improving the shooting success rate.

[0055] It can be understood that when taking the second shot, a preview image and a fused image can be displayed on the shooting interface. Among them, the preview image is used to display the preview image that changes in real - time, and the fused image is used to display the fused image of the preview image and the first image. The area to be optimized can be displayed in the preview image or in the fused image. The user can determine whether to adjust the shooting angle (such as avoiding pedestrians) or shooting parameters (such as magnification, brightness, etc.) based on the display change of the area to be optimized. When the fused image or the preview image meets the shooting requirements, the electronic device can prompt the user to take a shot.

[0056] In addition, it can be understood that the prompt information may include suggestions for adjusting shooting parameters (such as magnification, brightness, etc.), suggestions for adjusting the shooting angle, etc. The user can adjust the shooting parameters, shooting angle, etc. according to the prompt information. When the user adjusts the shooting parameters, shooting angle, etc. so that the merged image or the preview image meets the shooting requirements, the electronic device can prompt the user to take a photo.

[0057] In addition, it can be understood that when the shooting environment changes, such as when pedestrians in the area to be optimized leave, the electronic device can also determine in real time whether the shooting conditions are met, and then prompt the user to take a photo. Therefore, the above shooting method can assist the user in taking a satisfactory second image.

[0058] In some embodiments, the image captured by the user after the second shooting may be the preview image displayed in the preview screen, or the merged image of the preview image and the first image displayed in the merged screen. Among them, the merged image can merge the real information of the shooting environment at different times and spaces, and can retain more effective information, improving the image quality.

[0059] It can be understood that if the second shooting fails to obtain an image that meets the aesthetic quality requirements, the electronic device can also guide the user to perform a third shooting, a fourth shooting, etc. This application does not limit the number of shootings. For example, when the second score is less than or equal to the second threshold, the electronic device can first guide the user to perform a second shooting to obtain a second image, and then guide the user to perform a third shooting to obtain a third image based on the merged image of the second image and the preview image, until the image meets the aesthetic quality requirements or the user's requirements.

[0060] The shooting method of the embodiments of the present application will be described in detail below. The shooting method of the embodiments of the present application can be applied to an electronic device. It can be understood that the electronic devices applicable to this application can be mobile phones, tablet computers, desktop computers, laptop computers, handheld computers, netbooks, augmented reality (AR) / virtual reality (VR) devices, smart TVs, smart watches, etc. Electronic devices, which are not limited here.

[0061] Figure 3 It is a schematic diagram of the functional modules of an electronic device provided by an embodiment of the present application. As Figure 3As shown in the figure, the electronic device according to the embodiment of the present application may include an image acquisition module 101, a scene recognition module 102, a quality scoring module 103, an image fusion module 104, and an image display module 105. Among them, the image acquisition module 101 is used to acquire a first image, a preview image, a second image, etc. that contain the shooting target. The scene recognition module 102 is used to perform scene recognition on the first image to determine the scene type of the first image. The quality scoring module 103 performs an aesthetic quality score on images such as the first image according to a quality scoring model. The image fusion module 104 is used to fuse the first image and the preview image according to an image fusion algorithm to obtain a fused image. The image display module 105 is used to display the above-mentioned first image, preview image, and fused image in the shooting interface provided by the electronic device.

[0062] Taking the electronic device as a mobile phone 10 as an example, the following introduces the schematic hardware structure diagram of the mobile phone 10 that can implement the shooting method according to the embodiment of the present application.

[0063] As Figure 4 shown in the figure, the mobile phone 10 may include a processor 110, a power module 140, a memory 180, a mobile communication module 130, a wireless communication module 120, a sensor module 190, an audio module 150, a camera 170, an interface module 160, buttons 101, and a display screen 102, etc.

[0064] It can be understood that the structure schematically shown in the embodiment of the present invention does not constitute a specific limitation on the mobile phone 10. In other embodiments of the present application, the mobile phone 10 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0065] The processor 110 may include one or more processing units. For example, it may include a processing module or processing circuit such as a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro-programmed control unit (MCU), an artificial intelligence (AI) processor, or a field programmable gate array (FPGA). Among them, different processing units may be independent devices or integrated in one or more processors. A storage unit may be provided in the processor 110 for storing instructions and data. In some embodiments, the storage unit in the processor 110 is the cache memory 180. In some embodiments of the present application, the processor 110 may be used to execute the shooting method mentioned in the present application. The memory 180 may store relevant instructions for executing the shooting method mentioned in the present application.

[0066] The display screen 102 is used to display a human-computer interaction interface, images, videos, etc. The display screen 102 includes a display panel. The display panel may adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In the embodiments of the present application, the display screen 102 may be used to display the first image, preview image, second image, and fused image mentioned in the present application. The above image display module 105 may include the display screen 102.

[0067] The camera 170 is used to capture the first image, preview image, second image, etc. An object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the image signal processing (ISP) to convert it into a digital image signal. The mobile phone 10 may implement the shooting function through the ISP, the camera 170, the video codec, the GPU, the display screen 102, and the application processor, etc.

[0068] In some embodiments, the mobile phone 10 further includes a button 101, a motor, an indicator, etc. Among them, the button 101 may include a volume key, a power-on / off key, etc. The motor is used to make the mobile phone 10 produce a vibration effect. For example, in the embodiments of the present application, when the fused image meets the aesthetic quality requirements, vibration is generated to prompt the user to take a photo. The indicator may include a laser indicator, a radio frequency indicator, an LED indicator, etc.

[0069] It should be noted that the above-mentioned hardware function components of the mobile phone 10 can be changed according to the needs of the user. It can be understood that the specific embodiments described above are only a specific implementation manner of the electronic device, and other embodiments that can implement the embodiments of the present application are also within the scope of protection of the present application, which will not be elaborated here.

[0070] The following introduces a shooting method provided by the embodiments of the present application, and this method can be executed by an electronic device. Figure 5 is a schematic flowchart of a shooting method provided by the embodiments of the present application, Figure 6 is a schematic data flow diagram of a shooting method provided by the embodiments of the present application. Combining Figure 5 and Figure 6 as shown, the shooting method may include:

[0071] S1: The electronic device acquires a first image.

[0072] Among them, the first image may be an image obtained by the first shooting.

[0073] In some embodiments, when the electronic device detects a shooting instruction indicating shooting, it can use the camera to collect images and generate a first image. The shooting instruction is generated based on the user's shooting operation. Among them, the shooting instruction may be at least one of a touch instruction, a voice instruction, and a gesture instruction. Taking the shooting instruction as a touch instruction as an example, the shooting operation may include the user's click operation on the shooting control on the shooting interface.

[0074] In practical applications, the user can aim the camera at the shooting target. When the shooting target is placed in the viewfinder of the shooting interface, directly click the shooting control to take a photo, and the electronic device collects the first image. Or, the user can first select a template image, and then the electronic device processes the image containing the shooting target taken by the user according to the style of the template image to obtain a first image similar to the style of the template image. Here, being similar in style may mean being similar in shooting target, composition, color, etc. Therefore, in some embodiments, the electronic device can, while detecting the shooting instruction, collect images using the camera based on the template image selected by the user to obtain a first image similar to the style of the template image.

[0075] In some other embodiments, the electronic device detects a user's selection operation and uses the selected image by the user as the first image. Exemplarily, the user's selection operation may be to select a previously taken image from the gallery as the first image, which may be an image taken at the current location at a certain previous moment.

[0076] S2: The electronic device performs scene recognition on the first image to determine the scene type of the first image.

[0077] Among them, the electronic device can perform scene recognition on the first image through relevant recognition methods such as scene recognition algorithms or image recognition models to determine the scene type of the first image.

[0078] In some embodiments, the electronic device can use a trained image recognition model to recognize the type of the first image, so as to quickly determine the scene type of the first image.

[0079] In some embodiments, the electronic device pre-trains the image recognition model. According to actual applications and requirements, during the model training process, different scene types, such as portrait photos, landscape photos, etc., are labeled for a large number of training images, and then the labeled training images are used to train the model to obtain a trained image recognition model.

[0080] It can be understood that by optimizing the structure of the model and the labeled data, more fine-grained scene recognition can be achieved. For example, the electronic device can directly output scenes such as pedestrian removal scene shown above Figure 2a the uneven illumination scene shown above Figure 2b the group photo scene shown above Figure 2c or the telephoto scene shown above Figure 2d etc.

[0081] It should be noted that the scene types are classified in combination with the actual application scenarios, and this application does not limit the scene types and scene recognition methods.

[0082] S3: The electronic device performs an aesthetic quality score on the first image according to the quality scoring model.

[0083] In the embodiments of this application, the electronic device can input the first image into the quality scoring model, use the quality scoring model to perform an aesthetic quality score on the first image, and determine the first score of the first image and whether there is a first area in the first image. Among them, the first score is the basis for the electronic device to determine whether to guide the user to take a second shot, and the first area is the area in the first image that does not meet the shooting requirements and is the area that needs to focus on its changes during the second shot. It can be understood that if there is a first area in the first image, it will result in a lower first score of the first image.

[0084] In some embodiments, when the first score is less than or equal to the first threshold, it indicates that there is a first region in the first image, that is, the first image does not meet the shooting requirements.

[0085] In some embodiments, the electronic device may use different quality scoring models for aesthetic quality scoring according to different scene types. Specifically, the algorithm principles adopted by different quality scoring models are different. For example, when the scene type is the above-mentioned pedestrian removal scene, the quality scoring model usually uses detection algorithms such as pedestrian detection algorithms, main region detection algorithms, image registration and image difference algorithms, and determines the first score by detecting whether there are pedestrians blocking the main region. For example, when the scene type is the above-mentioned uneven illumination scene, the quality scoring model usually evaluates each pixel based on image parameters such as contrast, saturation, and brightness, and determines the first score by detecting whether the image exposure is appropriate. For example, when the scene type is the above-mentioned group photo scene, the quality scoring model usually uses face detection algorithms and expression recognition algorithms to determine the first score by detecting whether the expressions of the people are appropriate. For example, when the scene type is the above-mentioned telephoto scene, the quality scoring model usually detects whether there is truncation of the shooting target to determine the first score. It can be understood that according to the different scene types of the first image, the first region determined by the electronic device may include the region where pedestrians exist, the region with exposure problems, the region with poor expressions, or the region where the shooting target is truncated, etc.

[0086] In some embodiments, the first image may be recognized as multiple scene types, and the electronic device may sequentially use multiple quality scoring models corresponding to the multiple scene types to perform aesthetic quality scoring on the first image, and then fuse the results of multiple scores to obtain the first score, and obtain one or more first regions. For example, the first image may be recognized as a pedestrian removal scene and an insufficient illumination scene at the same time, then the electronic device may use two quality scoring models to perform two aesthetic quality scorings on the first image, fuse the results of the two scores to obtain the first score, and determine the region where pedestrians are located and the region with exposure problems.

[0087] S4: The electronic device determines the area to be optimized.

[0088] Among them, the area to be optimized may include the first region determined by the quality scoring model, and may also include the second region manually added by the user.

[0089] In some embodiments, the electronic device may display the first image after the user's first shot. After scene recognition and quality scoring, based on the first score being less than or equal to the first threshold, the first region is displayed in the first image.

[0090] Further, the electronic device may determine a prompt message including a shooting suggestion for the user based on the first score being less than or equal to the first threshold, and display the prompt message while displaying the first area. Here, the prompt message is used to remind the user to pay attention to the first area, and the prompt message may include displayable elements such as text and patterns. In other embodiments, the prompt message may also be in any form that can be perceived by the user, such as voice and vibration.

[0091] In some embodiments, the prompt message may provide shooting suggestions including shooting parameter adjustment suggestions and shooting angle adjustment suggestions determined based on the first image. Among them, the shooting parameters include but are not limited to parameters such as magnification and brightness, and the shooting angles include but are not limited to the height and inclination of the user holding the electronic device. The prompt message will be described in detail below and will not be elaborated here.

[0092] In some embodiments, the user may adjust the size and position of the first area displayed in the first image. For example, the first area may be represented by a first display element, and the user can adjust the size and position of the first area by performing operations such as dragging, enlarging, or shrinking the first display element in the first image. Among them, the shape of the first display element may be a rectangle, a circle, or the like, and the present application does not limit the shape and color of the first display element.

[0093] Exemplarily, when the electronic device detects a click operation on the first display element, the display form of the first display element may be changed, and the display form includes shape and color. For example, it changes from the first color to the second color and from the first shape to the second shape.

[0094] In some embodiments, the user may manually add a second area in the first image. The electronic device may display the second area corresponding to the operation in response to the user's manual addition operation. For example, the user may copy the first display element representing the first area to form a second display element, and the user may move the second display element to any area in the first image to form a second area. Another example is that the electronic device provides a control for adding a second area in the display interface of the first image. The user may click on the control, and the electronic device may display a second display element in the first image. The user can perform operations such as dragging, enlarging, or shrinking the first display element to form a second area in any area of the first image. Another example is that the user may directly draw a second display element on the first image to form a second area.

[0095] S5: The electronic device performs a second shooting and obtains a preview image of the second shooting.

[0096] In some embodiments, based on the first score being less than or equal to the first threshold, the electronic device guides the user to take a second shot. The electronic device can display a shooting interface for the second shot in response to the user's re - shooting instruction. Here, the re - shooting instruction indicates that the electronic device enters the state of the second shot, rather than executing the second shot to generate a captured image. The form and triggering method of the re - shooting instruction can refer to the embodiments of the shooting instruction in S1 above and will not be elaborated here.

[0097] In the shooting interface for the second shot, the electronic device displays a preview screen. The preview screen can refer to the picture in the viewfinder, which is used to display a preview image that changes in real - time. The electronic device can display an area to be optimized in the preview screen and can also display prompt information about shooting suggestions for the user to guide the user to adjust the shooting angle, shooting parameters, etc.

[0098] S6: The electronic device performs image synthesis processing on the preview image and the first image.

[0099] Generally, when taking the second shot, if the electronic device directly uses the preview image displayed in the preview screen when the user presses the shooting control as the second image obtained from the second shot, it may cause the loss of valid information in the first image. However, in the embodiments of the present application, the electronic device uses the fused image of the preview image displayed in the preview screen when the user presses the shooting control and the first image as the second image obtained from the second shot, which can retain the valid information of the images in each shot.

[0100] In some embodiments, the electronic device can adopt different image synthesis algorithms for different scene types. Specifically, the synthesis principles and purposes corresponding to different image synthesis algorithms are different. The electronic device can adopt the image synthesis algorithm corresponding to the scene type according to the scene type identified in S2. For example, when the scene type is the above - mentioned pedestrian removal scene, the image synthesis algorithm mainly replaces the area with pedestrians in the first image by means of cropping, replacement, etc. using the preview image. When the scene type is the above - mentioned uneven illumination scene, the image synthesis algorithm mainly optimizes the area with exposure problems in the first image by means of pixel fusion, etc. using the preview image.

[0101] In some embodiments, when the first image is recognized as multiple scene types, the electronic device can determine the corresponding image synthesis algorithm according to the weights of the scene types. For example, when the weight of the first image belonging to the pedestrian removal scene is greater than the weight of the uneven illumination scene, it can be determined to adopt the image synthesis algorithm corresponding to the pedestrian removal scene. Among them, the weights of the scene types can be custom - set in combination with actual applications. For example, the weights of the multiple scene types provided in the above embodiments from large to small are long - focal - length scene, pedestrian removal scene, uneven illumination scene, group photo scene.

[0102] It can be understood that the preview image in the preview screen changes in real time. In some embodiments, the electronic device may perform a synthesis process on each frame of the preview image that changes in real time with the first image, or may perform a synthesis process on the preview image and the first image every preset number of frames or every preset duration. Among them, the preset number of frames and the preset duration can be determined in combination with the acquisition frequency of the camera of the electronic device. Exemplarily, the preset number of frames can be 3 frames, 5 frames, etc., and the preset duration can be in milliseconds, microseconds, etc. In this way, the frequency of the image synthesis process is reduced, and the hardware requirements for the electronic device are lowered.

[0103] S7: The electronic device displays the fused image in the shooting interface of the second shooting.

[0104] In the shooting interface of the second shooting, the electronic device can display not only the above-mentioned preview screen, but also the fused screen. The fused screen is used to display the fused image obtained by performing an image synthesis process on the preview image and the first image. Correspondingly, the electronic device can display the area to be optimized and the prompt information of the shooting suggestion for the user not only in the preview screen, but also in the fused screen.

[0105] As described above, the area to be optimized may include a first area and a second area. In some embodiments, the user can manually add the second area in the first image. In this way, in the shooting interface of the second shooting, the first area and the second area can be directly displayed in the preview screen or the fused screen.

[0106] In other embodiments, the user can manually add the second area in the preview screen or the fused screen in the shooting interface of the second shooting. In an application scenario, the electronic device may not display the first image when the first image is acquired. After scene recognition and quality scoring, based on the first score being less than or equal to the first threshold, the first area is first displayed in the preview screen or the fused screen, and then in response to the user's manual addition operation, the second area corresponding to the operation is determined and displayed. The manual addition operation can refer to the previous embodiments and will not be elaborated here.

[0107] In some embodiments, in the shooting interface of the second shooting, the electronic device displays the first area with a first display element and the second area with a second display element. The shapes and colors of the first display element and the second display element may be the same or different.

[0108] It can be understood that electronic devices have various different forms, such as foldable, straight screen, etc. The electronic device can determine how to display the fused image and the preview image in the shooting interface according to different forms. For example, for a foldable electronic device, the shooting interface may include a main screen interface and a secondary screen interface. The foldable electronic device can display the preview image in the main screen interface and the fused image in the secondary screen interface. For a straight screen electronic device, only one shooting interface can be displayed, and most of the shooting interface is used to display the preview image. Then, the straight screen electronic device can display the fused image in the form of a picture-in-picture in the shooting interface. Here, the straight screen electronic device can set the ratio of the preview image to the fused image to avoid the fused image blocking the preview image.

[0109] S8: The electronic device performs an aesthetic quality score on the fused image according to the quality scoring model.

[0110] In the embodiments of the present application, the electronic device can input the fused image into the quality scoring model, use the quality scoring model to perform an aesthetic quality score on the fused image, and determine the second score of the fused image.

[0111] In some embodiments, the electronic device can perform an aesthetic quality score on each frame of the fused image, or can perform an aesthetic quality score on the fused image every preset number of frames or every preset duration. Here, the preset number of frames and the preset duration can be determined in combination with the actual application requirements, and the present application does not limit this. Exemplarily, the preset number of frames can be 3 frames, 5 frames, etc., and the preset duration can be in milliseconds, microseconds, etc.

[0112] In some embodiments, the electronic device can determine whether the area to be optimized meets the shooting conditions based on the second score. If the second score is greater than or equal to the second threshold, the electronic device can determine that the area to be optimized meets the shooting conditions.

[0113] Further, corresponding to the second score being greater than or equal to the second threshold, the electronic device can display a prompt message for shooting. Corresponding to the second score being less than the second threshold, the electronic device can display a prompt message including shooting suggestions. Here, the shooting suggestions can include shooting suggestions based on the first image or can also include shooting suggestions based on the fused image.

[0114] Further, corresponding to the second score being greater than or equal to the second threshold, the electronic device can eliminate or transform the first display element and the second display element displayed in the preview image or the fused image, so that the user can timely understand that the first area and the second area have met the shooting conditions. Corresponding to the second score being less than the second threshold, neither the first display element nor the second display element is eliminated or changed.

[0115] In some embodiments, the electronic device may also determine whether the area to be optimized meets the shooting conditions based on the display change of the area to be optimized. Specifically, the electronic device may determine the display change of the area to be optimized based on the fused image and the first image. For example, for the area to be optimized including the area where a pedestrian is located in the first image, it is determined whether there is still a pedestrian in this area in the fused image. If there is no pedestrian, it means that this area meets the shooting conditions. Another example is that for the area to be optimized including an area with insufficient lighting, it is determined whether the lighting in this area is sufficient in the fused image. If the lighting is sufficient, it means that this area meets the shooting conditions.

[0116] S9: The electronic device acquires a second image.

[0117] Wherein, the second image is an image generated during the second shooting.

[0118] In some embodiments, when the electronic device detects a shooting instruction indicating shooting, it uses the fused image of the preview image and the first image as the second image. The shooting instruction is generated based on the user's shooting operation, and the embodiment of the shooting instruction in S1 above can be referred to.

[0119] Specifically, the user performs a shooting operation based on the shooting prompt information displayed on the electronic device, triggers the generation of the shooting instruction, completes the second shooting, and obtains the second image. In other embodiments, the user may actively perform a shooting operation based on the display change of the area to be optimized, trigger the generation of the shooting instruction, complete the second shooting, and obtain the second image.

[0120] As described above, the second image is the fused image of the preview image and the first image. In some embodiments, the electronic device may use the fused image generated before the shooting instruction is generated as the second image. It may also perform image synthesis processing on the preview image and the first image displayed in the preview screen when the shooting instruction is detected to obtain a new fused image as the second image.

[0121] It can be understood that if the second shooting cannot make the area to be optimized meet the shooting conditions, for example, the second score of the fused image of the preview image and the first image cannot reach the second threshold, or there are too many areas to be optimized and the shooting conditions cannot be met simultaneously, the electronic device may guide the user to perform a third shooting, and determine whether the area to be optimized meets the shooting conditions based on the fused image of the preview image and the second image in the preview screen during the third shooting. If it still does not meet the conditions, a fourth shooting, a fifth shooting, etc. can also be performed until the area to be optimized meets the shooting conditions to obtain the final captured image.

[0122] Taking the electronic device as the mobile phone 10 and performing two shootings as an example, respectively based on Figure 2a the pedestrian removal scenario shown, Figure 2b the uneven lighting scenario shown,Figure 2c The group photo scene shown and Figure 2d The telephoto scene shown, introduce in detail the shooting method proposed by the embodiments of the present application.

[0123] Next, based on Figure 2a The pedestrian removal scene shown, combined with Figure 7 and Figure 8 Introduce a shooting method provided by the embodiments of the present application. Figure 7 Shows a Figure 2a Schematic diagram of the interaction process between the user and the mobile phone 10 in the pedestrian removal scene shown, Figure 8 According to some embodiments of the present application, shows a Figure 2a Schematic diagram of the process of the mobile phone 10 guiding shooting in the pedestrian removal scene shown.

[0124] As Figure 7 shown, the interaction process includes the following steps:

[0125] S101: The end user shoots the shooting target, performs a shooting operation, and forms a first image.

[0126] In this step, the user uses the mobile phone 10 to shoot the shooting target, performs a shooting operation, and forms a first image. The image acquisition module 101 in the mobile phone 10 can acquire the first image to perform subsequent steps.

[0127] As Figure 8 shown, the shooting operation of the user can be that the user clicks the shooting control 2011 in the shooting interface 201 of the mobile phone 10.

[0128] In this embodiment, the shooting interface 201 may include a shooting control 2011, a zoom control 2012, a preview screen 2013, and other controls. The user can click the shooting control 2011 to form a first image A1.

[0129] S102: The mobile phone 10 performs scene recognition on the first image.

[0130] In this step, after the image acquisition module 101 of the mobile phone 10 acquires the first image, it sends the first image to the scene recognition module 102. The scene recognition module 102 performs scene recognition on the first image, and sends the recognized scene type of the first image and the first image to the quality scoring module 103 and the image fusion module 104.

[0131] As Figure 8 shown, the first image A1 includes a shooting target P11 and a pedestrian P12, and the scene type of the first image can be a pedestrian removal scene.

[0132] S103: The mobile phone 10 performs an aesthetic quality score on the first image, determines the area to be optimized and the corresponding prompt information.

[0133] In this step, the quality scoring module 103 of the mobile phone 10 determines a quality scoring model corresponding to the scene type of the first image according to the scene type of the first image, and uses the quality scoring model to perform an aesthetic quality score on the first image to obtain a first score and the area to be optimized in the first image.

[0134] As Figure 8 shown, the quality scoring module 103 can select a quality scoring model corresponding to the pedestrian removal scene to perform an aesthetic quality score on the first image A1, and the focus is on whether there are irrelevant pedestrians affecting the quality of the first image A1. Exemplarily, the quality scoring model can include pedestrian detection and main area detection algorithms. The quality scoring module 103 can perform main area detection on the first image A1, determine that the shooting target P11 is the main body, and the pedestrian P12 is an irrelevant person. The pedestrian P12 will affect the score of the first image A1, making the first image A1 not meet the aesthetic quality requirements, and the score of the first image A1 cannot reach the first threshold. Therefore, the quality scoring module 103 can determine the area where the pedestrian P12 is located as the area to be optimized U11. And the mobile phone 10 can generate corresponding prompt information T11 according to the scoring result, and send the first image A1, the position information of the area to be optimized U11 and the prompt information T11 to the image display module 105.

[0135] Specifically, the quality scoring module 103 can set the full score of the quality score to 100 and the first threshold to 80. When the first score is greater than or equal to 80 points, it is considered that the image meets the aesthetic quality requirements and is a high-quality image; the quality scoring module 103 can also set the full score of the quality score to 1 and the threshold to 0.8. When the score of the image is greater than or equal to 0.8, it is considered that the image meets the aesthetic quality requirements and is a high-quality image. It should be understood that the full score and threshold of the quality score can also be set to other values, and the present application does not limit this.

[0136] Exemplarily, if the full score of the quality score is set to 100 and the first threshold is 80, since there is an area to be optimized U11 in the first image A1, the score of the first image can be 75 points, which does not reach the first threshold, and the first image A1 needs to be optimized.

[0137] Exemplarily, as Figure 8 shown, the prompt information T11 can be a text prompt, such as the words "Please select or adjust the area to be optimized and take another photo", and the text prompt can also be other content. The present application does not limit the specific text content of the prompt information.

[0138] In some embodiments, if the first image is an image with a similar style taken based on a template image selected by the user from the image library, the quality scoring model may adopt image registration and image difference algorithms to perform aesthetic quality scoring on the first image based on the template image.

[0139] S104: The mobile phone 10 visually displays the area to be optimized in the first image.

[0140] In this step, after the image display module 105 obtains the first image, the position information of the area to be optimized, and the prompt information from the quality scoring module 103, it visually displays at least one of the first image, the area to be optimized, and the prompt information to the user.

[0141] As Figure 8 shown, the shooting interface 301 may include the first image A1, the area to be optimized U11, the prompt information T11, the "OK" control 3011, and the "Cancel" control 3012. The user can interact with the mobile phone 10 according to the prompt information T11 in the shooting interface 301 and perform subsequent operations by clicking the "OK" control 3011 or the "Cancel" control 3012.

[0142] It can be understood that the area to be optimized U11 can be displayed with a first display element, and the first display element can be a rectangular frame as Figure 8 shown.

[0143] It can be understood that the shooting interface 301 may further include other controls. The above-mentioned first image A1, prompt information T11, "OK" control 3011, and "Cancel" control 3012 may also be displayed at other positions in the shooting interface 301, and the present application does not limit this.

[0144] S105: The end user adjusts the area to be optimized or adds a new area to be optimized and performs a second shooting.

[0145] In this step, the user can adjust the area to be optimized or add a new area to be optimized according to the prompt information and enter the shooting interface for the second shooting by clicking the "OK" control 3011.

[0146] In the embodiments of the present application, as Figure 8 shown, the user can perform corresponding operations according to the prompt information T11 in the shooting interface 301. For example, the user can adjust the area to be optimized U11 by clicking, dragging, zooming in or out, etc., to avoid the situation where the person to be retained is removed or the person to be removed is retained. The user can also manually add an operation in the first image A1 to add a new area to be optimized. After the operation is completed, the user can click the "OK" control 3011 to enter the shooting interface 401 for the second shooting.

[0147] In some embodiments, the user may also not perform corresponding operations according to the prompt information. For example, the user may not adjust the area to be optimized and directly click the "OK" control 3011 to enter the shooting interface 401. Another example is that the user may also click the "Cancel" control 3012 to return to the shooting interface 201.

[0148] S106: The mobile phone 10 acquires a preview stream.

[0149] In this step, the preview stream includes continuously acquired preview images. After the mobile phone 10 enters the second shooting, the image acquisition module 101 will acquire in real time through the camera preview images containing the shooting target and send the preview stream to the image fusion module 104 and the image display module 105.

[0150] As Figure 8 shown, after entering the second shooting, the mobile phone 10 can display the shooting interface 401, and the shooting interface 401 includes other controls such as a shooting control 2011, a zoom control 2012, and a preview screen 4013. The preview screen 4013 displays a preview image A2.

[0151] S107: The mobile phone 10 fuses the preview image and the first image to obtain a fused image.

[0152] In this step, the image fusion module 104 selects a corresponding image synthesis algorithm according to the scene type of the first image. After acquiring the preview image, the preview image and the first image are fused through the selected image synthesis algorithm to obtain a fused image. The image fusion module 104 sends the fused image to the image display module 105 for visual display to the user, and sends the fused image to the quality scoring module 103 for aesthetic quality scoring.

[0153] As Figure 8 shown, the shooting interface 401 may further include a fused screen 4014, and the fused screen 4014 is displayed in the shooting interface 401 in a picture-in-picture form, and the area to be optimized U11 is displayed in the fused screen 4014.

[0154] S108: The mobile phone 10 performs real-time aesthetic quality scoring on the fused image of the preview image and the first image.

[0155] In this step, the quality scoring module 103 performs real-time aesthetic quality scoring on the fused image of the preview image and the first image to determine whether the area to be optimized in the fused image is optimized.

[0156] It can be understood that the quality scoring module 103 performs aesthetic quality scoring on the fused image to determine the second score. The main purpose is to judge whether the area to be optimized in the fused image is optimized, such as Figure 8 whether the area to be optimized U11 in

[0157] S109: The mobile phone 10 displays the fused image of the preview image and the first image, the prompt information and the preview image in real time.

[0158] In this step, the image display module 105 displays the preview image, the fused image of the preview image and the first image, and the prompt information in the shooting interface of the second shooting. The prompt information can be used to guide the user to perform the second shooting.

[0159] As Figure 8 shown, since there is an unrelated pedestrian P12 in the area U11 to be optimized, the quality scoring module 103 can perform pedestrian motion estimation on the fused image A3, determine whether the unrelated pedestrian P12 leaves the area, etc. When it is detected that the pedestrian P12 leaves the area, it means that the area U11 to be optimized is optimized, and the fused image A3 meets the shooting conditions. At this time, the corresponding prompt information T12 can be generated and displayed.

[0160] Specifically, when the quality scoring module 103 detects that the unrelated pedestrian P12 is moving slowly, and there is still an unrelated pedestrian P12 or a part of the unrelated pedestrian P12 in the area U11 to be optimized in the fused image A3, the corresponding prompt information T12 is generated, and the prompt can be the text prompt "Please wait"; when the quality scoring module 103 detects that the unrelated pedestrian P12 is in a stationary state, and there is still an unrelated pedestrian P12 or a part of the unrelated pedestrian P12 in the area U11 to be optimized in the fused image A3, the corresponding prompt information T12 is generated, and the prompt can be the text prompt "Adjust the shooting angle"; when the quality scoring module 103 detects that there is no unrelated pedestrian in the area U11 to be optimized in the fused image A3, at this time, the second score of the fused image A3 reaches the second threshold, and the corresponding prompt information T12 generated can be the text prompt "Please shoot".

[0161] In some embodiments, when it is determined that the area U11 to be optimized is not optimized, the mobile phone 10 can display the color of the rectangular frame corresponding to the area U11 to be optimized as red, and when it is determined that the area U11 to be optimized is optimized, the color of the rectangular frame corresponding to the area U11 to be optimized can be displayed as green.

[0162] In some embodiments, the prompt information T12 can also be the specific value of the second score. For example, the second score of the fused image can be directly displayed in the preview screen 4013.

[0163] In some embodiments, as Figure 9As shown, when the mobile phone 10 is a folding screen mobile phone, the shooting interface 401 includes a main screen shooting interface 401a and a secondary screen display interface 401b. The fused image can also be displayed at a specific position (such as the lower half) of the secondary screen display interface 401b of the mobile phone 10. At this time, the first image can also be simultaneously displayed at a specific position (such as the upper half) of the secondary screen display interface 401b of the mobile phone 10, and a text description corresponding to the image can be displayed in the secondary screen display interface 401b. For example, the words "Final effect preview" are displayed above the fused image.

[0164] It should be understood that the fused image can also be displayed in the upper half of the secondary screen display interface 401b. Correspondingly, the first image can also be displayed in the lower half of the secondary screen display interface 401b. The proportions of the fused image and the first image in the secondary screen display interface 401b can be the same or different. The fused image can occupy more or less of the screen. This application does not make any limitations in this regard. The text description corresponding to the image can also be displayed below the image or at other positions. This application does not make any limitations in this regard.

[0165] S110: The end user performs relevant operations or remains unchanged according to the prompt information.

[0166] In this step, the user performs relevant operations or remains unchanged according to the prompt information in the shooting interface of the second shooting.

[0167] As Figure 8 shown, when the prompt information T12 is the text prompt "Please wait", the user can keep the current posture unchanged; when the prompt information T12 is the text prompt "Adjust the shooting angle", the user can appropriately adjust the shooting angle.

[0168] In some embodiments, when Figure 8 the rectangular frame of the area U11 to be optimized in is displayed in red, the user can also appropriately adjust the shooting angle.

[0169] It should be understood that the user can also not follow the prompt information and independently adjust the shooting angle or perform other operations.

[0170] S111: The mobile phone 10 performs an aesthetic quality score on the fused image of the preview image and the first image in real time, adjusts the prompt information; when the fused image meets the shooting conditions, it prompts the user that they can shoot.

[0171] In this step, the quality scoring module 103 performs an aesthetic quality score on the fused image of the preview image and the first image that changes in real time, adjusts the prompt information, and when it determines that the fused image meets the shooting conditions, it prompts the user that they can shoot.

[0172] Specifically, with reference to the prompt information corresponding to the judgment result of the quality scoring module 103 in step S108, the prompt information is adjusted accordingly based on the real-time judgment result of the fused image.

[0173] As Figure 8 shown, when the user adjusts the shooting angle and the irrelevant pedestrian P12 leaves the area U11 to be optimized, the quality scoring module 103 will determine that the fused image A3 meets the shooting conditions and prompt the user to shoot. Exemplarily, the prompt information T12 can be adjusted to the text prompt "Please shoot", prompting the user to click the shooting control 2011 to perform the shooting operation; or the color of the rectangular frame of the area U11 to be optimized can be adjusted from red to green, prompting the user to click the shooting control 2011 to perform the shooting operation; or a "√" pattern can be displayed on the shooting control 2011, prompting the user to click the shooting control 2011 to perform the shooting operation.

[0174] In some scenarios, as Figure 8 shown, even if there are other irrelevant pedestrians P13 in the preview image A2, since there are no pedestrians P13 in the first image A1 in this area, there will be no pedestrians P13 in the fused image A3 either.

[0175] S112: The end user performs a shooting operation to form a second image.

[0176] In this step, when the user sees the prompt information indicating that shooting is possible in the shooting interface of the second shooting, the user can click the shooting control to perform the shooting operation to form the second image of the second shooting.

[0177] S113: The mobile phone 10 fuses the preview image and the first image to obtain a second image.

[0178] In this step, the image acquisition module 101 acquires the preview image and sends the preview image to the image fusion module 104. The image fusion module 104 fuses the preview image with the first image to obtain a second image and sends the second image to the image display module 105.

[0179] In some embodiments, as Figure 8 shown, the mobile phone 10 can directly use the fused image A3 displayed in the fused screen 4014 as the second image. Alternatively, the image fusion module 104 fuses the preview image displayed in the preview screen when the user clicks the shooting control with the first image to obtain a new fused image, and uses this fused image as the second image.

[0180] S114: The mobile phone 10 presents the second image to the user as the final captured image.

[0181] In this step, the image display module 105 presents the second image to the user as the final captured image and saves the second image in the photo gallery of the mobile phone 10.

[0182] It should be understood that during the second shooting process, steps S106 to S111 are executed in a real-time loop before the user clicks the shooting control.

[0183] It can be understood that the embodiments of the present application can display the area to be optimized in the first image obtained by the first shooting. During the subsequent shooting process, the user can be guided to take multiple shots by means of prompt messages to optimize the image obtained by the first shooting. By fusing the images obtained by multiple shots, the defects in the image obtained by the first shooting can be compensated, so that a high-quality image can be obtained. Moreover, during the shooting process, the user can see the final shooting effect in real time, and it is easier to obtain an image satisfactory to the user, avoiding missing the best shooting opportunity.

[0184] Next, based on Figure 2b the shown uneven illumination scenario, combined with Figure 10 a shooting method proposed by the embodiments of the present application will be introduced. Figure 10 According to some embodiments of the present application, a schematic diagram of the process of guiding the shooting of the mobile phone 10 in the Figure 2b shown uneven illumination scenario is shown.

[0185] As Figure 10 shown, in this scenario, the illumination is very strong in some places and very weak in some places, and the difference is too large to obtain a satisfactory imaging result using a set of exposure parameters. When the user takes the first shot, the mobile phone 10 displays the shooting interface 201, and the shooting target P31 can be displayed in the preview screen 2013. When the user presses the shooting control 2011, the first image A4 is formed. After the image acquisition module 101 acquires the first image A4, the first image A4 is sent to the scene recognition module 102. The scene recognition module 102 performs scene recognition on the first image A4, determines that the scene type of the first image A3 is an uneven illumination scenario, and sends the scene recognition result to the quality scoring module 103 and the image fusion module 104.

[0186] In the embodiments of the present application, the quality scoring module 103 selects a corresponding quality scoring model according to the scene recognition result of the first image A4, and performs an aesthetic quality score on the first image A4. The focus is on whether there are underexposed or overexposed areas in the first image A4, which affects the score of the first image A4. Specifically, the quality scoring module 103 can determine that there is underexposure in the area U31 of the first image A4, resulting in a low score of the first image A4 and failing to reach the first threshold. The image display module 105 can display the area U31 in the shooting interface 301 and display the corresponding prompt message T31. It can be understood that the acquisition of the first image and the relevant description of the shooting interface can refer to the previous steps S101 to S104, which will not be elaborated here.

[0187] As Figure 10 shown, after the user clicks the "OK" control 3011 in the shooting interface 301 and enters the shooting interface 401 for the second shooting, the user can perform corresponding operations according to the prompt message T32. When the area U31 in the fused image is optimized, the user is prompted to take a picture.

[0188] Exemplarily, corresponding to the underexposure of the area U31 to be optimized, the prompt message T32 can be the words "Increase the exposure gain (gain) value"; corresponding to the overexposure of the area U31 to be optimized, the prompt message T32 can be the words "Decrease the exposure gain (gain) value", and the user can adjust the exposure parameters according to the prompt message T32. The specific content of the prompt message in the present application is not limited.

[0189] In some embodiments, the mobile phone 10 can also automatically calculate and adjust the exposure parameters without manual adjustment by the user.

[0190] It can be understood that the relevant description of the second shooting process can refer to the relevant description of the previous steps S105 to S114, which will not be elaborated here.

[0191] In the embodiments of the present application, by guiding the user to take multiple shots and fusing the images taken multiple times, an imaging result with appropriate exposure for each part can be obtained, enabling the user to obtain high-quality images.

[0192] Next, based on Figure 2c shown in the group photo scenario of multiple people, combined with Figure 11 this, a shooting method proposed in the embodiments of the present application will be introduced.

[0193] Figure 11 According to some embodiments of the present application, a schematic diagram of the process of the mobile phone 10 guiding shooting in the Figure 2c shown group photo scenario of multiple people is shown.

[0194] As Figure 11As shown, the shooting target P21 in this scenario is multiple people, and problems such as some people closing their eyes, having an unpleasant smile, or being blocked are likely to occur. After the first shooting to obtain the first image A5, the scene recognition module 102 first performs scene recognition on the first image A5, determines that this scene is a group photo scene, and the focus is on whether the expressions of each face in the shooting target P21 meet the requirements. The quality scoring module 103, according to the scene recognition result, first performs face detection on the first image A5 to obtain the position of each face, then performs aesthetic quality scoring on the obtained faces, and based on the scoring result, determines whether there is a face area with a low score, and takes the face area with a low score as the area to be optimized, and obtains the position information of this area. The image display module 105 displays the first image A5, the prompt information T21, and the area to be optimized U21 in the shooting interface 301 for visual display to the user.

[0195] In the embodiment of the present application, the user can also manually select the face to be optimized as the area to be optimized, and then click the "OK" control 3011 to enter the second shooting process.

[0196] It can be understood that the relevant descriptions of the acquisition of the first image and the visual display of the shooting interface can refer to the previous steps S101 to S104, and will not be elaborated here.

[0197] In the embodiment of the present application, after the user clicks the "OK" control 3011 in the shooting interface 301 to enter the shooting interface 401 with prompt information, the user can perform corresponding operations according to the prompt information T22. When the area to be optimized U21 in the fused image needs to be optimized, the user is prompted to take a photo.

[0198] Exemplarily, during the second shooting process, the prompt information T22 can also include a scoring score. For example, the scoring score of the face in the area to be optimized U21 in the fused image can be directly displayed above the area to be optimized U21 in the fused screen 4014. When the scoring of the face in the area to be optimized U21 does not reach the second threshold, it can be prompted in the form of text, such as the words "Please adjust your expression", to prompt the user that further adjustment is needed; when the scoring of the face in the area to be optimized U21 reaches the second threshold, it can be prompted in the form of text, such as the words "Please take a photo", to prompt the user to take a photo. Or the prompt information T22 can also be a pattern color prompt. For example, when the scoring of the face in the area to be optimized U21 does not reach the threshold, the outer frame color of the area to be optimized U21 is displayed in red, and when it reaches the threshold, the outer frame color of the area to be optimized U21 is displayed in green.

[0199] In some embodiments, the faces in the shooting target P21 can also be marked with a regular-shaped outer frame, and the color of the outer frame is used to prompt the user whether the faces in the shooting target P21 meet the scoring threshold. For example, a green frame indicates that the face meets the scoring threshold, and a red frame indicates that it does not meet the scoring threshold.

[0200] It can be understood that during the second shooting process, the image fusion module 104 can fuse only the face regions that need to be optimized, and the other regions that do not need to be optimized can remain unchanged.

[0201] It can be understood that other relevant descriptions of the second shooting process can refer to the relevant descriptions in the previous steps S105 to S114, and will not be elaborated here.

[0202] In the embodiments of the present application, by guiding the user to perform multiple shootings and fusing the images obtained from the multiple shootings, it can be ensured that each face in the shooting target meets the aesthetic quality requirements, thereby obtaining high-quality images.

[0203] Next, based on Figure 2d the long-focus scene shown, in combination with Figure 12 a shooting method proposed in the embodiments of the present application will be introduced.

[0204] Figure 12 According to some embodiments of the present application, a schematic diagram of the process of guiding the shooting of the mobile phone 10 in the Figure 2d long-focus scene shown is illustrated.

[0205] As Figure 12 shown, in this scene, due to the limited shooting range of the 10x long-focus camera, it is difficult to obtain a complete target image. After the first image A6 is obtained by shooting, the scene recognition module 102 first performs scene recognition on the first image A6, determines that the scene is a long-focus scene, and the focus is on whether there is a subject truncation in the shooting target P41. The quality scoring module 103 generates a corresponding prompt message T41 according to the scene recognition result. The image display module 105 displays the first image A6 and the prompt message T41 in the shooting interface 301 and visually presents them to the user. In this scene, there is no need to mark the area to be optimized.

[0206] It can be understood that the relevant descriptions of the acquisition of the first image and the visual display of the shooting interface can refer to the previous steps S101 to S104, and will not be elaborated here.

[0207] In the embodiments of the present application, after the user clicks the "OK" control 3011 in the shooting interface 301 and enters the shooting interface 401 with prompt information, the user can perform corresponding operations according to the prompt information T42. When the fused image meets the aesthetic quality requirements, the user is prompted that they can shoot.

[0208] Specifically, during the second shooting process, the image acquisition module 101 calls the main camera lens to obtain a panoramic image containing the complete shooting target P41, determines the position of the first image A6 in this panoramic image, and then generates one or more candidate viewfinder frames near the position of the first image A6 in the panoramic image. Refer to Figure 13 As shown, from the first image A6 captured by the telephoto lens and the panoramic image captured by the main camera, it can be seen that the viewfinder range of the main camera lens is wider than that of the telephoto lens. Therefore, a panoramic image of the shooting target P41 can be obtained through the main camera lens, and the position of the first image A6 (the area shown by the solid line frame) in the panoramic image can be determined, and the candidate viewfinder frames (the areas shown by the dashed line frames) near this position can be determined. It can be understood that according to the actual position of the first image in the panoramic image, one or more candidate viewfinder frames can be determined, and then the viewfinder frame with the optimal composition can be selected from these one or more candidate viewfinder frames. Among them, the viewfinder frame with the optimal composition is the candidate viewfinder frame that can completely accommodate the shooting target P41. Then, the direction in which the user needs to move the telephoto lens is calculated, and the corresponding prompt message T42 is generated. This prompt message T42 can be at least one of "move up", "move down", "move left", and "move right". The image fusion module 104 fuses the preview image and the first image A6 and shows the fusion result to the user in real time. The quality scoring module 103 performs an aesthetic quality score on the current fused image in real time. According to the scoring result, it is judged whether there is a problem of subject truncation in the current fused image, or whether an optimal composition effect has been achieved. When the fused image meets the aesthetic quality requirements, the user is prompted to take a photo.

[0209] Exemplarily, the panoramic image obtained by the main camera lens can be an image of 1000×1000, and the first image obtained by the telephoto lens can be an image of 100×100. And in the panoramic image of the first image, the panoramic image can be divided into 100 regions of 100×100 in size, the position or coordinates of the first image in the panoramic image are determined, candidate viewfinder frames are generated near the first image, the viewfinder frame with the optimal composition is selected from them, and according to the position of this candidate viewfinder frame, the direction in which the user needs to move is determined.

[0210] It can be understood that in this embodiment, when the image fusion module 104 fuses the preview image and the first image A6, it can be to splice the preview image and the first image A6. Exemplarily, both the first image A6 and the preview image are square. The same parts of the images in the first image A6 and the preview image can be overlapped, and the parts that are different from each other are jointly spliced into a fused image. Since this fused image may be a polygon, it is also necessary to fill this fused image into a regular rectangle. For example, the area that cannot be covered by the viewfinder can be filled with black or filled with the corresponding viewfinder area in the main camera result.

[0211] In the embodiments of the present application, after the user performs a second shooting operation to obtain a current preview image, the image fusion module 104 fuses the preview image and the first image to obtain a second image. The second image can be obtained by stitching the current preview image and the first image. Therefore, it is also necessary to crop the second image with the maximum inscribed rectangle or use an intelligent cropping algorithm to crop it to obtain a square image with a regular size as the final fused image to be presented to the user.

[0212] It can be understood that the fused image presented in the fusion screen 4014 of the shooting interface 401 with prompt information is only for showing the user the final fusion effect in real time. The image obtained by filling the area that cannot be covered by the viewfinder with black cannot be used as the final captured image because of the black part. The image obtained by filling it with the corresponding viewfinder area in the main camera result cannot be used as the final captured image either because there is a large difference in clarity between the image in the main camera result and the image obtained by the telephoto lens, and the clarity of each area in this image is inconsistent.

[0213] It can be understood that other related descriptions of the second shooting process can refer to the related descriptions in the previous steps S105 to S114, which will not be elaborated here.

[0214] In the embodiments of the present application, by guiding the user to perform multiple shootings and fusing and stitching the images obtained from the multiple shootings, the effect of expanding the field of view (FOV) of telephoto photography can be achieved, thereby obtaining high-quality images.

[0215] The reference in the specification to "an embodiment" or "embodiments" means that the specific features, structures, or characteristics described in connection with the embodiments are included in at least one exemplary implementation or technique disclosed in accordance with the embodiments of the present application. The appearance of the phrase "in an embodiment" in various places in the specification does not necessarily refer to the same embodiment.

[0216] The disclosure of the embodiments of the present application also relates to an operating device for performing the operations in the text. The device can be specifically constructed for the required purpose or it can include a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program can be stored in a computer-readable storage medium, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, application-specific integrated circuits (ASICs), or any type of medium suitable for storing electronic instructions, and each can be coupled to the computer system bus. In addition, the computers mentioned in the specification can include a single processor or can be an architecture involving multiple processors for increased computing power.

[0217] Additionally, the language used in this specification has been principally selected for readability and instructional purposes and may not have been selected to delineate or circumscribe the disclosed subject matter. Accordingly, the disclosure of embodiments of this application is intended to be illustrative rather than to limit the scope of the concepts discussed herein.

Claims

1. A shooting method, applied to an electronic device, characterized in that, including: in response to a first shooting instruction, obtaining a first image and displaying a first prompt message for prompting the user to shoot again; in response to a re-shooting instruction, displaying a shooting interface, wherein the shooting interface includes a preview screen and a second prompt message related to an area to be optimized of the first image; wherein the preview screen is used to display a preview image that changes in real time; corresponding to the area to be optimized meeting the shooting conditions, displaying a third prompt message for the user to shoot; in response to a second shooting instruction, obtaining a second image, where the second image includes a fusion image obtained by image synthesis processing of the preview image and the first image.

2. The shooting method according to claim 1, wherein further including: displaying a fusion screen in the shooting interface, and the second prompt message is displayed in the fusion screen or the preview screen; wherein the fusion screen is used to display the fusion image.

3. The shooting method according to claim 2, characterized in that, the fusion screen does not overlap with the preview screen; or at least part of the fusion screen overlaps with the preview screen.

4. The shooting method according to claim 1, characterized in that, the area to be optimized includes a first area obtained by performing image detection on the first image; the method further includes: determining the first prompt message and the second prompt message based on the image detection result of the first image; wherein the second prompt message includes shooting adjustment suggestions related to the first area.

5. The photographing method according to claim 4, wherein further including: determining the scene type of the first image; performing image detection on the first image according to the quality scoring model corresponding to the scene type to obtain the image detection result of the first image, and the image detection result of the first image includes a first score; corresponding to the first score being less than or equal to a first threshold, the image detection result of the first image includes the position information of the first area.

6. The photographing method according to claim 5, characterized in that displaying the first prompt message for prompting the user to shoot again includes: based on the position information of the first area, displaying the first area in the first image with a first display element to form the first prompt message for prompting the user to shoot again.

7. The photographing method according to claim 6, wherein further including: corresponding to the first area meeting the shooting conditions, changing the attribute of the first display element, and the attribute includes at least one of shape and color.

8. The photographing method according to any one of claims 1 to 6, characterized in that, further including: in response to a first operation by the user to add an area to be optimized, displaying a second area corresponding to the first operation in the first image with a second display element.

9. The photographing method according to claim 2, wherein further including: determining whether the area to be optimized meets the shooting conditions based on the image detection result of the fusion image; the image detection result of the fusion image includes a second score of the fusion image; corresponding to the second score being greater than or equal to a second threshold, determining that the area to be optimized meets the shooting conditions.

10. The shooting method according to claim 1, characterized in that, further including: determining whether the area to be optimized meets the shooting conditions based on the display change of the preview image in the preview screen; corresponding to a display change in the area corresponding to the area to be optimized in the preview image, determining that the area to be optimized meets the shooting conditions.

11. An electronic device, characterized in that, including: One or more processors, one or more memories, wherein the one or more memories store one or more programs, and when the one or more programs are executed by the one or more processors, cause the electronic device to execute the shooting method according to any one of claims 1 to 10.

12. A readable storage medium, characterized in that, Instructions are stored on the readable storage medium, and when executed on an electronic device, cause the electronic device to execute the shooting method according to any one of claims 1 to 10.

Images