Image acquisition method and device

Abstract

The invention discloses an image acquisition method and device, relates to the technical field of terminals, and can improve the dynamic definition of a target object in an image. The method comprises the following steps: acquiring multiple groups of images including the same target object through a first camera; if it is determined that relative motion exists between the target object and the electronic equipment based on the multiple sets of images and preset display content influencing the definition of the multiple sets of images exists in the multiple sets of images, multiple second short exposure frames are collected through the first camera; performing image preprocessing on the plurality of second short exposure frames to obtain a plurality of first target short exposure frames; and obtaining a first target image including the target object based on the plurality of first target short exposure frames. Based on the image acquisition method provided by the invention, the dynamic definition of the target object in the first target image can be improved, so that an image with high definition can be obtained in a target object snapshot scene, especially a portrait snapshot scene.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to an image acquisition method and device. Background Technology

[0002] With the development of camera functions on electronic devices, more and more users are accustomed to recording their lives or sharing their joys by taking pictures or videos, such as recording exercise, travel, and family time. In shooting scenarios, the target object (such as a person) in the field of view is often in motion. When electronic devices capture images of people in motion, the images of the people in the images are often blurry. Summary of the Invention

[0003] This application provides an image acquisition method and device that can improve the dynamic clarity of target objects in an image.

[0004] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0005] A first aspect provides an image acquisition method applied to an electronic device, the electronic device including a first camera. The method includes: acquiring multiple sets of images through the first camera, each set of images including the same target object, each set of images including at least one first short exposure frame and at least one first normal exposure frame; if it is determined based on the multiple sets of images that there is relative motion between the target object and the electronic device, and that preset display content exists in the multiple sets of images, acquiring multiple second short exposure frames through the first camera, the exposure duration of the second short exposure frames being less than the exposure duration of the first short exposure frames in each set of images, and the preset display content affecting the clarity of the multiple sets of images; performing image preprocessing on the multiple second short exposure frames to obtain multiple first target short exposure frames, the image preprocessing being used to remove the preset display content in the corresponding frames; and obtaining a first target image including the target object based on the multiple first target short exposure frames.

[0006] The solution provided in the first aspect above determines that there is relative motion between the target object and the electronic device based on multiple sets of images, and when multiple sets of images contain preset display content, multiple second short exposure frames are captured by the first camera, so that the multiple second short exposure frames can more clearly present the action of the target object. Then, image preprocessing is performed on the multiple second short exposure frames to eliminate the preset display content in the multiple second short exposure frames that affects the image clarity, thereby improving the overall image clarity of all short exposure frames. This also improves the dynamic clarity of the target object in the first target image, thus enabling the acquisition of high-clarity images in target object capture scenarios, especially in portrait capture scenarios.

[0007] In one possible implementation, the first camera acquires multiple second short exposure frames at a first frequency, and the electronic device further includes a second camera. The method further includes: while the first camera acquires multiple second short exposure frames, the second camera simultaneously acquires multiple third short exposure frames at the first frequency; performing image preprocessing on the multiple third short exposure frames to obtain multiple second target short exposure frames; and obtaining a first target image based on the multiple first target short exposure frames and combining the multiple second target short exposure frames. Thus, based on multiple sets of images, when it is determined that there is relative motion between the target object and the electronic device, and multiple sets of images contain preset display content, multiple second short exposure frames are captured by the first camera, and multiple third short exposure frames are captured simultaneously by the second camera. This allows the multiple second and third short exposure frames to more clearly present the action of the target object. Then, image preprocessing is performed on the multiple second and third short exposure frames respectively, which can eliminate the preset display content in the multiple second and third short exposure frames, thereby improving the overall image clarity of all short exposure frames. This also improves the dynamic clarity of the target object in the first target image, thus enabling the acquisition of high-clarity images in target object capture scenarios, especially in portrait capture scenarios.

[0008] As an example, the above method further includes: acquiring multiple second normal exposure frames at a third frequency using a first camera, wherein the multiple second normal exposure frames and multiple second short exposure frames are acquired alternately by the first camera, and the exposure duration of the second normal exposure frames is longer than the exposure duration of the second short exposure frames; while the first camera acquires the second normal exposure frames, acquiring multiple third normal exposure frames at a third frequency using a second camera, wherein the multiple third normal exposure frames and multiple third short exposure frames are acquired alternately by the second camera.

[0009] In one possible implementation, the preset display content includes one or more of the following: bars with varying brightness, screen objects, or black stripes. This takes into account the impact of non-target objects in the current shooting environment on image sharpness, thereby improving the sharpness of the subsequently generated target image.

[0010] In one possible implementation, the method further includes: while the first camera is capturing multiple second short exposure frames, the second camera is simultaneously capturing multiple third short exposure frames at a second frequency; the first frequency is N times the second frequency, where N is a positive integer.

[0011] As an example, N is 1. In this way, the power consumption of electronic devices can be reduced.

[0012] As an example, N > 1. Thus, compared to the example where N is 1, the power consumption of electronic devices can be further reduced.

[0013] In one possible implementation, the above-mentioned method of obtaining a first target image based on multiple first target short exposure frames and multiple second target short exposure frames includes: obtaining a first fused image based on multiple first target short exposure frames; obtaining a second fused image based on multiple second target short exposure frames; obtaining a first depth image based on the first fused image and the second fused image; and performing a blurring process on the first fused image based on the first depth image to obtain the first target image. In this way, the first target image can be fused with image content from different viewpoints, improving the clarity of the first target image. Furthermore, by blurring the first fused image using the first depth image, the first target image can maintain both dynamic clarity of the target object and a blurring effect. This is particularly beneficial in portrait capture scenarios, where it can also produce images with high dynamic clarity of the subject and a blurred background.

[0014] In one possible implementation, the method further includes: performing a first image segmentation process on the first fused image based on the first depth image, wherein the first image segmentation process is used to segment the target object and the background in the first fused image; and blurring the background in the first fused image. This allows the target object and the background in the first fused image to be separated, making it easier for the user's gaze to focus on the target object when viewing the target image.

[0015] In one possible implementation, the above-mentioned blurring process of the first fused image based on the first depth image further includes: performing a second image segmentation process on the target object in the first fused image based on the first depth image, wherein the second image segmentation process is used to segment local regions of the target object in the first segmented image; and processing the local regions according to the corresponding optimization algorithm. This can enhance the aesthetics or visual appeal of the local regions.

[0016] In one possible implementation, the method further includes: in a preview scenario, displaying the first target image as a preview image; or in a shooting scenario, displaying the first target image as a captured image. This improves the flexibility of image display.

[0017] In one possible implementation, the method further includes: in a shooting scenario, acquiring multiple auxiliary frames using a first camera, the auxiliary frames including: auxiliary long exposure frames and / or auxiliary short frames, wherein the exposure duration of the auxiliary long exposure frames is greater than the exposure duration of the first normal exposure frame in each group of images, and the brightness of the auxiliary short frames is less than the brightness of the first normal exposure frame and the first short exposure frame in each group of images; obtaining a third fused image based on the multiple first target short exposure frames and the multiple auxiliary frames; obtaining a second depth image based on the second fused image and the third fused image; and blurring the third fused image based on the second depth image to obtain a second target image including the target object. In this way, the real-time dynamics of the target object can be provided to the second target image, thereby reducing the difference between the second target image and the image observed by the user's naked eye.

[0018] In one possible implementation, the electronic device further includes a display screen. In a preview scenario, the method further includes: if the contrast ratio of the shooting environment is greater than a preset contrast ratio, displaying the fused display image corresponding to each group of images on the display screen; the fused display image is obtained by fusing the first short exposure frame and the first normal exposure frame in the corresponding image group; if the ambient brightness of the shooting environment is lower than a preset brightness, displaying the first normal exposure frame in each group of images on the display screen; if multiple groups of images contain preset display content, displaying the first short exposure frame in each group of images on the display screen. In this way, users can adjust parameters based on preview images that better match the shooting environment, making it easier for the electronic device to generate images that meet user expectations.

[0019] In one possible implementation, the method further includes: determining a reference exposure duration for the second short exposure frame based on the movement speed of the target object; if the reference exposure duration is greater than or equal to a preset minimum exposure duration, using the reference exposure duration as the exposure duration of the second short exposure frame, where the preset minimum exposure duration is determined based on the flicker frequency of the artificial light source, which causes bands of varying brightness in multiple images; if the reference exposure duration is less than the preset minimum exposure duration, using the preset minimum exposure duration as the exposure duration of the second short exposure frame. Thus, by determining the reference exposure duration based on the movement speed and then determining whether to directly use the reference exposure duration as the exposure duration of the second short exposure frame based on the flicker frequency of the artificial light source, the influence of the target object's movement speed and the artificial light source on image sharpness can be comprehensively considered, resulting in a more accurate exposure duration for the second short exposure frame.

[0020] In one possible implementation, the electronic device further includes a second camera, and the method further includes: if it is determined based on multiple sets of images that there is no relative motion between the target object and the electronic device, the first camera acquires multiple fourth short exposure frames at a first frequency and multiple second normal exposure frames at a third frequency, wherein the multiple second normal exposure frames and multiple fourth short exposure frames are acquired alternately by the first camera, the exposure duration of the second normal exposure frames is longer than the exposure duration of the fourth short exposure frames, and the exposure duration of the fourth short exposure frames is the same as the exposure duration of the first short exposure frames; while the first camera acquires the fourth short exposure frames, the second camera simultaneously acquires multiple fifth short exposure frames at a second frequency; image preprocessing is performed on the multiple fourth short exposure frames to obtain multiple third target short exposure frames; image preprocessing is performed on the multiple fifth short exposure frames to obtain multiple fourth target short exposure frames; and a third target image is obtained based on the multiple third target short exposure frames, the multiple fourth target short exposure frames, and the multiple second normal exposure frames. Thus, by acquiring only the fifth short exposure frames through the second camera, the power consumption of the electronic device can be reduced. Furthermore, obtaining the third target image based on the fourth fused image and the third depth image can improve the spatial sense of the third target image.

[0021] As an example, the above method further includes: obtaining a fourth fused image based on multiple second normal exposure frames; obtaining a third depth image based on multiple third target short exposure frames and multiple fourth target short exposure frames; and obtaining a third target image based on the fourth fused image and the third depth image.

[0022] In one possible implementation, the method further includes: if it is determined based on multiple sets of images that there is no relative motion between the target object and the electronic device, acquiring multiple second normal exposure frames using a first camera at a third frequency; while the first camera acquires the second normal exposure frames, simultaneously acquiring multiple third normal exposure frames using a second camera at the third frequency; and obtaining a fourth target image based on the multiple second and third normal exposure frames. Since the exposure duration of a normal exposure frame is longer than that of a short exposure frame, the normal exposure frame can more clearly capture the details and textures of the still object, thus improving the clarity of the fourth target image.

[0023] In a second aspect, an image acquisition device is provided, comprising: one or more cameras for acquiring images; a memory for storing computer program instructions; and a processor for executing the computer program instructions to support the image acquisition device in implementing the method as described in any possible implementation of the first aspect.

[0024] Thirdly, an electronic device is provided, comprising: one or more cameras for capturing images; a memory for storing computer program instructions; and a processor for executing the computer program instructions to support the electronic device in implementing the methods as described in any possible implementation of the first aspect.

[0025] In some embodiments, the image acquisition device may be equivalent to an electronic device; in other embodiments, the image acquisition device may be part of an electronic device, and this application does not limit this.

[0026] Fourthly, a computer-readable storage medium is provided that stores computer program instructions that, when executed by a processor, implement the method as described in any possible implementation of the first aspect.

[0027] Fifthly, a computer program product containing instructions is provided, which, when run on a computer, causes the computer to implement the method as described in any possible implementation of the first aspect.

[0028] Sixthly, a chip system is provided, comprising processing circuitry and a storage medium storing computer program instructions; when executed by a processor, the computer program instructions implement the method as described in any possible implementation of the first aspect. The chip system may be composed of chips or may include chips and other discrete devices. Attached Figure Description

[0029] Figure 1 This is a schematic diagram illustrating the effect of capturing an image.

[0030] Figure 2 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application;

[0031] Figure 3 A schematic flowchart illustrating an image acquisition method provided in an embodiment of this application;

[0032] Figure 4 This is a schematic diagram of a target image generated according to the image acquisition method provided in the embodiments of this application;

[0033] Figure 5 A schematic flowchart illustrating an image acquisition method according to another embodiment of this application;

[0034] Figure 6 This is a schematic diagram of a target image generation process provided in an embodiment of this application;

[0035] Figure 7 A schematic diagram illustrating the image acquisition process of a first camera and a second camera, provided for an embodiment of this application;

[0036] Figure 8 A preferred business process diagram provided for embodiments of this application when both the first camera and the second camera synchronously acquire images in stagger mode;

[0037] Figure 9 A flowchart illustrating an image acquisition method provided in another embodiment of this application;

[0038] Figure 10 A schematic diagram illustrating another image acquisition process of the first and second cameras provided in an embodiment of this application;

[0039] Figure 11 A schematic diagram illustrating another image acquisition process of the first and second cameras provided in an embodiment of this application;

[0040] Figure 12 This is a schematic diagram illustrating another process for generating a target image, as provided in an embodiment of this application.

[0041] Figure 13 A preferred business process diagram provided for embodiments of this application, when the second camera only captures short exposure frames at a second frequency (equal to the first frequency);

[0042] Figure 14 A preferred business process diagram provided for embodiments of this application, when the second camera only captures short exposure frames at a second frequency (e.g., equal to 1 / 2 of the first frequency);

[0043] Figure 15 This is a schematic diagram illustrating a method of controlling an electronic device in a portrait capture application scenario, as provided in an embodiment of this application.

[0044] Figure 16 This is a schematic diagram illustrating another method of controlling electronic devices in a portrait capture application scenario, as provided in an embodiment of this application. Detailed Implementation

[0045] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0046] In the following text, the terms "first," "second," etc., are used only to distinguish different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects. For example, if the described object is a "field," then the ordinal numbers before "field" in "first field" and "second field" do not limit the position or order of the "fields." "First" and "second" do not limit whether the "fields" they modify are in the same message, nor do they restrict the order of "first field" and "second field." Similarly, if the described object is a "level," then the ordinal numbers before "level" in "first level" and "second level" do not limit the priority of the "levels." Furthermore, the quantity of described objects is not limited by ordinal numbers and can be one or more; for example, in "first device," the number of "devices" can be one or more. In addition, objects modified by different prefixes can be the same or different. For example, if the described object is "device," then "first device" and "second device" can be devices of the same type or different types. Similarly, if the described object is "information," then "first information" and "second information" can be information with the same content or information with different content. In summary, the use of ordinal numbers and other prefixes used to distinguish the described objects in the embodiments of this application does not constitute a limitation on the described objects. The description of the described objects is given in the claims or the context of the embodiments, and the use of such prefixes should not constitute an unnecessary limitation.

[0047] Furthermore, in the embodiments of this application, "connection" can be a direct connection or an indirect connection; in addition, it can refer to an electrical connection or a communication connection; for example, the connection of two electrical components A and B can refer to A and B being directly connected, or it can refer to A and B being indirectly connected through other electrical components or connection media, or it can refer to A and B being indirectly connected through other communication devices or communication media, as long as it enables communication between A and B.

[0048] Currently, when capturing relative motion between an electronic device and a target object, excessively long exposure times can cause motion blur. Taking portrait photography as an example, one possible approach is for the electronic device to simultaneously activate two cameras. When motion is detected, the exposure time of both cameras is reduced, and then binocular depth calculation is performed using the images captured by both cameras to generate an image with reduced blur and a bokeh effect. However, in practice, to ensure the overall image quality after reducing the exposure time, the reduction is usually limited, resulting in insufficient dynamic sharpness of the subject in the image. For instance, when there is relative motion between the electronic device and the subject, and the subject's speed exceeds a preset speed, the exposure time can be reduced to generate an image with reduced blur and a blurred effect. Figure 1The image shown still exhibits significant motion blur due to the excessive speed of the person's movement. Similarly, in medium or low brightness environments, when there is relative motion between the electronic device and the person, the reduction in exposure time to minimize the impact of ambient brightness on overall image sharpness is limited, potentially resulting in significant motion blur in the generated image. The medium and low brightness environments can be determined based on a preset mapping relationship between brightness levels and ambient brightness ranges.

[0049] To address the motion blur issue that occurs when capturing images of a target object and improve the dynamic clarity of the target object in the image, this application provides an image acquisition method. This method uses two cameras to capture images of a target object (such as a person, cat, or dog). When relative motion between the target object and the electronic device is determined based on multiple sets of images captured by one camera, and when the multiple sets of images captured by one camera contain preset display content that affects image clarity, multiple short-exposure frames can be captured by one camera at a first frequency. The exposure time of these short-exposure frames is shorter than the exposure time of the multiple sets of images previously captured by one camera. Image preprocessing is performed on these short-exposure frames to remove the preset display content. While one camera is capturing multiple short-exposure frames, another camera simultaneously captures multiple short-exposure frames at the first frequency. Image preprocessing is also performed on these short-exposure frames to remove the preset display content. Thus, the target image can be obtained from the multiple short-exposure frames corresponding to the two cameras, each with the preset display content removed.

[0050] For example, preset display content affecting image clarity may include, but is not limited to, one or more of the following: screen objects, bars of varying brightness, or black stripes. A screen object can be understood as a screen existing within the image acquisition range; bars of varying brightness may be directly caused by flicker; black stripes may be indirectly caused by flicker. The cause of flicker can be that artificial light sources (such as fluorescent lamps, LED lights, etc.) generally operate under alternating current (AC). Affected by AC, the brightness of the artificial light source changes accordingly with the periodic variation of the AC amplitude, causing flicker. For example, if an artificial light source operates under 50 Hz AC, it flickers 100 times per second.

[0051] Furthermore, most digital cameras, mobile phones, tablets, and other electronic devices currently use a rolling shutter exposure method, which is line-by-line exposure. This means that in this method, the sensor (e.g., a complementary metal-oxide-semiconductor (CMOS) image sensor) begins exposure from the first row of pixels in an image, and after a one-line interval, it begins exposing the second row of pixels, resulting in differences in exposure time between different rows of pixels. Since there is an artificial light source in the currently captured image, the brightness of this artificial light source changes during exposure, causing differences in brightness between different rows of pixels, thus resulting in banding patterns in the image displayed by the electronic device. Moreover, even when there is no artificial light source in the currently captured image, but an artificial light source exists in the corresponding real space, the light from the artificial light source can be reflected in the real space, causing the captured image to be affected by reflected light, resulting in black stripes in the image displayed by the electronic device.

[0052] The embodiments of this application do not limit the reasons for the generation of preset display content such as screen objects, bars with varying brightness, or black stripes.

[0053] The image acquisition method provided in this application, when determining that there is relative motion between a target object and an electronic device based on multiple sets of images captured by a single camera, and when the multiple sets of images captured by the single camera contain preset display content, acquires multiple short-exposure frames by reducing the exposure time. This allows the multiple short-exposure frames to more clearly present the movement of the target object. Simultaneously, another camera can synchronously acquire multiple short-exposure frames at the same acquisition frequency as the first camera. Next, image preprocessing is performed on the multiple short-exposure frames acquired by each of the two cameras to remove the preset display content from the multiple short-exposure frames, thereby improving the overall clarity of all short-exposure frames. Then, the target image is obtained from the multiple short-exposure frames corresponding to each of the two cameras after removing the preset display content, thus improving the dynamic clarity of the target object in the target image. This enables the acquisition of high-clarity images even in target object capture scenarios, especially in portrait capture scenarios.

[0054] The electronic devices provided in this application have image capture capabilities. Exemplary examples include, but are not limited to, smartphones, netbooks, tablets, in-vehicle devices, wearable devices (such as smartwatches, smart bracelets, smart glasses, etc.), cameras (such as SLR cameras, point-and-shoot cameras, etc.), personal computers (PCs), handheld computers, smart TVs, personal digital assistants (PDAs), portable multimedia players (PMPs), projection devices, smart screen devices, augmented reality (AR) / virtual reality (VR) devices, mixed reality (MR) devices, televisions, or motion-sensing game consoles in human-computer interaction scenarios. This application does not limit the specific functions and structures of the electronic devices.

[0055] The hardware structure diagram of electronic device 100 can be shown as follows Figure 2 As shown. In Figure 2 The electronic device 100 may include a processor 110, a memory 120, a display screen 130, and a camera 140.

[0056] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0057] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), audio processor / digital processor, controller, memory, video codec, audio codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. The NPU, by referencing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, can quickly process input information and continuously learn, enabling electronic device 100 to achieve intelligent cognitive applications, such as image recognition, face recognition, speech recognition, and text understanding. Different processing units may be independent devices or integrated into one or more processors.

[0058] In this embodiment, the processor 110 can be used to collaboratively process images (such as images captured by the first camera 141 and the second camera 142), execute user operation commands, and realize image capture, video capture, etc. For example, the processor 110 can collaboratively process images using a GPU and an ISP.

[0059] The memory 120 may include volatile memory, such as random access memory (RAM). The memory 120 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD). The memory 120 stores executable program code, which the processor 110 executes to implement the image acquisition method provided in this embodiment.

[0060] As an example, the program code stored in the memory is mainly used to implement the following process: Images of a target object (such as a person, cat, or dog) are acquired using two cameras. Motion detection and preset display content detection are performed on multiple sets of images acquired by one camera. Motion detection is used to detect whether there is relative motion between the target object and the electronic device, and preset display content detection is used to detect whether preset display content exists in the multiple sets of images. When it is determined whether there is relative motion between the target object and the electronic device, and that preset display content exists in the multiple sets of images acquired by one camera, multiple short-exposure frames are acquired by one camera at a first frequency. The exposure duration of these short-exposure frames is less than the exposure duration of the multiple sets of images previously acquired by one camera. Image preprocessing is performed on these short-exposure frames to remove the preset display content. While one camera is acquiring multiple short-exposure frames, another camera synchronously acquires multiple short-exposure frames at the first frequency. Image preprocessing is also performed on these short-exposure frames to remove the preset display content. Thus, the target image is obtained from the multiple short-exposure frames corresponding to the two cameras, each with the preset display content removed.

[0061] Electronic device 100 implements display functions through a graphics processing unit (GPU), a display screen 130, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 130 and the application processor. The GPU performs data and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0062] The display screen 130 is used to display images, videos, etc. The display screen 130 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or more display screens 130.

[0063] In this embodiment, the display screen 130 can be used to display an interface including a camera icon, a shooting interface including operation function options such as video shooting buttons, preview images, and captured images.

[0064] The electronic device 100 can implement the shooting function through an image signal processor (ISP), a camera 140, a video codec, a GPU, a display screen 130, and an application processor. In some embodiments, the ISP can be located in the camera 140, but this application does not limit this.

[0065] Camera 140 is used to capture still images or videos. An object passes through the lens, generating an optical image that is projected onto a photosensitive element. This photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP (Image Signal Processor) for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP (Digital Signal Processor) for further processing. The DSP converts the digital image signal into standard RGB (red, green, blue) image signals, YUV, or other formats.

[0066] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP performs Fourier transforms on the frequency energy.

[0067] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc. In this embodiment, camera 140 may include at least a first camera 141 and a second camera 142. Both the first camera 141 and the second camera 142 may be front-facing cameras or both may be rear-facing cameras. The cameras included in camera 140 may be optical zoom lenses, etc., and this application does not limit this.

[0068] In this setup, one of the first camera 141 and the second camera 142 can be a primary camera, and the other can be a wide-angle camera. The primary camera can refer to the most important camera in the electronic device 100, and is typically the camera with the highest pixel count among the cameras 140. The wide-angle camera can refer to a camera with a wider field of view than the primary camera, enabling the capture of a larger scene.

[0069] In this embodiment, to improve the quality of the acquired images, the first camera and the second camera can switch according to the current focal length of the application with shooting function in the electronic device (such as a social application, shopping application, camera application, etc.). For example, the electronic device 100 may have a main camera and a wide-angle camera. When the application with shooting function is currently in the 1x-3x focal length range, the first camera 141 can be the main camera, and the second camera 142 can be the wide-angle camera; when the application with shooting function is currently in the 3x-10x focal length range, the first camera 141 can switch from the main camera to the wide-angle camera, and the second camera 142 can switch from the wide-angle camera to the main camera.

[0070] In some examples, the first camera 141 and the second camera 142 can alternately capture normal exposure frames and short exposure frames, respectively.

[0071] In some examples, the first camera 141 alternately captures normal exposure frames and short exposure frames; the second camera 142 can capture short exposure frames but not normal exposure frames.

[0072] In some examples, when relative motion between the target object and the electronic device 100 is detected, the exposure duration of the short-exposure frames captured by the first camera 141 and the second camera 142 can be reduced.

[0073] In some examples, the first camera 141 can respond to a shooting command by capturing long exposure frames, short frames, etc.

[0074] The following will describe in detail an image acquisition method provided by an embodiment of this application, with reference to the accompanying drawings.

[0075] like Figure 3 As shown, the image acquisition method may include S310-S360:

[0076] Step S310: Acquire multiple sets of images through the first camera. All sets of images include the same target object. Each set of images includes at least one first short exposure frame and at least one first normal exposure frame.

[0077] The target object can refer to the object that the user wants to photograph. The user can photograph one or more objects. The target object can include, but is not limited to, one or more of the following: people, cats, dogs, vehicles, etc.

[0078] As an example, multiple sets of images may all include the same target object, such as multiple sets of images all including person A. Optionally, any set of images may also include other target objects, and the other target objects included in multiple sets of images may be the same or different, without limitation.

[0079] As an example, the exposure time of the first short exposure frame is shorter than that of the first normal exposure frame. The brightness of the first short exposure frame is the same as that of the first normal exposure frame, so the sensitivity of the first short exposure frame can be higher than that of the first normal exposure frame.

[0080] As one possible approach, an application with a camera function can be installed in the electronic device. When the application with the camera function is launched, the electronic device can use a first camera to capture multiple first short exposure frames and multiple first normal exposure frames according to the stagger mode (a mode in which short exposure frames and normal exposure frames are captured alternately) to obtain multiple sets of images.

[0081] The first camera may first capture a first normal exposure frame and then capture a first short exposure frame; or the first camera may first capture a first short exposure frame and then capture a first normal exposure frame. This application does not limit this.

[0082] Optionally, applications with camera functionality can be launched by the user via voice control of the electronic device. For example, the user can wake up the electronic device's artificial intelligence (AI) assistant and launch the application with camera functionality through voice interaction with the AI ​​assistant. Alternatively, applications with camera functionality can be launched by the user clicking on the corresponding controls displayed on the electronic device's screen.

[0083] Optionally, when an application with shooting capabilities is launched, the first and second cameras can be activated simultaneously.

[0084] Step S320: If it is determined that there is relative motion between the target object and the electronic device based on multiple sets of images, and there is preset display content in the multiple sets of images, multiple second short exposure frames are captured by the first camera. The exposure time of the second short exposure frame is less than the exposure time of the first short exposure frame in each set of images. The preset display content will affect the clarity of the multiple sets of images.

[0085] As an example, relative motion between the target object and the electronic device can include the following situations: The first is that the position of the target object does not change, but the position of the electronic device changes. The change in the position of the target object can refer to a change in the overall position of the target object or a change in the position of a local area of ​​the target object, such as tilting the head or swaying the body; the second is that the position of the target object changes, but the position of the electronic device does not change; the third is that the positions of both the target object and the electronic device change, but there is a speed difference between the target object and the electronic device.

[0086] As an example, preset content may include one or more of the following: a band with varying brightness, a screen object, or a black stripe.

[0087] As one possible implementation, motion detection and preset display content detection can be performed based on multiple sets of images to obtain motion detection results and preset display content detection results. If it is determined based on the motion detection results and preset display content detection results that there is relative motion between the target object and the electronic device, and preset display content exists in multiple sets of images, multiple second short exposure frames can be captured by the first camera.

[0088] Optionally, the first camera can capture multiple second short exposure frames at a first frequency. For example, the first frequency can be 30 FPS (frames per second), which means capturing 30 second short exposure frames per second.

[0089] Optionally, motion detection of the same target object in multiple sets of images can be performed using optical flow to determine whether there is relative motion between the target object and the electronic device.

[0090] Optionally, object detection algorithms can be used to identify whether there is preset display content in multiple sets of images. Object detection algorithms may include, but are not limited to, one or more of the following: region-based convolutional neural network (R-CNN), you only look once (YOLO), single shotultibox detector (SSD), etc.

[0091] In this embodiment of the application, by setting the preset display content to one or more of the following: a stripe, a screen object, or a black stripe that affects the brightness of the image, the influence of non-target objects in the current shooting environment on the image clarity can be taken into account, thereby improving the clarity of the target image generated subsequently.

[0092] Optionally, to improve the efficiency of motion detection and preset display content detection, multiple sets of images can be downsampled (e.g., SVE) to obtain multiple sets of compressed images. The number of pixels in the multiple sets of compressed images can be less than that in the multiple sets of images. Thus, motion detection and preset display content detection can be performed based on the multiple sets of compressed images to obtain motion detection results and preset display content detection results.

[0093] Optionally, the image used for motion detection and preset display content detection can be the same as the image displayed on the screen. After the application with shooting function is launched, multiple sets of images captured by the first camera can be displayed on the screen. When the first short exposure frame from the multiple sets of images is displayed on the screen, motion detection and preset display content detection can be performed based on the multiple first short exposure frames to obtain motion detection results and preset display content detection results. When the first normal exposure frame from the multiple sets of images is displayed on the screen, motion detection and preset display content detection can be performed based on the multiple first normal exposure frames to obtain motion detection results and preset display content detection results. When the first short exposure frame and the first normal exposure frame from the multiple sets of images are fused and displayed on the screen, motion detection and preset display content detection can be performed based on the multiple first short exposure frames and the multiple first normal exposure frames to obtain motion detection results and preset display content detection results.

[0094] Step S330: While the first camera is capturing multiple second short exposure frames, the second camera is simultaneously capturing multiple third short exposure frames at the first frequency.

[0095] The exposure duration of the third short exposure frame can be the same as that of the second short exposure frame.

[0096] As one possible approach, while the first camera is capturing multiple second short exposure frames, the second camera can simultaneously capture multiple third short exposure frames at a first frequency. In other words, at the same time, a second short exposure frame from the first camera and a third short exposure frame from the second camera can be acquired.

[0097] Optionally, when the first camera captures multiple second short exposure frames, multiple third short exposure frames can be captured simultaneously by the second camera at the first frequency without capturing normal exposure frames.

[0098] Optionally, when the first camera captures multiple second short exposure frames, it can not only capture multiple third short exposure frames synchronously at the first frequency, but also capture multiple third normal exposure frames at the third frequency. The multiple third normal exposure frames and the multiple third short exposure frames are captured alternately by the second camera, and the exposure duration of the third normal exposure frame is longer than that of the third short exposure frame.

[0099] In this embodiment, by acquiring multiple third short exposure frames without acquiring normal exposure frames, the power consumption of the electronic device can be reduced. By acquiring multiple third normal exposure frames and multiple third short exposure frames, the number and types of images acquired can be increased. This allows for the selection of multiple third normal exposure frames or multiple third short exposure frames to generate the target image according to different scenarios during subsequent image acquisition, thereby improving the flexibility of the image acquisition method and enhancing the quality of the target image (such as sharpness and color richness) in different scenarios.

[0100] As another possible implementation, while the first camera is capturing multiple second short exposure frames, the second camera can simultaneously capture multiple third short exposure frames at a second frequency, where the first frequency can be N times the second frequency, and N is a positive integer.

[0101] Optionally, N can be equal to 1, in which case the second frequency is the same as the first frequency. In this case, multiple third short exposure frames can be collected synchronously by the second camera at the second frequency, without collecting normal exposure frames.

[0102] Optionally, N can be an integer greater than 1 (such as 2, 3, 4, etc.), in which case the second frequency is less than the first frequency. In this case, the second camera can synchronously capture multiple third short-exposure frames at the second frequency without capturing normal exposure frames. For example, when the first frequency is 30 FPS and N = 2, the second frequency can be 15 FPS, meaning that when the first camera captures two second short-exposure frames, the second camera only captures one third short-exposure frame. In this embodiment, when N is an integer greater than 1, the frequency at which the second camera captures third short-exposure frames can be reduced. Compared to N = 1 and capturing multiple third short-exposure frames without capturing normal exposure frames, this can further reduce the power consumption of the electronic device.

[0103] Step S340: Perform image preprocessing on multiple second short exposure frames to obtain multiple first target short exposure frames. The image preprocessing is used to remove preset display content in the corresponding frames.

[0104] As an example, image preprocessing can remove preset display content from a corresponding frame using one or more algorithms, including but not limited to AI-debanding and dithering. AI-debanding algorithms can refer to removing preset display content through deep learning or machine learning methods, such as functions in deep learning or machine learning software development tools used for removing preset display content, or network models trained on large amounts of image data for removing preset display content. Dithering algorithms remove preset display content by adding random pixel offsets to the corresponding frame, essentially adding random noise. The first target short exposure frame can refer to the second short exposure frame after the preset display content has been removed.

[0105] As one possible implementation, multiple second short exposure frames can be preprocessed to obtain multiple first target short exposure frames.

[0106] Step S350: Perform image preprocessing on multiple third short exposure frames to obtain multiple second target short exposure frames.

[0107] The second target short exposure frame can refer to the third short exposure frame with the preset display content removed.

[0108] As one possible approach, multiple third short exposure frames can be preprocessed according to the implementation method of step S340 to obtain multiple second target short exposure frames.

[0109] Step S360: Based on multiple first target short exposure frames, combine multiple second target short exposure frames to obtain a first target image including the target object.

[0110] The first target image can refer to the image generated by the electronic device in a scenario where there is relative motion between the target object and the electronic device, and the acquired image contains preset display content.

[0111] One possible approach is to directly obtain the first target image based on multiple short-exposure frames of the first target and multiple short-exposure frames of the second target.

[0112] In this embodiment, a first target image is obtained by directly combining multiple first target short exposure frames with multiple second target short exposure frames. This allows the first target image to fuse image content from different viewpoints, improving the clarity of the first target image. Furthermore, by blurring the first fused image using a first depth image, a blurring effect can be achieved in the first target image. This ensures that the first target image maintains both the dynamic clarity of the target object and the blurring effect. In particular, in portrait capture scenarios, images with high dynamic clarity of the person and a blurred background effect can be obtained.

[0113] As another possible implementation, a preset display content detection can be performed on multiple first target short exposure frames and multiple second target short exposure frames. If the multiple first target short exposure frames and multiple second target short exposure frames are found to meet preset conditions, then a first target image can be obtained based on the multiple first target short exposure frames and multiple second target short exposure frames. If the multiple first target short exposure frames and multiple second target short exposure frames are found to not meet the preset conditions, then a fourth target image can be obtained based on multiple second normal exposure frames and multiple third normal exposure frames.

[0114] The preset conditions may include: there is no preset display content in any of the multiple first target short exposure frames and multiple second target short exposure frames, or the specified percentage is less than the preset percentage. The specified percentage may refer to the ratio of the number of frames in the multiple first target short exposure frames and multiple second target short exposure frames without preset display content to the total number of the multiple first target short exposure frames and multiple second target short exposure frames. The preset percentage can be set according to actual needs, such as 80%.

[0115] In this embodiment, when multiple first target short exposure frames and multiple second target short exposure frames meet preset conditions, a first target image is obtained based on the multiple first target short exposure frames and combined with the multiple second target short exposure frames. This avoids the presence of preset display content in the first target image and improves the overall clarity of the first target image. Furthermore, by confirming whether the multiple first target short exposure frames and multiple second target short exposure frames meet the preset conditions, feedback on the image preprocessing results of steps S340 and S350 can be obtained, allowing researchers to determine whether the currently used image preprocessing algorithm needs to be improved.

[0116] As another approach, multiple short-exposure frames of the first target can be directly fused to obtain the first target image.

[0117] In this embodiment, the first target image is obtained by directly fusing multiple short exposure frames of the first target. This can reduce the blurriness of the first target image to a certain extent and save the resources required for image processing.

[0118] In this embodiment, when relative motion between the target object and the electronic device is determined based on multiple sets of images, and multiple sets of images contain preset display content, multiple second short-exposure frames are captured by the first camera, and multiple third short-exposure frames are simultaneously captured by the second camera. This allows the multiple second and third short-exposure frames to more clearly present the movement of the target object. Then, image preprocessing is performed on the multiple second and third short-exposure frames respectively to eliminate the preset display content in the multiple second and third short-exposure frames, improving the overall clarity of all short-exposure frames. This also improves the dynamic clarity of the target object in the first target image, thereby enhancing the image clarity in target object capture scenarios, especially portrait capture scenarios, compared to... Figure 4 The image shown on the right was obtained using the image acquisition method provided in this application. Figure 4 The image shown on the left has higher resolution.

[0119] The following will, in conjunction with the accompanying drawings, take as an example an image acquisition method provided by an embodiment of this application, in which both the first and second cameras of an electronic device acquire images in stagger mode.

[0120] like Figure 5 As shown, an image acquisition method provided in this application embodiment may include S5010-S5110:

[0121] Step S5010: Acquire multiple sets of images through the first camera. All sets of images include the same target object. Each set of images includes at least one first short exposure frame and at least one first normal exposure frame.

[0122] Optionally, when the first camera acquires multiple sets of images, the second camera can also acquire multiple sets of images through stagger mode. Each set of images includes two images, one of which has the same exposure duration as the first short exposure frame, and the other image has the same exposure duration as the first normal exposure frame.

[0123] Step S5020: If it is determined that there is relative motion between the target object and the electronic device based on multiple sets of images, and there is preset display content in the multiple sets of images, multiple second short exposure frames are acquired by the first camera at a first frequency. The exposure time of the second short exposure frame is less than the exposure time of the first short exposure frame in each set of images. The preset display content will affect the clarity of the multiple sets of images.

[0124] As one possible approach, when it is determined that there is relative motion between the target object and the electronic device, and there is preset display content in multiple sets of images, the exposure duration of the second short exposure frame can be determined based on the movement speed of the target object and the flicker frequency of the artificial light source. In other words, the exposure duration of the second short exposure frame can be adjusted in real time based on the movement speed of the target object and the flicker frequency of the artificial light source.

[0125] Optionally, the reference exposure duration of the second short exposure frame can be determined based on the movement speed of the target object. If the reference exposure duration is greater than or equal to a preset minimum exposure duration, the reference exposure duration is used as the exposure duration of the second short exposure frame. The preset minimum exposure duration is determined based on the flicker frequency of the artificial light source, which causes bands of brightness variation to exist in multiple sets of images. If the reference exposure duration is less than the preset minimum exposure duration, the preset minimum exposure duration is used as the exposure duration of the second short exposure frame. For example, a preset minimum exposure duration corresponding to a flicker frequency of 50Hz can be 10ms.

[0126] Optionally, the motion speed can have a one-to-one mapping relationship with the reference exposure time. The reference exposure time can be determined directly based on the motion speed of the target object. The faster the motion speed, the shorter the reference exposure time can be.

[0127] Optionally, the motion speed level (such as low speed, medium speed, high speed, etc.) can have a one-to-one mapping relationship with the reference exposure time. Each motion speed level corresponds to a motion speed range. The motion speed level can be determined based on the motion speed, and the reference exposure time can be determined based on the motion speed level.

[0128] For example, the mapping relationship between movement speed and movement speed level can be shown in Table 1. When the movement speed of the target object is detected to be a1 km / h, since a1 is within (0, A1], the movement speed level of the target object can be low. The relationship between movement speed level and reference exposure time can be shown in Table 2. When the movement speed level of the target object is low, the reference exposure time can be determined to be 20 ms according to Table 2.

[0129] Table 1

[0130] Speed ​​of movement (km / h) Sport speed gear (0，A1] low speed (A1, A2) medium speed (A2, A3) high speed

[0131] Table 2

[0132] Sport speed gear Reference exposure time (ms) low speed 20 medium speed 15 high speed 8

[0133] In this embodiment, the reference exposure time is determined by the movement speed, and then the flicker frequency of the artificial light source is used to determine whether the reference exposure time is directly used as the exposure time of the second short exposure frame. This can comprehensively consider the movement speed of the target object and the influence of the artificial light source on the image clarity, and obtain a more accurate exposure time of the second short exposure frame.

[0134] As one possible approach, when it is determined that there is relative motion between the target object and the electronic device, and there is no preset display content in multiple sets of images, the exposure duration of the second short exposure frame can be determined based on the speed of motion of the target object.

[0135] Step S5030: The first camera acquires multiple second normal exposure frames at a third frequency. The multiple second normal exposure frames and multiple second short exposure frames are acquired alternately by the first camera. The exposure duration of the second normal exposure frame is longer than the exposure duration of the second short exposure frame.

[0136] As one possible approach, if relative motion between the target object and the electronic device is determined based on multiple sets of images, and there is preset display content in the multiple sets of images, multiple second normal exposure frames can be acquired by the first camera at a third frequency. The multiple second normal exposure frames and multiple second short exposure frames are acquired alternately by the first camera, and the exposure duration of the second normal exposure frame is longer than the exposure duration of the second short exposure frame.

[0137] The third frequency can be the same as the first frequency. The first camera can first capture a second normal exposure frame and then capture a second short exposure frame; or the first camera can first capture a second short exposure frame and then capture a second normal exposure frame. This application does not limit this.

[0138] It should be noted that since multiple second normal exposure frames and multiple second short exposure frames are captured alternately by the first camera, and the first frequency and the third frequency are the same, the actual acquisition frequency of the first camera can be twice the first frequency. For example, when the first frequency is 30 FPS, the actual acquisition frequency of the first camera can be 60 FPS.

[0139] Optionally, when acquiring multiple second normal exposure frames, the exposure duration of the second normal exposure frame can be adjusted in real time based on the current ambient brightness and the proportion of bright and dark areas in the currently acquired second normal exposure frame.

[0140] Step S5040: While the first camera is capturing multiple second short exposure frames, the second camera is simultaneously capturing multiple third short exposure frames at the first frequency.

[0141] For details regarding the second camera simultaneously capturing multiple third short-exposure frames at the first frequency, please refer to the above text. Figure 3 The introduction of the S330 will not be repeated here.

[0142] Step S5050: When the first camera captures the second normal exposure frame, the second camera captures multiple third normal exposure frames at a third frequency. The multiple third normal exposure frames and multiple third short exposure frames are captured alternately by the second camera.

[0143] As one possible approach, while the first camera is capturing the second normal exposure frame, the second camera captures multiple third normal exposure frames at a third frequency, wherein the exposure duration of the third normal exposure frame is longer than the exposure duration of the third short exposure frame.

[0144] Optionally, the exposure duration of the third normal exposure frame can be synchronized with the exposure duration of the second normal exposure frame.

[0145] Step S5060: Perform image preprocessing on multiple second short exposure frames to obtain multiple first target short exposure frames. The image preprocessing is used to remove preset display content in the corresponding frames.

[0146] For information on image preprocessing of multiple second short-exposure frames to obtain multiple first target short-exposure frames, please refer to the above text. Figure 3 The introduction of the S340 will not be repeated here.

[0147] Step S5070: Perform image preprocessing on multiple third short exposure frames to obtain multiple second target short exposure frames.

[0148] For information on image preprocessing of multiple third short-exposure frames to obtain multiple second target short-exposure frames, please refer to the above text. Figure 3 The introduction of the S350 will not be repeated here.

[0149] Step S5080: Based on multiple short exposure frames of the first target, and combined with multiple short exposure frames of the second target, a first target image is obtained.

[0150] As one possible way to achieve this, after executing Figure 6 After the target object shown in S610 moves relative to the electronic device, such as Figure 6 As shown in S650, a first fused image is obtained based on multiple short-exposure frames of the first target; a second fused image is obtained based on multiple short-exposure frames of the second target; as shown in... Figure 6 As shown in S660, a first depth image is obtained based on the first fused image and the second fused image; as... Figure 6 As shown in S670, the first fused image is blurred based on the first depth image to obtain the first target image.

[0151] The first fused image can refer to an image obtained by fusing multiple short-exposure frames of a first target, and the second fused image can refer to an image obtained by fusing multiple short-exposure frames of a second target. In this embodiment, the first fused image and the second fused image can be YUV images. The first depth image can refer to a depth image obtained by performing binocular depth calculation based on the first fused image and the second fused image, where the grayscale value of each pixel can be used to characterize the distance between the corresponding location in the scene and the electronic device.

[0152] Optionally, a first image segmentation process can be performed on the first fused image based on the first depth image. The first image segmentation process is used to segment the target object and the background in the first fused image; and the background in the first fused image is blurred (Bokeh).

[0153] Here, the background can refer to the content in the first fused image other than the target object. The algorithm corresponding to the first image segmentation processing can be an edge detection algorithm (such as, but not limited to, one or more of Canny edge detection, Sobel operator, etc.), a deep learning-based algorithm (such as, but not limited to, one or more of FCN, U-Net, etc.), etc.

[0154] In this embodiment of the application, performing a first image segmentation process on the first fused image can separate the target object and the background in the first fused image, making it easier for the user to focus their gaze on the target object when viewing the target image.

[0155] Optionally, a second image segmentation process can be performed on the target object in the first fused image based on the first depth image. The second image segmentation process is used to segment out the local region of the target object in the first segmented image; and the local region is processed according to the optimization algorithm corresponding to the local region.

[0156] Taking a person as an example, the local region can refer to hair, face, eyes, etc. The optimization algorithm for the local region can refer to an algorithm that adjusts the local region to better suit the user's aesthetic preferences (such as pursuing beauty or a sense of fun). For example, a hair segmentation algorithm can be used to highlight the details of a person's hair, making the hair in the first target image more dynamic and flowing. Hair segmentation algorithms can include, but are not limited to, semantic soft segmentation (SSS). Another example is using one or more algorithms such as dilation and liquefaction to enlarge the person's eyes.

[0157] In this embodiment of the application, the target object in the first fused image is subjected to second image segmentation processing, and the segmented local regions are optimized, which can increase the aesthetics or interest of the local regions.

[0158] As another possible implementation, a first fused image can be obtained based on multiple short exposure frames of the first target; a second fused image can be obtained based on multiple short exposure frames of the second target; a first depth image can be obtained based on the first fused image and the second fused image; the first fused image can be blurred based on the first depth image to obtain a reference image; the reference image can be optimized, and the optimized reference image can be used as the first target image.

[0159] Among them, image optimization processing can optimize the reference image through beautification algorithms, face super-resolution (Face SR) technology, etc. Face SR technology can be used to reconstruct facial details.

[0160] Optionally, in a preview scenario, the first target image is displayed as a preview image; or, in a shooting scenario, the first target image is displayed as a captured image.

[0161] In this context, a preview scene can refer to a scenario where the captured image (preview image) is displayed in real-time on the screen of an electronic device, allowing the photographer to preview the shooting effect. In the preview scene, the user can adjust shooting parameters (such as focal length, brightness, etc.) based on the preview effect to ensure the shooting result meets expectations. A shooting scene can refer to a scenario where the electronic device receives a shooting command and generates an image (captured image) stored in the electronic device. Shooting commands can be triggered by the user touching the electronic device's camera button (such as volume buttons), clicking the camera control displayed on the screen, receiving a voice command to take a picture, or capturing a specific facial expression (such as a smile) or gesture of the target subject.

[0162] In the embodiments of this application, the first target image can be displayed as both a preview image and a captured image, thereby improving the flexibility of image display.

[0163] In the preview scenario, in addition to displaying the first target image as the preview image, a preview image can also be generated based on multiple sets of images captured by the first camera.

[0164] Optionally, if the contrast between light and dark in the shooting environment is greater than the preset contrast (such as in non-high dynamic range imaging (HDR) shooting scenarios), the fused display image corresponding to each group of images can be displayed on the screen. The fused display image is obtained by fusing the first short exposure frame and the first normal exposure frame in the corresponding image group. If the ambient brightness of the shooting environment is lower than the preset brightness, the first normal exposure frame in each group of images is displayed on the screen. If multiple groups of images have preset display content, the first short exposure frame in each group of images is displayed on the screen.

[0165] The system can display the merged image, the first normal exposure frame, and the first short exposure frame on the screen in the order of image acquisition time. The preset contrast and preset brightness values ​​can be determined through multiple experiments.

[0166] In this embodiment, the preview image is determined based on the shooting environment, which allows the user to adjust parameters based on the preview image that better matches the shooting environment, thereby making it easier for the electronic device to generate an image that meets the user's expectations.

[0167] Optionally, the fused display image / first normal exposure frame / first short exposure frame, which serves as a preview image, can be processed by the ISP and then displayed on the screen. ISP processing may include, but is not limited to, monocular depth calculation, bokeh effect, and image preprocessing.

[0168] In one implementation, when the first target image is displayed as a preview image, the electronic device can switch to the shooting scene in response to receiving the shooting command, and in the shooting scene, acquire multiple auxiliary frames through the first camera, and obtain a third fused image based on multiple first target short exposure frames and multiple auxiliary frames; obtain a second depth image based on the second fused image and the third fused image; and perform blurring processing on the third fused image based on the second depth image to obtain a second target image including the target object.

[0169] The third fused image can refer to an image obtained by fusing multiple short-exposure frames of the first target and multiple auxiliary frames. In this embodiment, the third fused image can be a YUV image. The second depth image can refer to a depth image obtained by performing binocular depth calculation based on the third fused image and the second fused image.

[0170] In HDR shooting scenarios, auxiliary frames can be images captured using a smart AE algorithm. A smart AE algorithm refers to an algorithm used for automatic exposure control, which adjusts image brightness to ensure ideal shooting results under different lighting conditions. Multiple auxiliary frames can include one or more auxiliary short exposure frames and one or more auxiliary normal exposure frames. The exposure duration of the auxiliary short exposure frame can be the same as the exposure duration of the most recently captured short exposure frame before receiving the shooting command, and the exposure duration of the auxiliary normal exposure frame can be the same as the exposure duration of the most recently captured normal exposure frame before receiving the shooting command. In addition to one or more auxiliary short exposure frames and one or more auxiliary normal exposure frames, multiple auxiliary frames can also include auxiliary long exposure frames and / or auxiliary extreme short frames. The exposure duration of the auxiliary long exposure frame is longer than the exposure duration of the first normal exposure frame in each image set, and the brightness of the auxiliary short frame is less than the brightness of the first normal exposure frame and the first short exposure frame in each image set.

[0171] In this embodiment of the application, multiple auxiliary frames are additionally acquired by the first camera in the shooting scene, and a second target image is generated based on the additionally acquired auxiliary frames. This can provide the real-time dynamics of the target object for the second target image, thereby reducing the difference between the second target image and the image observed by the user's naked eye.

[0172] To gain a clearer understanding of the image acquisition process of the first and second cameras, please refer to [link / reference]. Figure 7 In the preview scene, both the first and second cameras can acquire multiple sets of images using stagger mode. When relative motion between the target object and the electronic device is determined based on the multiple sets of images acquired by the first camera, the first camera alternately acquires a second short-exposure frame and a second normal-exposure frame, and the second camera alternately acquires a third short-exposure frame and a third normal-exposure frame. These acquired frames are then stored in a cache (such as a zero-shot lag (ZSL) cache). When switching from the preview scene to the shooting scene in response to a shooting command, multiple auxiliary frames can be acquired by the first camera. Based on the image in the cache at the time of the shooting command and the acquired auxiliary frames, the second target image can be obtained. These auxiliary frames include, but are not limited to, auxiliary short-exposure frames and auxiliary normal-exposure frames. In HDR scenes, these auxiliary frames also include auxiliary short frames and auxiliary long-exposure frames.

[0173] It should be noted that during the entire image acquisition process, the images acquired in the preview scene can be stored in the cache, and the images in the cache can be updated in real time, so that the latest images are always stored. For example, the cache can always store the latest four sets of images from the first camera and the second camera respectively.

[0174] Step S5090: If it is determined based on multiple sets of images that there is no relative motion between the target object and the electronic device, multiple second normal exposure frames are acquired by the first camera at a third frequency.

[0175] As one possible approach, if it is determined from multiple sets of images that there is no relative motion between the target object and the electronic device, multiple second normal exposure frames can be acquired by the first camera at a third frequency, and multiple fourth short exposure frames can be acquired at a first frequency.

[0176] The exposure duration of the fourth short exposure frame can be the same as that of the first short exposure frame in each group of images. This means that when there is no relative motion between the target object and the electronic device, the exposure duration of the short exposure frame captured by the first camera will not be adjusted.

[0177] Step S5100: While the first camera is capturing the second normal exposure frame, the second camera simultaneously captures multiple third normal exposure frames at a third frequency.

[0178] As one possible approach, while the first camera is capturing the second normal exposure frame, the second camera can capture multiple third normal exposure frames at a third frequency and multiple fifth short exposure frames at a first frequency.

[0179] The exposure duration of the fifth short exposure frame can be the same as that of the fourth short exposure frame. In other words, the adjustment of the exposure duration of the short exposure frames captured by the second camera is synchronized with the adjustment of the exposure duration of the short exposure frames captured by the first camera.

[0180] Step S5110: Based on multiple second normal exposure frames and multiple third normal exposure frames, obtain the fourth target image.

[0181] The fourth target image can refer to the image generated by the electronic device in a non-motion scene (i.e., there is no relative motion between the target object and the electronic device).

[0182] As one possible way to achieve this, such as Figure 6 As shown in S620, multiple second normal exposure frames can be fused to obtain a fourth fused image, and multiple third normal exposure frames can be fused to obtain a fifth fused image; as shown in S620. Figure 6As shown in S630, binocular depth calculation is performed based on the fourth fused image and the fifth fused image to obtain the fourth depth image; as... Figure 6 As shown in S640, the fourth target image is obtained based on the fourth depth image and the fourth fused image.

[0183] The fourth target image shares similarities with the first target image (the image generated in a moving scene) in that: in a preview scene, the fourth target image can be displayed as a preview image; or, in a shooting scene, the fourth target image can be displayed as a captured image. The image fusion and blurring methods used in the generation process of the first target image are also applicable to the fourth target image. Furthermore, when the fourth target image is used as a preview image, it can also respond to a shooting command by additionally acquiring multiple auxiliary frames through the first camera. The processing method for generating the captured image based on these multiple auxiliary frames, multiple second normal exposure frames, and multiple third normal exposure frames is the same as the method used to generate the second target image. In this embodiment, when there is no relative motion between the target object and the electronic device, the fourth target image is generated using the normal exposure frames acquired by the first and second cameras respectively. Since the exposure duration of the normal exposure frames is longer than that of the short exposure frames, the normal exposure frames can more clearly capture the details and textures of the still object, thereby improving the clarity of the fourth target image.

[0184] To better understand the solution of this embodiment, a preferred business process will be described below with reference to the accompanying drawings.

[0185] Please see Figure 8 When both the first and second cameras are synchronously acquiring images in stagger mode, motion detection and preset display content detection can be performed on multiple sets of images acquired by the first camera. Based on the detection results, the exposure time of the images acquired by the first camera is adjusted according to the aforementioned method, and the exposure time of the images acquired by the second camera is adjusted synchronously. The cache is updated in real time based on the images acquired by the first and second cameras. When a shooting command is received, the first camera can acquire multiple additional auxiliary frames. When the detection result of the preset display content detection is that preset display content exists in the image, image preprocessing can be performed on multiple auxiliary frames and images in the cache.

[0186] When the current shooting scene is determined to be a moving scene based on motion detection results, a fused image corresponding to the first camera (in this case, a third fused image) can be obtained based on multiple auxiliary frames and a second short-exposure frame from the first camera in the cache; and a fused image corresponding to the second camera (in this case, a second fused image) can be obtained based on a third short-exposure frame from the second camera in the cache; then, a target image (in this case, a second target image) can be further obtained by performing binocular depth calculation on the fused images corresponding to the first and second cameras. In a moving scene, the cache may include a second short-exposure frame and a second normal exposure frame from the first camera, as well as a third short-exposure frame and a third normal exposure frame from the second camera.

[0187] When the current shooting scene is determined to be a non-motion scene based on the motion detection results, a fused image corresponding to the first camera (the sixth fused image at this time) can be obtained based on multiple auxiliary frames and the second normal exposure frame from the first camera in the buffer; and a fused image corresponding to the second camera (the fifth fused image at this time) can be obtained based on the third normal exposure frame from the second camera in the ZSL buffer; then, a target image (the fifth target image at this time) can be further obtained by performing binocular depth calculation on the fused image corresponding to the first camera and the fused image corresponding to the second camera. In the non-motion scene, the buffer may include the fourth short exposure frame and the second normal exposure frame from the first camera, as well as the fifth short exposure frame and the third normal exposure frame from the second camera.

[0188] In this embodiment, when relative motion between the target object and the electronic device is determined based on multiple sets of images captured by the first camera, and the multiple sets of images captured by the first camera contain preset display content, multiple short-exposure frames are captured by reducing the exposure time. This allows the multiple short-exposure frames to more clearly present the movement of the target object. Simultaneously, the second camera can synchronously capture multiple short-exposure frames according to the acquisition frequency of the first camera. Next, image preprocessing is performed on the multiple short-exposure frames captured by each of the two cameras to eliminate the preset display content in the multiple short-exposure frames captured by each of the two cameras. This can improve the overall image clarity of all short-exposure frames. Then, the first target image is obtained through the first target short-exposure frame and the second target short-exposure frame, thereby improving the dynamic clarity of the target object in the first target image. Thus, in target object capture scenarios, especially in portrait capture scenarios, high-clarity images can also be obtained. Furthermore, when it is determined that there is no relative motion between the target object and the electronic device based on multiple sets of images captured by the first camera, a fourth target image is obtained based on multiple second normal exposure frames and multiple third normal exposure frames. This can reduce the complexity of the image processing process in non-motion scenes, thereby improving the generation speed of the fourth target image while ensuring the overall clarity of the fourth target image.

[0189] The following description, in conjunction with the accompanying drawings, illustrates an image acquisition method provided in this application embodiment, using the example of a first camera of an electronic device acquiring short exposure frames at a first frequency and normal exposure frames at a third frequency, while a second camera acquires short exposure frames only at a second frequency.

[0190] like Figure 9 As shown, an image acquisition method provided in this application embodiment may include S9010-S9120:

[0191] Step S9010: Acquire multiple sets of images through the first camera. All sets of images include the same target object. Each set of images includes at least one first short exposure frame and at least one first normal exposure frame.

[0192] For more information on acquiring multiple images using the first camera, please refer to the section above. Figure 3 The introduction of the S310 will not be repeated here.

[0193] Step S9020: If it is determined that there is relative motion between the target object and the electronic device based on multiple sets of images, and there is preset display content in the multiple sets of images, multiple second short exposure frames are captured by the first camera. The exposure time of the second short exposure frame is less than the exposure time of the first short exposure frame in each set of images. The preset display content will affect the clarity of the multiple sets of images.

[0194] For details regarding the first camera capturing multiple second short-exposure frames, please refer to the section above. Figure 3 The introduction of the S320 will not be repeated here.

[0195] Step S9030: The first camera acquires multiple second normal exposure frames at a third frequency. The multiple second normal exposure frames and multiple second short exposure frames are acquired alternately by the first camera. The exposure duration of the second normal exposure frame is longer than the exposure duration of the second short exposure frame.

[0196] For details regarding the first camera capturing multiple second normal exposure frames at a third frequency, please refer to the above text. Figure 5 The introduction of the S5030 will not be repeated here.

[0197] Step S9040: While the first camera is acquiring multiple second short exposure frames, the second camera is simultaneously acquiring multiple third short exposure frames at a second frequency.

[0198] The first frequency is N times the second frequency, where N is a positive integer.

[0199] As one possible approach, while the first camera captures multiple second short exposure frames, the second camera simultaneously captures multiple third short exposure frames at a second frequency.

[0200] Optionally, when N=1, the second frequency is the same as the first frequency, and the image acquisition process of the first and second cameras can be as follows: Figure 10 As shown, Figure 10 and Figure 7 The difference is that the second camera does not capture normal exposure frames.

[0201] Optionally, when N is an integer greater than 1, the second frequency can be less than the first frequency. In this case, the image acquisition process of the first and second cameras can be as follows: Figure 11 As shown, Figure 11 and Figure 7 The difference is that the second camera does not capture normal exposure frames, and the frequency at which the second camera captures short exposure frames is different from the frequency at which the first camera captures short exposure frames.

[0202] Step S9050: Perform image preprocessing on multiple second short exposure frames to obtain multiple first target short exposure frames. The image preprocessing is used to remove preset display content in the corresponding frames.

[0203] One possible approach is to preprocess all the second short exposure frames to obtain multiple first target short exposure frames.

[0204] As another approach, when N=1, image preprocessing can be performed on all second short exposure frames to obtain multiple first target short exposure frames; when N is an integer greater than 1, image preprocessing can be performed on multiple second short exposure frames that are acquired at the same time as the third short exposure frame to obtain multiple first target short exposure frames.

[0205] Step S9060: Perform image preprocessing on multiple third short exposure frames to obtain multiple second target short exposure frames.

[0206] For information on image preprocessing of multiple third short-exposure frames to obtain multiple second target short-exposure frames, please refer to the above text. Figure 3 The introduction of the S350 will not be repeated here.

[0207] Step S9070: Based on multiple short exposure frames of the first target, and combined with multiple short exposure frames of the second target, obtain the image of the first target.

[0208] For information on image preprocessing of multiple third short-exposure frames to obtain multiple second target short-exposure frames, please refer to the above text. Figure 3 The introduction of the S360 will not be repeated here.

[0209] Step S9080: If it is determined based on multiple sets of images that there is no relative motion between the target object and the electronic device, multiple fourth short exposure frames are acquired by the first camera at a first frequency and multiple second normal exposure frames are acquired at a third frequency. The multiple second normal exposure frames and multiple fourth short exposure frames are acquired alternately by the first camera. The exposure duration of the second normal exposure frame is longer than the exposure duration of the fourth short exposure frame, and the exposure duration of the fourth short exposure frame is the same as the exposure duration of the first short exposure frame.

[0210] As one possible approach, if it is determined from multiple sets of images that there is no relative motion between the target object and the electronic device, multiple fourth short exposure frames are acquired by the first camera at a first frequency, and multiple second normal exposure frames are acquired at a third frequency.

[0211] Step S9090: While the first camera is capturing the fourth short exposure frame, the second camera simultaneously captures multiple fifth short exposure frames at the second frequency.

[0212] As one possible approach, while the first camera is capturing the fourth short exposure frame, the second camera is simultaneously capturing multiple fifth short exposure frames at a second frequency.

[0213] Optionally, when N=1, the second frequency is the same as the first frequency; when N is an integer greater than 1, the second frequency can be less than the first frequency.

[0214] Step S9100: Perform image preprocessing on multiple fourth short exposure frames to obtain multiple third target short exposure frames.

[0215] As one possible approach, multiple fourth short exposure frames can be preprocessed in the manner described in step S340 to obtain multiple third target short exposure frames.

[0216] Step S9110: Perform image preprocessing on multiple fifth short exposure frames to obtain multiple fourth target short exposure frames.

[0217] As one possible approach, multiple fifth short exposure frames can be preprocessed according to the implementation of step S340 to obtain multiple third target short exposure frames.

[0218] Step S9120: Based on multiple short exposure frames of the third target, multiple short exposure frames of the fourth target, and multiple normal exposure frames of the second target, obtain the image of the third target.

[0219] As one possible way to achieve this, such as Figure 12 As shown in S2, image fusion is performed on multiple second normal exposure frames to obtain a fourth fused image; a seventh fused image can be obtained based on multiple third target short exposure frames; image fusion is performed on multiple fourth target short exposure frames to obtain an eighth fused image; as shown in S2. Figure 12 As shown in S3, a third depth image is obtained by performing binocular depth calculation based on the seventh and eighth fused images, as shown in [example image]. Figure 12 As shown in S4, the third target image is obtained based on the third depth image and the fourth fused image. (Further details omitted) Figure 12 For S1, S5-S7, please refer to the above text for details. Figure 6 Introduction to S610, S650 to S670.

[0220] Optionally, since there is a time difference between the fourth depth image and the fourth fused image, the fourth depth image can be mapped to the fourth fused image using optical flow to obtain a depth image (fifth depth image) that has no time difference with the fourth fused image. Furthermore, the third target image can be obtained based on the fifth depth image and the fourth fused image.

[0221] Optionally, when N=1, since the second frequency is the same as the first frequency, the number of short exposure frames of the third target acquired in the same time period is the same as the number of short exposure frames of the fourth target. Therefore, all the multiple short exposure frames of the third target can be image fused to obtain the seventh fused image.

[0222] Optionally, when N is an integer greater than 1, since the second frequency can be less than the first frequency, the number of third target short-exposure frames acquired within the same time period will be greater than the number of fourth target short-exposure frames. Therefore, to ensure that the obtained seventh and eighth fused images have no time deviation, the third target short-exposure frames acquired at the same time as the fourth target short-exposure frames can be selected from the multiple third target short-exposure frames for image fusion to obtain the seventh fused image. In this embodiment, acquiring only the fifth short-exposure frame through the second camera can reduce the power consumption of the electronic device. Furthermore, obtaining the third target image based on the fourth fused image and the third depth image can improve the spatial sense of the third target image.

[0223] To better understand the solution of this embodiment, a preferred service process when the second frequency is equal to the first frequency will be described below with reference to the accompanying drawings.

[0224] Please see Figure 13 When the first camera captures images in stagger mode and the second camera only captures short-exposure frames at the second frequency (first frequency), motion detection and preset display content detection can be performed on multiple sets of images captured by the first camera. Based on the detection results, the exposure time of the images captured by the first camera is adjusted according to the aforementioned method, and the exposure time of the images captured by the second camera is adjusted synchronously. The cache is updated in real time based on the images captured by the first and second cameras. When a shooting command is received, the first camera can additionally capture multiple auxiliary frames. When the detection result of the preset display content detection is that preset display content exists in the image, image preprocessing can be performed on multiple auxiliary frames and images in the cache.

[0225] When the current shooting scene is determined to be a moving scene based on the motion detection results, the fused image corresponding to the first camera (which is the third fused image at this time) can be obtained based on multiple auxiliary frames and the second short exposure frame from the first camera in the buffer; and the fused image corresponding to the second camera (which is the second fused image at this time) can be obtained based on the third short exposure frame from the second camera in the ZSL buffer; then, the target image (which is the second target image at this time) can be further obtained by performing binocular depth calculation on the fused image corresponding to the first camera and the fused image corresponding to the second camera.

[0226] In motion scenarios, the cache may include a second short exposure frame and a second normal exposure frame from the first camera, as well as a third short exposure frame from the second camera.

[0227] When the current shooting scene is determined to be a non-moving scene based on the motion detection results, the fused image corresponding to the first camera (the sixth fused image at this time) can be obtained based on multiple auxiliary frames and the second normal exposure frame from the first camera in the cache; the fused image corresponding to the first camera (the first fused image at this time) can be obtained based on the second short exposure frame from the first camera in the cache; and the fused image corresponding to the second camera (the seventh fused image at this time) can be obtained based on the fifth short exposure frame from the second camera in the cache; then, by performing binocular depth calculation on the first fused image and the seventh fused image, the depth image and the sixth fused image can be obtained, and the target image (the sixth target image at this time) can be obtained.

[0228] In non-motion scenarios, the cache may include a fourth short exposure frame and a second normal exposure frame from the first camera, as well as a fifth short exposure frame from the second camera.

[0229] In this embodiment, by using the second camera to capture the third or fifth short exposure frame only at the second frequency, the power consumption of the electronic device can be reduced.

[0230] To better understand the solution of this embodiment, a preferred service process when the first frequency is twice the second frequency will be described below with reference to the accompanying drawings.

[0231] Please see Figure 14 When the first camera captures images in stagger mode and the second camera captures short-exposure frames only at the second frequency (1 / 2 of the first frequency), motion detection and preset display content detection can be performed on multiple sets of images captured by the first camera. Based on the detection results, the exposure time of the images captured by the first camera is adjusted according to the aforementioned method, and the exposure time of the images captured by the second camera is adjusted synchronously. The cache is updated in real time based on the images captured by the first and second cameras. When a shooting command is received, the first camera can capture multiple additional auxiliary frames. When the detection result of the preset display content detection is that preset display content exists in the image, image preprocessing can be performed on multiple auxiliary frames and images in the cache.

[0232] When the current shooting scene is determined to be a moving scene based on the motion detection results, the fused image corresponding to the first camera (the ninth fused image at this time) can be obtained based on the second short exposure frame that is the same as the third short exposure frame in the multiple auxiliary frames and the multiple second short exposure frames stored in the cache. And, based on the third short exposure frame from the second camera in the cache, the fused image corresponding to the second camera (the tenth fused image at this time) can be obtained. Then, the target image (the seventh target image at this time) can be further obtained by performing binocular depth calculation on the fused image corresponding to the first camera and the fused image corresponding to the second camera.

[0233] In motion scenarios, the cache may include a second short exposure frame and a second normal exposure frame from the first camera, as well as a third short exposure frame from the second camera.

[0234] When the current shooting scene is determined to be a non-moving scene based on the motion detection results, the fused image corresponding to the first camera (the eleventh fused image at this time) can be obtained based on multiple auxiliary frames and the second normal exposure frame from multiple stored second normal exposure frames that are acquired at the same time as the third normal exposure frame; and the fused image corresponding to the first camera (the first fused image at this time) can be obtained based on the second short exposure frame from the first camera in the cache; and the fused image corresponding to the second camera (the seventh fused image at this time) can be obtained based on the fifth short exposure frame from the second camera in the cache; then, by performing binocular depth calculation on the eleventh fused image and the seventh fused image, a depth image and a sixth fused image can be obtained based on the binocular depth calculation, and the target image (the eighth target image at this time) can be obtained.

[0235] In non-motion scenarios, the cache may include a fourth short exposure frame and a second normal exposure frame from the first camera, as well as a fifth short exposure frame from the second camera.

[0236] In this embodiment, by using the second camera to capture the third or fifth short exposure frame only at the second frequency, the power consumption of the electronic device can be reduced.

[0237] To better understand the solutions in the embodiments of this application, the solutions involved in the embodiments of this application will be described below in conjunction with application scenarios.

[0238] Taking a mobile phone as an example, when a user needs to capture / flash a portrait, they can launch an application with a camera function, causing the corresponding content to appear on the screen. The user can then tap the "Portrait" control on the screen to enter portrait mode; alternatively, the device can automatically switch to portrait mode when a person is present in the preview image and display it on the screen. Figure 15 The interface shown in the left image. You can then click... Figure 15 The "Effects" control in the left image will display on the screen. Figure 15 The portrait mode shown in the right figure can achieve blurring effects such as snapshot / flash capture (i.e., the image acquisition method proposed in this application), circles, hearts, etc. When the user clicks the "snap capture / flash capture" control, the electronic device can work according to the image acquisition method proposed in this application.

[0239] Alternatively, the portrait capture mode can be entered in the following ways: When a portrait needs to be captured / snap, the user can launch an application with shooting capabilities, causing the corresponding content of the application to appear on the electronic device's screen. At this time, the user can click the "Snap" control on the screen to enter the snap / capture mode; or, the electronic device can automatically switch to snap / capture mode when a moving object is present in the preview image and display it on the screen. Figure 16 The interface shown in the left image. You can then click... Figure 16 The "Effects" control in the left image will display on the screen. Figure 16 The blurring effects that can be achieved in the flash / capture mode shown in the right figure include portraits (i.e., the image acquisition method proposed in this application), circles, hearts, etc. When the user clicks the "portrait" control, the electronic device can work according to the image acquisition method proposed in this application.

[0240] As described above, in the image acquisition method proposed in this application, the first camera can alternately capture normal exposure frames and short exposure frames, and the second camera can capture images in the following ways: First, it alternately captures normal exposure frames and short exposure frames, and remains synchronized with the first camera; that is, when the first camera captures one normal exposure frame, the second camera also captures one normal exposure frame; when the first camera captures one short exposure frame, the second camera also captures one short exposure frame. Second, it only captures short exposure frames, and remains synchronized with the first camera; that is, when the first camera captures one short exposure frame, the second camera also captures one short exposure frame. Third, it only captures short exposure frames, and is not synchronized with the first camera; for example, when the first camera captures two short exposure frames, the second camera will only capture one short exposure frame.

[0241] In this scenario, when the second camera acquires images using the first method, the number of images acquired in the same amount of time is greater than the other two methods. This provides more detail of the target object for the subsequent image generation, thus improving the quality of the target image. When the second camera acquires images using the third method, the number of images acquired in the same amount of time is the smallest among the three methods, thereby reducing the power consumption of the electronic device. When the second camera acquires images using the second method, the number of images acquired is in the middle among the three methods, resulting in a better improvement in target image quality than the third method and a better reduction in power consumption than the first method.

[0242] When an electronic device enters portrait capture mode, there are multiple ways to determine how the second camera captures images.

[0243] As a first implementation method, the image acquisition method of the second camera can be determined based on the current remaining battery power of the electronic device.

[0244] Optionally, when the remaining battery power is greater than a first preset value, the second camera is determined to capture images in the first manner; when the remaining battery power is greater than a second preset value and less than a first preset value, the second camera is determined to capture images in the second manner; when the remaining battery power is less than a second preset value, the second camera is determined to capture images in the third manner.

[0245] It should be noted that when the remaining battery power is equal to the first preset value, the second camera is determined to capture images in either the first or second manner; when the remaining battery power is equal to the second preset value, the second camera is determined to capture images in either the second or third manner. This application does not impose any limitations on this.

[0246] Optionally, the first and second preset values ​​can be default values ​​in the electronic device.

[0247] Optionally, the first and second preset values ​​can be values ​​set according to the personal usage habits of the user corresponding to the electronic device. Specifically, the first and second preset values ​​for personal usage habits such as liking to watch short videos and play games can be lower than the first and second preset values ​​for personal usage habits such as liking to browse text and not frequently using the phone.

[0248] As a second implementation method, the image acquisition method of the second camera can be determined based on the current remaining battery level of the electronic device and the user's current location information.

[0249] Optionally, when it is determined, based on the user's current location information, that the distance between the user and the preset location is less than a preset distance, if the remaining battery power is greater than or equal to a third preset value, then the second camera is determined to acquire images in the first manner; if the remaining battery power is less than the third preset value, then the second camera is determined to acquire images in the second manner. When it is determined, based on the user's current location information, that the distance between the user and the preset location is greater than or equal to the preset distance, the image acquisition method of the second camera can be determined based on the previous implementation method.

[0250] The preset location can refer to a place with charging equipment (such as home, office, shopping mall, restaurant, etc.). The preset distance can be a distance that the user can reach by walking in a specified time, such as a 10-minute walk. The third preset value can be greater than the second preset value and less than the first preset value.

[0251] As a third implementation method, the image acquisition method of the second camera can be determined based on the current remaining battery level of the electronic device and the current background application.

[0252] Optionally, when it is determined, based on the current background applications, that the electronic device is performing a download task and / or the number of background applications exceeds a preset number, if the remaining battery power is greater than a fourth preset value, then the second camera is determined to capture images in a second manner; if the remaining battery power is less than or equal to the fourth preset value, then the second camera is determined to capture images in a third manner. When it is determined, based on the current background applications, that the electronic device is not performing a download task and the number of background applications is less than or equal to a preset number, the image capture method of the second camera can be determined based on the first implementation method.

[0253] The preset quantity can be determined based on multiple experiments. The fourth preset value can be greater than the second preset value but less than the first preset value.

[0254] In this application embodiment, the first implementation method can simply and efficiently determine how the second camera acquires images; the second implementation method, by considering the user's current location information, can reduce the requirement for the remaining battery power of the electronic device when the user is close to the preset location, thereby maximizing the quality of the target image; the third implementation method, by considering the current background applications of the electronic device, can avoid errors in the background application operation due to low battery power during shooting. In practical applications, the electronic device can perform intelligent analysis, and the specific implementation method can be determined based on the results of the intelligent analysis.

[0255] It should be understood that the various solutions in the embodiments of this application can be used in a reasonable combination, and the explanations or descriptions of the various terms appearing in the embodiments can be referenced or explained to each other in the various embodiments, without limitation.

[0256] It should also be understood that, in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0257] The methods provided in the embodiments of this application above are described from the perspective of an electronic device as the executing entity. To implement the functions of the methods provided in the embodiments of this application above, the electronic device may include hardware structures and / or software modules, implementing the above functions in the form of hardware structures, software modules, or a combination of hardware structures and software modules. Whether a particular function is executed in the form of hardware structures, software modules, or a combination of hardware structures and software modules depends on the specific application and design constraints of the technical solution.

[0258] This application embodiment can divide electronic devices into functional modules. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0259] It should be understood that the various modules in an electronic device can be implemented in software and / or hardware, without specific limitations. In other words, input devices are presented in the form of functional modules. Here, "module" can refer to application-specific integrated circuits (ASICs), circuits, processors and memories that execute one or more software or firmware programs, integrated logic circuits, and / or other devices that can provide the above functions.

[0260] In the above embodiments, the terms "when..." or "after..." can be interpreted, depending on the context, as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, the phrases "when..." or "if (the stated condition or event) is detected" can be interpreted, depending on the context, as meaning "if...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)". Furthermore, in the above embodiments, relational terms such as "first" and "second" are used to distinguish one entity from another, without limiting any actual relationship or order between these entities.

[0261] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0262] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this embodiment are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)). Where there is no conflict, the solutions in the above embodiments can be combined.

[0263] The steps of the methods or algorithms described in conjunction with the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, portable hard disk, CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in an electronic device. Of course, the processor and storage medium can also exist as discrete components in an electronic device.

[0264] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

Claims

1. An image acquisition method, characterized in that, Applied to an electronic device, the electronic device including a first camera, the method includes: Multiple sets of images are acquired through the first camera. All sets of images include the same target object. Each set of images includes at least one first short exposure frame and at least one first normal exposure frame. If it is determined based on the multiple sets of images that there is relative motion between the target object and the electronic device, and there is preset display content in the multiple sets of images, multiple second short exposure frames are captured by the first camera. The exposure time of the second short exposure frame is less than the exposure time of the first short exposure frame in each set of images. The preset display content will affect the clarity of the multiple sets of images. Image preprocessing is performed on the plurality of second short exposure frames to obtain a plurality of first target short exposure frames, wherein the image preprocessing is used to remove the preset display content in the corresponding frames; Based on the plurality of first target short exposure frames, a first target image including the target object is obtained.

2. The method according to claim 1, characterized in that, The first camera acquires the plurality of second short-exposure frames at a first frequency, the electronic device further includes a second camera, and the method further includes: While the first camera is capturing the plurality of second short exposure frames, the second camera is simultaneously capturing a plurality of third short exposure frames at the first frequency. The image preprocessing is performed on the plurality of third short exposure frames to obtain a plurality of second target short exposure frames; The step of obtaining a first target image including the target object based on the plurality of first target short exposure frames includes: The first target image is obtained by combining the plurality of first target short exposure frames with the plurality of second target short exposure frames.

3. The method according to claim 1 or 2, characterized in that, The preset display content includes one or more of the following: bars with varying brightness, screen objects, or black stripes.

4. The method according to claim 2 or 3, characterized in that, The method further includes: While the first camera is capturing the plurality of second short exposure frames, the second camera is simultaneously capturing a plurality of third short exposure frames at a second frequency; the first frequency is N times the second frequency, where N is a positive integer.

5. The method according to any one of claims 2-4, characterized in that, The process of obtaining the first target image based on the plurality of first target short exposure frames and the plurality of second target short exposure frames includes: Based on the multiple short-exposure frames of the first target, a first fused image is obtained; Based on the multiple short-exposure frames of the second target, a second fused image is obtained; Based on the first fused image and the second fused image, a first depth image is obtained; The first fused image is blurred based on the first depth image to obtain the first target image.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: In the preview scene, the first target image is displayed as the preview image; or, In the shooting scenario, the first target image is sent as the captured image for display.

7. The method according to claim 5, characterized in that, The method further includes: In the shooting scenario, multiple auxiliary frames are captured by the first camera. The multiple auxiliary frames include: auxiliary long exposure frames and / or auxiliary short frames. The exposure duration of the auxiliary long exposure frames is greater than the exposure duration of the first normal exposure frames in each group of images. The brightness of the auxiliary short frames is less than the brightness of the first normal exposure frames and the first short exposure frames in each group of images. Based on the multiple first target short exposure frames and the multiple auxiliary frames, a third fused image is obtained; A second depth image is obtained based on the second fused image and the third fused image; The third fused image is blurred based on the second depth image to obtain a second target image including the target object.

8. The method according to any one of claims 5-7, characterized in that, The step of blurring the first fused image based on the first depth image includes: Based on the first depth image, the first image segmentation process is performed on the first fused image to segment the target object and the background in the first fused image. The background in the first fused image is blurred.

9. The method according to claim 8, characterized in that, The step of blurring the first fused image based on the first depth image further includes: Based on the first depth image, a second image segmentation process is performed on the target object in the first fused image. The second image segmentation process is used to segment out the local region of the target object in the first segmented image. The local region is processed according to the optimization algorithm corresponding to the local region.

10. The method according to any one of claims 1-5 or 7-9, characterized in that, The electronic device also includes a display screen, and in a preview scenario, the method further includes: If the contrast between light and dark in the shooting environment is greater than the preset contrast, the fused display image corresponding to each group of images is displayed on the display screen. The fused display image is obtained by fusing the first short exposure frame and the first normal exposure frame in the corresponding image group. If the ambient brightness of the shooting environment is lower than the preset brightness, the first normal exposure frame of each group of images will be displayed on the display screen; If the preset display content exists in the multiple sets of images, the first short exposure frame of each set of images will be displayed on the display screen.

11. The method according to any one of claims 1-10, characterized in that, The method further includes: The reference exposure duration of the second short exposure frame is determined based on the movement speed of the target object; If the reference exposure duration is greater than or equal to the preset minimum exposure duration, the reference exposure duration is used as the exposure duration of the second short exposure frame. The preset minimum exposure duration is determined based on the flicker frequency of the artificial light source, and the artificial light source causes the presence of bands with varying brightness in the multiple sets of images. If the reference exposure duration is less than the preset minimum exposure duration, the preset minimum exposure duration shall be used as the exposure duration of the second short exposure frame.

12. The method according to claim 1, characterized in that, The electronic device further includes a second camera, and the method further includes: If it is determined based on the multiple sets of images that there is no relative motion between the target object and the electronic device, the first camera acquires multiple fourth short exposure frames at the first frequency and multiple second normal exposure frames at the third frequency. The multiple second normal exposure frames and the multiple fourth short exposure frames are acquired alternately by the first camera. The exposure duration of the second normal exposure frame is longer than the exposure duration of the fourth short exposure frame, and the exposure duration of the fourth short exposure frame is the same as the exposure duration of the first short exposure frame. While the first camera is capturing the fourth short exposure frame, the second camera is simultaneously capturing multiple fifth short exposure frames at the second frequency. The image preprocessing is performed on the plurality of fourth short exposure frames to obtain a plurality of third target short exposure frames; The image preprocessing is performed on the plurality of fifth short exposure frames to obtain a plurality of fourth target short exposure frames; A third target image is obtained based on the plurality of third target short exposure frames, the plurality of fourth target short exposure frames, and the plurality of second normal exposure frames.

13. The method according to claim 1, characterized in that, The electronic device further includes a second camera, and the method further includes: If it is determined based on the multiple sets of images that there is no relative motion between the target object and the electronic device, multiple second normal exposure frames are acquired by the first camera at a third frequency; While the first camera is capturing the second normal exposure frame, multiple third normal exposure frames are simultaneously captured by the second camera at the third frequency. A fourth target image is obtained based on the plurality of second normal exposure frames and the plurality of third normal exposure frames.

14. An image acquisition device, characterized in that, The image acquisition device includes: One or more cameras are used to capture images; Memory is used to store computer program instructions; A processor for executing the computer program instructions to support the image acquisition device in implementing the method as described in any one of claims 1-13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processing circuit, implement the method as described in any one of claims 1-13.

16. A computer program product containing instructions, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-13.

Images