Image capture method, terminal, storage medium and program product

Abstract

The embodiment of the present application provides an image capture method, terminal, storage medium and program product, which relates to the field of image processing technology. The above method includes: in response to a user clicking a capture button, obtaining an image captured by an image sensor; determining the scene type of the shooting scene based on the motion information of the object in the image captured by the image sensor; if the scene type is a non-motion scene, selecting a reference image from the image captured by the image sensor based on the first moment and the static quality characterization value of the image captured by the image sensor, wherein the first moment is: the generation moment of the image capture instruction triggered by the user clicking the capture button; if the scene type is a motion scene, selecting a reference image from the image captured by the image sensor based on the motion information of the object in the image captured by the image sensor; generating a captured image based on the reference image. The image capture scheme provided by the embodiment of the present application can capture a better image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image capture method, a terminal, a storage medium and a program product. Background Art

[0002] Mobile phones, tablet computers and other terminals are generally equipped with image sensors and thus have a camera function. People often take photos to relax and record their lives, and have a higher pursuit of photo-taking experience, hoping to capture better images when taking photos.

[0003] However, many people have limited photography skills and find it difficult to control the terminal to capture better images, and the captured images are often unsatisfactory.

[0004] In view of the above situation, it is necessary to provide an image capture solution to capture better images. Summary of the invention

[0005] In view of this, the present application provides an image capture method, a terminal, a storage medium and a program product to capture a better image.

[0006] In a first aspect, an embodiment of the present application provides an image capture method, the method comprising:

[0007] In response to the user clicking a capture button, obtaining an image captured by the image sensor;

[0008] Determining a scene type of a shooting scene based on motion information of an object in an image captured by the image sensor;

[0009] If the scene type is a non-motion scene, a reference image is selected from the images captured by the image sensor based on a first moment and a static quality characterization value of the image captured by the image sensor, wherein the first moment is: the moment when the image sensor receives an image capture instruction triggered by the user clicking a capture button;

[0010] If the scene type is a motion scene, selecting a reference image from the images captured by the image sensor based on motion information of the object in the image captured by the image sensor;

[0011] Based on the reference image, a captured image is generated.

[0012] As can be seen from the above, in the solution provided by this embodiment, before selecting a reference image from the image captured by the image sensor, the scene type of the shooting scene is first distinguished based on the motion information of the object in the image, and the scene type is determined as a motion scene or a non-motion scene, so that the reference image can be selected from the image captured by the image sensor in different ways based on the determined scene type. In this way, the reference image can be selected in a targeted manner according to the characteristics of the two scene types of motion scene and non-motion scene, so that the selected reference image is more reasonable and targeted, so that the subsequent captured image generated based on the selected reference image is more reasonable and targeted. In summary, it can be seen that the solution provided by the embodiment of the present application can capture better images.

[0013] In one embodiment of the present application, the step of selecting a reference image from images captured by the image sensor based on the first moment and the static quality characterization value of the image captured by the image sensor includes:

[0014] Based on a first moment, selecting a first image from images acquired by the image sensor;

[0015] Obtaining a static quality representation value of the first image;

[0016] A reference image is selected from the first images according to the obtained static quality representation value.

[0017] In the solution provided by this embodiment, when selecting a reference image, firstly, based on the first moment, a first image is selected from the images captured by the image sensor, and then, based on the static quality characterization value of the first image, a reference image is selected from the first images. Since the first moment is the moment when the image capture instruction is received, based on the first moment, a first image associated with the moment when the image capture instruction is received can be selected from the images captured by the image sensor. The first image associated with the moment when the image capture instruction is received is generally similar to the picture that the user actually sees when clicking to take a photo. Therefore, by selecting a reference image from the first image, the subsequent captured image generated based on the reference image can be closer to the picture that the user sees when clicking the capture button, thereby obtaining a better captured image.

[0018] In one embodiment of the present application, the selecting the first image from the images acquired by the image sensor based on the first moment includes:

[0019] The first image is selected in at least one of the following ways:

[0020] Determine a first image captured before the first moment among the images captured by the image sensor;

[0021] A first image acquired after the first moment among the images acquired by the image sensor is determined.

[0022] The first moment is the moment when the image sensor receives the image acquisition instruction. Due to the existence of the instruction delay, the first moment is located after the second moment. In this way, the image acquired before the first moment is an image acquired close to the second moment. Determining the above image as the first image is conducive to the subsequent acquisition of a reference image acquired close to the second moment based on the first image. Since the acquisition time of the reference image is close to the second moment, the reference image is similar to the picture seen by the user when clicking the capture button. Therefore, based on the reference image, a capture image similar to the picture seen by the user when clicking the capture button can be generated. For the user, the above capture image is a better image, which improves the user experience.

[0023] In addition, the first moment is the moment when the image acquisition instruction is received. When a user uses a terminal to take a photo, he or she often holds the terminal steadily for a period of time after clicking the photo button. Therefore, the image acquired after the first moment is often an image acquired when the terminal is in a stable posture, and its static image quality is often better. Therefore, the image acquired after the first moment is determined as the first image, and an image with higher static image quality can be obtained, so that the image quality of the reference image subsequently selected from the first image is higher, which improves the image quality of the captured image generated based on the reference image. For the user, the above-mentioned captured image is a better image, which improves the user experience.

[0024] In one embodiment of the present application, selecting a reference image from the first image according to the obtained static quality representation value includes:

[0025] Determine the second moment when the user clicks the snapshot button;

[0026] Based on the second moment and the obtained static quality characterization value, a reference image is selected from the first images.

[0027] Since the second moment is the moment when the user actually clicks the snapshot button, based on the second moment, a reference image close to the second moment can be selected from the first image, that is, an image that is relatively similar to the picture seen at the moment when the user actually clicks the snapshot button is selected, which is conducive to generating a snapshot image based on the reference image that is relatively similar to the picture seen at the moment when the user actually clicks the snapshot button. The above image is closer to the picture seen when the user clicks the snapshot button. In this way, for the user, the above snapshot image is a better image, which improves the user experience.

[0028] In one embodiment of the present application, the selecting a reference image from the first image based on the second moment and the obtained static quality characterization value includes:

[0029] Based on the obtained static quality characterization value, a reference image whose acquisition time is closest to the second time and whose image quality meets the standard is determined from the first image.

[0030] In this way, under the premise that the image quality meets the standard, the reference image that is closest to the acquisition moment and the second moment can be determined. Since the second moment is the moment when the user actually clicks the snapshot button, the reference image that is most similar to the picture seen by the user when he actually clicks the snapshot button can be determined as much as possible. The above image is close to the picture seen by the user when he clicks the snapshot button, that is, "what you see is what you get". In this way, for the user, the above snapshot image is a better image, which improves the user experience.

[0031] In one embodiment of the present application, the static quality characterization value includes at least one of the following information:

[0032] Automatic exposure AE convergence of the image captured by the image sensor;

[0033] Auto-focus AF convergence degree of the image captured by the image sensor;

[0034] Automatic white balance (AWB) convergence of the image captured by the image sensor;

[0035] The clarity of the image captured by the image sensor;

[0036] The contrast of the image captured by the image sensor;

[0037] Color saturation of the image captured by the image sensor;

[0038] Color uniformity of the image captured by the image sensor.

[0039] It can be seen that the static quality characterization value can include information corresponding to various types of static evaluation indicators, so as to more comprehensively and accurately reflect the quality information of the above-mentioned image in the static dimension or spatial dimension.

[0040] In one embodiment of the present application, the action information includes at least one of the following information:

[0041] the motion amplitude of the object in the image captured by the image sensor;

[0042] The difference between the motion amplitude of the object in the image captured by the image sensor and the motion amplitude of the object in the adjacent image.

[0043] The above motion amplitude reflects the motion of an object in a single image in the time dimension, and can more accurately measure the dynamic changes of the detection image in the time dimension. Therefore, based on the above motion amplitude, the scene type of the shooting scene can be more accurately determined.

[0044] In addition, the above differences reflect the differences between the motion amplitudes of objects in multiple images that are continuous in time, and reflect the changes in the motion of objects in multiple images in the time dimension. They can more accurately measure the dynamic changes of images in the time dimension, and thus, based on the above differences, the scene type of the shooting scene can be more accurately determined.

[0045] In one embodiment of the present application, selecting a reference image from the image captured by the image sensor based on the motion information of the object in the image captured by the image sensor includes:

[0046] A reference image is selected from the images captured by the image sensor based on the motion information of the object in the image captured by the image sensor and the static quality representation value of the image.

[0047] In this way, the images captured by the image sensor can be evaluated more comprehensively from both the motion dimension and the static dimension, which is conducive to obtaining a better reference image, and thus is conducive to generating a better captured image based on the reference image.

[0048] In one embodiment of the present application, generating a captured image based on the reference image includes:

[0049] Determine a second difference between the motion amplitude of the object in the reference image and the motion amplitude of the object in the second image, wherein the second image is: an image adjacent to the reference image among the images acquired by the image sensor;

[0050] If the second difference is greater than the difference threshold, generating a captured image based on the reference image;

[0051] Otherwise, a captured image is generated based on the reference image and the second image.

[0052] When the second difference is greater than the difference threshold, it means that there is a large difference between the reference image and the second adjacent image. At this time, directly generating the snapshot image based on the reference image can avoid the influence of the second image on the generation of the snapshot image; when the second difference is not greater than the difference threshold, it means that there is a small difference between the reference image and the second adjacent image, that is, the reference image is closer to the second image. At this time, generating the snapshot image based on the reference image and the second image takes the image contents of the reference image and the second image into comprehensive consideration, which is conducive to generating a better snapshot image.

[0053] In a second aspect, an embodiment of the present application further provides a terminal, including:

[0054] one or more processors, image sensors, and memory;

[0055] The memory is coupled to the one or more processors, and the memory is used to store computer program codes, where the computer program codes include computer instructions. The one or more processors call the computer instructions to enable the terminal to execute any one of the methods described in the first aspect.

[0056] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, comprising a computer program, which, when executed on a terminal, enables the terminal to execute any of the methods described in the first aspect.

[0057] In a fourth aspect, an embodiment of the present application further provides a computer program product, wherein the computer program product comprises executable instructions, and when the executable instructions are executed on a terminal, the terminal executes the method described in any one of the above-mentioned first aspects.

[0058] In the fifth aspect, an embodiment of the present application also provides a chip system, which is applied to a terminal, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the terminal inputs data into the chip system, and executes the method described in any one of the first aspects above to process the data and output the processing results.

[0059] The beneficial effects of the solutions provided by the embodiments in the second, third, fourth and fifth aspects can refer to the beneficial effects of the solutions provided by the embodiments in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0061] Figure 1 A schematic diagram of the structure of a terminal provided in an embodiment of the present application;

[0062] Figure 2 A software structure block diagram of a terminal provided in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of the process of the first image capture method provided in an embodiment of the present application;

[0064] Figure 4a A schematic diagram of the first image acquisition timing provided in an embodiment of the present application;

[0065] Figure 4b A schematic diagram of a second image acquisition time provided in an embodiment of the present application;

[0066] Figure 4c A schematic diagram of a third image acquisition time provided in an embodiment of the present application;

[0067] Figure 5 A schematic diagram of a flow chart of a second image capture method provided in an embodiment of the present application;

[0068] Figure 6 A schematic diagram of a first image capture effect provided in an embodiment of the present application;

[0069] Figure 7 A schematic diagram of a second image capture effect provided in an embodiment of the present application;

[0070] Figure 8 A schematic diagram of the structure of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0072] In order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first" and "second" are used to distinguish the same items or similar items with substantially the same functions and effects. For example, the first instruction and the second instruction are to distinguish different user instructions, and their order is not limited. Those skilled in the art can understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not necessarily limit them to be different.

[0073] It should be noted that, in this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in this application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0074] The solution provided in the embodiments of the present application can be applied to terminals equipped with image sensors, such as mobile phones, tablet computers, personal digital assistants (PDAs), smart watches, wearable electronic devices, augmented reality (AR) devices, virtual reality (VR) devices, robots, smart glasses, etc.

[0075] For example, Figure 1The schematic diagram of the structure of the terminal 100 is shown. The terminal 100 may include a processor 110, a display screen 120, an internal memory 130, a subscriber identification module (Subscriber Identification Module, SIM) card interface 140, a universal serial bus (Universal Serial Bus, USB) interface 150, a charging management module 160, a battery management module 161, a battery 162, a sensor module 170, a mobile communication module 180, a wireless communication module 190, an antenna 1 and an antenna 2, etc. The sensor module 170 may include a pressure sensor 170A, a fingerprint sensor 170B, a touch sensor 170C, an ambient light sensor 170D, an image sensor 170E, etc.

[0076] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the terminal 100. In other embodiments of the present application, the terminal 100 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0077] The processor 110 may include one or more processing units, for example: the processor 110 may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processor (NPU), etc. Among them, different processing units may be independent components or integrated into one or more processors. In some embodiments, the terminal 100 may also include one or more processors 110. Among them, the controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. In some other embodiments, a memory may also be set in the processor 110 for storing instructions and data. Exemplarily, the memory in the processor 110 may be a high-speed cache memory. The memory may store instructions or data that the processor 110 has just used or circulated. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the terminal 100 in processing data or executing instructions.

[0078] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an Inter-Integrated Circuit (I2C) interface, an Inter-Integrated Circuit Sound (I2S) interface, a Pulse Code Modulation (PCM) interface, a Universal Asynchronous Receiver / Transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a General-Purpose Input / Output (GPIO) interface, a SIM card interface and / or a USB interface, etc. Among them, the USB interface 150 is an interface that complies with the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 150 may be used to connect a charger to charge the terminal 100, and may also be used to transmit data between the terminal 100 and a peripheral device. The USB interface 150 may also be used to connect headphones to play audio through the headphones.

[0079] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is for illustrative purposes only and does not constitute a structural limitation on the terminal 100. In other embodiments of the present application, the terminal 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0080] The wireless communication function of the terminal 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 180, the wireless communication module 190, the modem processor and the baseband processor.

[0081] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0082] The terminal 100 implements the display function through a GPU, a display screen 120, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 120 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0083] The display screen 120 is used to display images, videos, etc. The display screen 120 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal 100 may include one or more display screens 120.

[0084] In some embodiments of the present application, when the display panel is made of materials such as OLED, AMOLED, FLED, etc., the above Figure 1 The display screen 120 in the embodiment can be bent. Here, the display screen 120 can be bent means that the display screen can be bent to any angle at any position and can be kept at the angle, for example, the display screen 120 can be folded in half from the middle to the left or right. It can also be folded in half from the middle to the top or bottom.

[0085] The display screen 120 of the terminal 100 may be a flexible screen. Currently, flexible screens have attracted much attention for their unique characteristics and great potential. Compared with traditional screens, flexible screens are flexible and bendable, which can provide users with a new interaction method based on the bendable characteristics, and can meet more user demands for the terminal. For a terminal equipped with a foldable display screen, the foldable display screen on the terminal can be switched between a small screen in a folded form and a large screen in an unfolded form at any time. Therefore, users are using the split-screen function on terminals equipped with foldable displays more and more frequently.

[0086] The terminal 100 can implement a shooting function through an ISP, an image sensor 170E, a video codec, a GPU, a display screen 120, and an application processor.

[0087] The ISP is used to process the data fed back by the image sensor 170E. For example, when shooting, the shutter is opened, and light is transmitted to the image sensor 170E through the lens. The light signal is converted into an electrical signal, and the image sensor 170E transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can perform algorithmic optimization on the noise, brightness and color of the image. The ISP can also optimize the exposure and color temperature of the shooting scene and other parameters.

[0088] The image sensor 170E is used to capture photos or videos. The object generates an optical image through the lens and projects it onto the image sensor 170E. The image sensor 170E may include a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The image sensor 170E converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard red green blue (RGB), YUV or other format. In some embodiments, the terminal 100 may include 1 or N image sensors 170E, where N is a positive integer greater than 1.

[0089] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0090] Video codecs are used to compress or decompress digital videos. Terminal 100 may support one or more video codecs. Thus, terminal 100 may play or record videos in multiple coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0091] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of the terminal 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0092] The internal memory 130 can be used to store one or more computer programs, which include instructions. The processor 110 can execute the above instructions stored in the internal memory 130, so that the terminal 100 executes the video cutout method provided in some embodiments of the present application, as well as various applications and data processing. The internal memory 130 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system; the program storage area can also store one or more applications (such as a gallery, contacts, etc.). The data storage area can store data (such as photos, contacts, etc.) created during the use of the terminal 100. In addition, the internal memory 130 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage components, flash memory components, universal flash memory (Universal Flash Storage, UFS), etc. In some embodiments, the processor 110 can execute the video cutout method provided in the embodiment of the present application, as well as other applications and data processing by running instructions stored in the internal memory 130 and / or instructions stored in a memory provided in the processor 110.

[0093] The internal memory 130 can be used to store the relevant programs of the video cutout method provided in the embodiment of the present application, and the processor 110 can be used to call the relevant programs of the video cutout method stored in the internal memory 130 when displaying information, and execute the video cutout method of the embodiment of the present application.

[0094] The sensor module 170 may include a pressure sensor 170A, a fingerprint sensor 170B, a touch sensor 170C, an ambient light sensor 170D, and the like.

[0095] The pressure sensor 170A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 170A can be set on the display screen 120. There are many types of pressure sensors 170A, for example, they can be resistive pressure sensors, inductive pressure sensors or capacitive pressure sensors. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When the force acts on the pressure sensor 170A, the capacitance between the electrodes changes, and the terminal 100 determines the intensity of the pressure according to the change in capacitance. When the touch operation acts on the display screen 120, the terminal 100 detects the touch operation according to the pressure sensor 170A. The terminal 100 can also calculate the position of the touch according to the detection signal of the pressure sensor 170A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on a short message application icon, an instruction to view the short message is executed; when a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on a short message application icon, an instruction to create a new short message is executed.

[0096] The fingerprint sensor 170B is used to collect fingerprints. The terminal 100 can use the collected fingerprint characteristics to realize functions such as unlocking, accessing application locks, taking pictures, and answering calls.

[0097] The touch sensor 170C is also called a touch control device. The touch sensor 170C can be arranged on the display screen 120, and the touch sensor 170C and the display screen 120 form a touch screen, which is also called a touch control screen. The touch sensor 170C is used to detect a touch operation acting on or near it. The touch sensor 170C can pass the detected touch operation to the application processor to determine the type of touch event. A visual output related to the touch operation can be provided through the display screen 120. In other embodiments, the touch sensor 170C can also be arranged on the surface of the terminal 100, and is arranged at a different position from the display screen 120.

[0098] The ambient light sensor 170D is used to sense the ambient light brightness. The terminal 100 can adaptively adjust the brightness of the display screen 120 according to the perceived ambient light brightness. The ambient light sensor 170D can also be used to automatically adjust the white balance when shooting. The ambient light sensor 170D can also transmit the environment information of the device to the GPU.

[0099] The ambient light sensor 170D is also used to obtain the brightness, light ratio, color temperature, etc. of the collection environment.

[0100] Figure 2 The software system of the terminal can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture.

[0101] The layered architecture divides the terminal's software system into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the software system can be divided into five layers, namely, application layer (applications), application framework layer (application framework), hardware abstract layer (HAL), driver layer and hardware layer.

[0102] The application layer may include a series of application packages, and the application layer runs applications by calling the application programming interface (API) provided by the application framework layer. For example, the application package may include applications such as browser, camera, gallery, music, and video. It can be understood that the port of each of the above applications can be used to receive data.

[0103] The application framework layer provides APIs and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. For example, the application framework layer may include a window manager, a content provider, a view system, a resource manager, a notification manager, a DHCP (Dynamic Host Configuration Protocol) module, a camera management module, etc.

[0104] The hardware abstraction layer can include multiple library modules, such as display library modules and motor library modules. The hardware abstraction layer can also include scene type detection modules and image selection modules. The terminal system can load the corresponding library modules for the device hardware, thereby achieving the purpose of the application framework layer accessing the device hardware.

[0105] The driver layer is a layer between hardware and software. The driver layer is used to drive the hardware so that the hardware works. The driver layer at least includes a display driver, an audio driver, a camera device driver, a sensor driver, and a motor driver, etc., which are not limited in the embodiments of the present application. It can be understood that the display driver, the audio driver, the camera device driver, the sensor driver, and the motor driver, etc. can all be regarded as a driver node. Each of the above-mentioned driver nodes includes an interface that can be used to receive data.

[0106] The hardware layer is the lowest part of the terminal system, which can be composed of various physical components required such as processor, memory, input and output interface (I / O), etc. The hardware layer can also include image sensors and image signal processors.

[0107] Combine the following Figure 2, the interaction process between various layers of the software system in the process of image capture using the solution provided in the embodiment of the present application is introduced.

[0108] First, the user triggers an image acquisition instruction through the camera application in the application layer, and sends the image acquisition instruction to the application framework layer. The application framework layer can receive the above image acquisition instruction through the camera access interface, and the camera management module in the application framework layer forwards the image acquisition instruction to the HAL layer. The HAL layer forwards the image acquisition instruction to the driver layer, and the driver layer finally forwards the image acquisition instruction to the image sensor and image signal processor in the hardware layer through the camera device driver module; the image sensor and image signal processor in the hardware layer respond to the above image acquisition instruction, acquire images, cache the images, and notify the HAL layer to perform scene type detection and image selection through forwarding by the driver layer.

[0109] Then, on the one hand, the scene type detection module in the HAL layer calls the algorithm from the camera algorithm library and determines the scene type of the shooting scene based on the above algorithm. On the other hand, the image selection module in the HAL layer calls the algorithm from the camera algorithm library and selects a reference image from the cached images according to the scene type of the shooting scene based on the above algorithm.

[0110] Finally, the HAL layer can generate a captured image based on the selected reference image, and feed the captured image back to the gallery application in the application layer via forwarding by the application framework layer, so that the captured image can be displayed in the gallery application.

[0111] Among them, the concepts such as scene types mentioned in the above content refer to the explanations in the subsequent specific embodiments and are not described in detail here.

[0112] The following describes some concepts involved in the image capture solution provided in the embodiments of the present application.

[0113] 1. Second Moment

[0114] The second moment represents the moment when the user clicks the capture button in the user interface of the camera application.

[0115] 2. The first moment

[0116] The user clicking the capture button may trigger an image acquisition instruction, and the moment when the image sensor receives the image acquisition instruction may be referred to as the first moment.

[0117] Since it takes a certain amount of time to transmit commands within the terminal, there is often a certain delay between the moment when the image sensor actually receives the image acquisition command and the moment when the user clicks the snapshot button, that is, the first moment is after the second moment.

[0118] The image capture solution provided in the embodiment of the present application is described in detail below through specific examples.

[0119] See also Figure 3 , is a flow chart of the first image capture method provided in an embodiment of the present application, and the method includes the following steps S301-S305.

[0120] Step S301: in response to the user clicking a capture button, obtaining an image captured by an image sensor.

[0121] The above-mentioned image sensor may refer to a device that utilizes a photoelectric conversion function to convert a light image on a photosensitive surface into an electrical signal that is proportional to the light image and generates an image based on the electrical signal. The image sensor may also be referred to as a SENSOR.

[0122] The user can click a capture button in the user interface of the camera application to trigger an image acquisition instruction. In this way, the terminal can obtain an image captured by the image sensor in response to the user clicking the capture button.

[0123] Depending on the timing when the image sensor starts to collect images, the image obtained in this step may also be different, which will be explained below respectively.

[0124] 1. The image sensor starts capturing images when the camera application is opened.

[0125] In this case, the image sensor can capture images before the user clicks the capture button. In this way, the images captured by the image sensor may include images captured before the second moment, and may also include images captured after the second moment.

[0126] In this case, in this step, an image whose acquisition time is within a time interval before and after the second time can be obtained, and the above time interval can be called a target interval.

[0127] The sub-interval in the target interval that is located before the second moment may be referred to as a preceding interval, and the sub-interval in the target interval that is located after the second moment may be referred to as a succeeding interval.

[0128] According to the second moment, it can be divided into two cases. Figure 4a and Figure 4b Give examples to illustrate each one.

[0129] First case:

[0130] For example, see Figure 4a , is a schematic diagram of the first image acquisition moment provided in an embodiment of the present application.

[0131] Figure 4aIn the figure, T1 represents the acquisition start time of the image sensor, that is, the program start time of the camera application, T2 represents the above second time, and T3 represents the above first time.

[0132] In this case, the target interval to which the image obtained in this step is collected is as follows: Figure 4a As shown, it is the interval shown as [T4, T5], that is, the image captured by the image sensor and the acquisition time is within [T4, T5] can be obtained.

[0133] Among them, T4 can be called the interval start time of the target interval, and [T4, T2] is the above-mentioned previous interval; T5 can be called the interval end time of the target interval, and [T2, T5] is the above-mentioned subsequent interval.

[0134] The above-mentioned time T4 can be determined based on a first preset length of the previous interval, such as T2-first preset length; the time T5 can be determined based on a second preset length of the subsequent interval, such as T2+second preset length.

[0135] In this case, the time length from the second time T2 to the collection start time T1 is longer, that is, [T1, T2]>the first preset length.

[0136] Second case:

[0137] See also Figure 4b , is a schematic diagram of the second image acquisition moment provided in an embodiment of the present application.

[0138] Figure 4b In the figure, T1 represents the start time of image sensor acquisition, that is, the time when the camera application is started, T2 represents the second time, T3 represents the first time, and T4 represents the end time of the interval.

[0139] In this case, the target interval to which the image obtained in this step is collected is as follows: Figure 4b As shown, it is the interval shown as [T1, T4], and T1 is also the starting time of the interval, that is, the image captured by the image sensor and the acquisition time is within [T1, T4] can be obtained.

[0140] In this case, the time length from the second moment T2 to the collection start moment is shorter, that is, [T1, T2]<the first preset length.

[0141] At this time, the length of the preceding interval cannot reach the first preset length, and [T1, T2] can be directly determined as the preceding interval to obtain the longest preceding interval possible. In order to ensure that the target interval reaches a certain length, the length of the succeeding interval can be appropriately extended, and [T2, T4] can be determined as the succeeding interval.

[0142] For example, in this case, the length of the subsequent interval may be the sum of the second preset length and the target length, and the target length may be the difference between the first preset length and the interval length of [T1, T2].

[0143] 2. The image sensor starts collecting images at the first moment.

[0144] In this case, the user has clicked the capture button, and the image sensor receives the image acquisition instruction at the first moment, so that the images acquired by the image sensor are all images acquired after the first moment.

[0145] See also Figure 4c , is a schematic diagram of the third image acquisition moment provided in an embodiment of the present application.

[0146] Figure 4c In the figure, T1 represents the second moment, T2 represents the first moment, and T2 also represents the collection start moment, and T3 represents the interval end moment.

[0147] It can be seen that in this case, the target interval to which the acquisition time of the obtained image belongs can be the time interval shown as [T2, T3], and T2 is also the starting time of the interval.

[0148] Among them, T3 can be determined based on the third preset length of the target interval, such as T2+the third preset length.

[0149] It should be noted that, in this case, the collection start time may not be the first time, but a time after the first time, and the embodiment of the present application does not limit this.

[0150] Step S302: determining the scene type of the shooting scene based on the motion information of the object in the image captured by the image sensor.

[0151] The objects in the image may be people, animals, etc., and the objects in the image may be identified based on various object detection algorithms or pre-trained object recognition models, which is not limited in the embodiments of the present application.

[0152] The action information of the object may be various types of information used to describe the action of the object, which is not limited in the embodiments of the present application.

[0153] For example, it may include the position information of the object, such as the head position, arm position, leg position, etc.; it may also include the movement amplitude information of the object, such as the front and back distance between the object's hands, the front and back distance between the two feet, the jumping height, the upward range of the mouth corners, the degree of body inclination, etc.

[0154] In one case, the above action information may include at least one of the following information:

[0155] 1. The amplitude of the motion of the object in the image

[0156] The above motion amplitude reflects the motion of an object in a single image in the time dimension, and can more accurately measure the dynamic changes of the detection image in the time dimension. Therefore, based on the above motion amplitude, the scene type of the shooting scene can be more accurately determined.

[0157] 2. The difference between the motion amplitude of an object in an image and the motion amplitude of objects in adjacent images.

[0158] An adjacent image of an image refers to an image whose corresponding acquisition time is adjacent to the image acquisition time of the image.

[0159] The meaning of the motion amplitude of the object in the adjacent images is the same as the meaning of the motion amplitude of the object in the detection image.

[0160] The above differences reflect the differences between the motion amplitudes of objects in multiple images that are continuous in time, and reflect the changes in the motion of objects in multiple images in the time dimension. They can more accurately measure the dynamic changes of images in the time dimension, and thus, based on the above differences, the scene type of the shooting scene can be more accurately determined.

[0161] Specifically, the scene type of the shooting scene may be determined based on the motion information of the object in the image in the following manner.

[0162] In one implementation, the scene type may be determined based on the difference in motion information between adjacent images:

[0163] For each image, a first difference between the motion amplitude of an object in the image and the motion amplitude of an object in an adjacent image is determined, and based on the obtained first difference, a scene type of the shooting scene is determined.

[0164] The difference between the motion amplitudes of objects in adjacent images reflects the motion of the objects in the time dimension. Therefore, the scene type of the shooting scene can be determined based on the first difference. Two cases are used as examples below.

[0165] In one case, if the number of objects in the images whose corresponding first difference is greater than the first difference threshold is greater than a certain proportion of the total number of images, the scene type of the shooting scene can be determined to be a motion scene, otherwise the scene type of the shooting scene is determined to be a non-motion scene.

[0166] When the number of targets is greater than a certain ratio of the total number of images, it means that there are more images with larger differences in the motion amplitudes of the objects in the images compared to their adjacent images, that is, the motion of the objects is constantly changing in the time dimension, so the scene type of the shooting scene can be determined to be a motion scene. Otherwise, the scene type can be determined to be a non-motion scene.

[0167] The first difference threshold may be set by a staff member based on experience or actual needs.

[0168] In another case, if the first difference corresponding to any image is greater than the second difference threshold, it can be determined that the scene type of the shooting scene is a motion scene; otherwise, it is determined that the scene type of the shooting scene is a non-motion scene.

[0169] The second difference threshold may be greater than the first difference threshold, and the specific value of the second difference threshold may be set by the staff based on experience or actual needs.

[0170] When the first difference corresponding to a certain image is greater than the second difference threshold, it means that the motion amplitude of the object in the image is greatly different from that of its adjacent images. Since the image and its adjacent images are images collected at consecutive times, and the time span between the collection times of the image and its adjacent images is often small, the above situation indicates that the object has undergone a large motion change within a short time interval, so the scene type of the shooting scene can be determined to be a motion scene. Otherwise, the scene type can be determined to be a non-motion scene.

[0171] In another implementation, the scene type may be determined based on the motion information of each single image:

[0172] It is determined whether there is an image in each image whose motion information is greater than the motion threshold. If yes, the scene type of the shooting scene is determined to be a motion type; otherwise, the scene type of the shooting scene is determined to be a non-motion type.

[0173] For example, the action information may include jumping height. If the jumping height of the object in the image P1 is h1> the motion threshold K1, the shooting scene can be directly determined to be a motion type. This situation indicates that the object in P1 is in a jumping state. Obviously, the above scene type motion type can be directly determined.

[0174] For another example, the motion information may include the distance between the hands and the distance between the feet. If the distance between the hands of the object in the image P2 is s1> the motion threshold K2, and the distance between the feet of the object is s2> the motion threshold K3, then the shooting scene can be directly determined to be a motion type. This situation indicates that the object in P2 is in a running state. Obviously, the above scene type and motion type can be directly determined.

[0175] Step S303: if the scene type is a non-motion scene, a reference image is selected from the images captured by the image sensor based on the first moment and the static quality representation value of the image captured by the image sensor.

[0176] A non-motion scene may also be referred to as a static scene. It should be noted that the static scene mentioned in the embodiments of the present application does not mean a scene in which the object is absolutely still, but rather means that the movement amplitude of the object in the scene is small.

[0177] The above static quality representation value can be obtained based on the static evaluation index of the image, and is used to evaluate the quality information of the image in the static dimension or the spatial dimension.

[0178] In one case, the static quality characterization value may include at least one of the following information:

[0179] Automatic exposure (AE) convergence of the image captured by the image sensor, automatic focus (AF) convergence of the image, automatic white balance (AWB) convergence of the image, clarity of the image, contrast of the image, color saturation of the image, and color uniformity of the image.

[0180] It can be seen that the static quality characterization value can include information corresponding to various types of static evaluation indicators, so as to more comprehensively and accurately reflect the quality information of the above-mentioned image in the static dimension or spatial dimension.

[0181] Specifically, when the scene type is a non-motion scene, the following method may be used to select a reference image from images captured by the image sensor.

[0182] In one implementation, based on the first moment, a first image may be selected from images captured by an image sensor, a static quality characterization value of the first image may be obtained, and a reference image may be selected from images captured by the image sensor according to the obtained static quality characterization value. Figure 5 Steps S503 to S505 in the illustrated embodiment are not described in detail here.

[0183] In another implementation, a preset number of target images whose acquisition times are closest to the target time may be determined from the images acquired by the image sensor, and a reference image may be selected from the target images based on static quality characterization values ​​of the target images.

[0184] The target time may be obtained based on the difference between the first time and the estimated instruction delay.

[0185] From the explanation of the above concepts, it can be seen that due to the delay in the command transmission inside the terminal, after the user clicks the capture button, the image sensor can actually receive the image acquisition command after a certain delay, that is, the first moment is after the second moment.

[0186] Therefore, the target moment obtained based on the difference between the first moment and the estimated instruction delay is actually closer to the second moment, that is, closer to the moment when the user clicks the snapshot button.

[0187] On this basis, a preset number of target images whose acquisition time is closest to the target time among the obtained images are images whose acquisition time is closer to the second time when the user clicks the snapshot button.

[0188] After determining the target image, an image with the largest static quality representation value can be determined from the target images as a reference image, a preset number of images with the largest static instruction representation values ​​can be determined from the selected images as reference images, or images with image quality representation values ​​greater than a preset threshold can be determined from the selected images as reference images, etc.

[0189] In this way, a reference image can be determined whose acquisition time is closer to the second moment when the user clicks the snapshot button. Since the acquisition time of the reference image is closer to the second moment, the reference image is similar to the picture the user sees when clicking the snapshot button.

[0190] Step S304: If the scene type is a motion scene, a reference image is selected from the images captured by the image sensor based on the motion information of the object in the image captured by the image sensor.

[0191] Specifically, the dynamic quality representation value of each image may be obtained based on the motion information of the object in the image captured by the image sensor, and a reference image may be selected from the above images based on the obtained dynamic quality representation value.

[0192] For example, an image with the largest corresponding dynamic quality representation value can be selected as a reference image, a preset number of images with the largest corresponding dynamic instruction representation values ​​can be selected as reference images, or images with corresponding dynamic quality representation values ​​greater than a preset threshold can be selected as reference images, etc.

[0193] The above-mentioned motion quality representation value can be obtained by fusing, weighted calculation and other processing of the information contained in the motion information corresponding to the image, which will not be elaborated here.

[0194] In one embodiment of the present application, if the scene type is a motion scene, a reference image may be selected from the images captured by the image sensor based on motion information of the object in the image captured by the image sensor and a static quality representation value of the image.

[0195] This implementation can be obtained by combining step S303 and the above implementation introduced in this step, for example, determining the image with the largest sum of the corresponding dynamic quality characterization value and static quality characterization value as the reference image, etc., which will not be repeated here.

[0196] In this way, the images captured by the image sensor can be evaluated more comprehensively from both the motion dimension and the static dimension, which is conducive to obtaining a better reference image, and thus is conducive to generating a better captured image based on the reference image.

[0197] Step S305: Generate a captured image based on the reference image.

[0198] Specifically, the captured image may be generated based on the reference image in the following manner.

[0199] In one implementation, the reference image may be directly determined as the captured image.

[0200] In another implementation, a second difference between the motion amplitude of the object in the reference image and the motion amplitude of the object in the second image may be determined. If the second difference is greater than a difference threshold, a captured image is generated based on the reference image; otherwise, a captured image is generated based on the reference image and the second image.

[0201] The second image is: an image adjacent to the reference image among the images captured by the image sensor.

[0202] When the captured image is generated based on the reference image and the second image, the reference image and the second image may be fused to generate the captured image.

[0203] When the second difference is greater than the difference threshold, it means that there is a large difference between the reference image and the second adjacent image. At this time, directly generating the snapshot image based on the reference image can avoid the influence of the second image on the generation of the snapshot image; when the second difference is not greater than the difference threshold, it means that there is a small difference between the reference image and the second adjacent image, that is, the reference image is closer to the second image. At this time, generating the snapshot image based on the reference image and the second image takes the image contents of the reference image and the second image into comprehensive consideration, which is conducive to generating a better snapshot image.

[0204] As can be seen from the above, when the solution provided in the embodiment of the present application is used to generate a captured image, first, in response to the user clicking the capture button, an image captured by the image sensor is obtained, and then based on the motion information of the object in the image captured by the image sensor, the scene type of the shooting scene is determined. When the scene type is a non-motion scene, a reference image is selected from the images captured by the image sensor based on the first moment when the image sensor receives the image acquisition instruction triggered by the user clicking the capture button and the static quality characterization value of the image captured by the image sensor; when the scene type is a motion scene, a reference image is selected from the images captured by the image sensor based on the motion information of the object in the image captured by the image sensor; finally, a captured image is successfully generated based on the selected reference image.

[0205] Among them, before selecting a reference image from the image captured by the image sensor, the scene type of the shooting scene is first distinguished based on the motion information of the object in the image, and the scene type is determined to be a motion scene or a non-motion scene, so that the reference image can be selected from the image captured by the image sensor in different ways based on the determined scene type. It can be seen that in this way, according to the characteristics of the two scene types of motion scene and non-motion scene, the reference image can be selected in a targeted manner, so that the selected reference image is more reasonable and targeted, so that the subsequent captured image generated based on the selected reference image is more reasonable and targeted.

[0206] From the above, it can be seen that the solution provided by the embodiments of the present application can capture better images.

[0207] exist Figure 3 On the basis of the illustrated embodiment, if the scene type of the shooting scene is a non-motion scene, when selecting a reference image from the images captured by the image sensor, the first image can be selected from the images captured by the image sensor based on the first moment, and then the reference image can be selected from the first image according to the static quality characterization value of the first image. In view of the above situation, the embodiment of the present application provides a second image capture method.

[0208] See also Figure 5 , is a flow chart of a second image capture method provided in an embodiment of the present application, the method includes the following steps S501-S507.

[0209] Step S501: in response to the user clicking a capture button, obtaining an image captured by an image sensor.

[0210] Step S502: determining the scene type of the shooting scene based on the motion information of the object in the image captured by the image sensor.

[0211] The above steps S501-S502 are the same as the above steps Figure 3 In the illustrated embodiment, steps S301 and S302 are the same and will not be described in detail here.

[0212] Step S503: If the scene type is a non-motion scene, then based on the first moment, a first image is selected from images captured by the image sensor.

[0213] Specifically, the first image may be selected in the following manner.

[0214] In one implementation, a first image captured before a first moment among images captured by an image sensor may be determined.

[0215] The first image may be all images whose acquisition time is before the first time in the obtained images, or may be a part of all images.

[0216] The above-mentioned partial images may be images randomly selected from all images, or may be a preset number of images whose acquisition time is closest to the first time from all images.

[0217] The first moment is the moment when the image sensor receives the image acquisition instruction. Due to the existence of the instruction delay, the first moment is located after the second moment. In this way, the image acquired before the first moment is an image acquired close to the second moment. Determining the above image as the first image is conducive to the subsequent acquisition of a reference image acquired close to the second moment based on the first image. Since the acquisition time of the reference image is close to the second moment, the reference image is similar to the picture seen by the user when clicking the capture button. Therefore, based on the reference image, a capture image similar to the picture seen by the user when clicking the capture button can be generated. For the user, the above capture image is a better image, which improves the user experience.

[0218] In one case, a first image captured by the image sensor before a first moment and after a second moment may be determined.

[0219] In this way, the acquisition time of the first image determined is between the moment when the user clicks the snapshot button and the moment when the image sensor receives the image acquisition instruction, that is, the first image that is similar to the picture seen at the second moment when the user clicks the snapshot button and the acquisition time is after the second moment can be determined, so that the determined first image is more reasonable.

[0220] In another implementation, a first image acquired after a first moment among images acquired by the image sensor may be determined.

[0221] Similarly, the first image may be all images in the obtained images whose acquisition time is after the first moment, or may be a portion of all images. Furthermore, the portion of images may be randomly selected images from all images, or may be a preset number of images in all images whose acquisition time is closest to the first moment.

[0222] The first moment is the moment when the image acquisition instruction is received. When a user uses a terminal to take a photo, he or she will often hold the terminal steadily for a period of time after clicking the photo button. Therefore, the image acquired after the first moment is often an image acquired when the terminal is in a stable posture, and its static image quality is often better. Therefore, the image acquired after the first moment is determined as the first image, and an image with higher static image quality can be obtained, so that the image quality of the reference image subsequently selected from the first image is higher, which improves the image quality of the captured image generated based on the reference image. For the user, the above-mentioned captured image is a better image, which improves the user experience.

[0223] In still another implementation, a first image acquired before a first moment among images acquired by the image sensor may be determined, and a first image acquired after the first moment among images acquired by the image sensor may be determined.

[0224] This embodiment can be obtained by combining the above two embodiments, which will not be described in detail here.

[0225] This is beneficial for generating a snapshot image closer to the first moment based on the reference image on the one hand, and can improve the image quality of the snapshot image generated based on the reference image on the other hand, thereby obtaining a better snapshot image.

[0226] Step S504: Obtain a static quality representation value of the first image.

[0227] The method of obtaining the static instruction representation value has been described above. Figure 3 This is described in step S303 in the illustrated embodiment and will not be described again here.

[0228] Step S505: selecting a reference image from the first image according to the obtained static quality representation value.

[0229] In one implementation, the second moment when the user clicks the snapshot button may be determined, and a reference image may be selected from the first images based on the second moment and the obtained static quality representation value.

[0230] Since the second moment is the moment when the user clicks the snapshot button, it is helpful to determine a reference image that is relatively similar to the picture seen by the user when the user clicks the snapshot button based on the second moment.

[0231] In one case, based on the obtained static quality characterization value, a reference image whose interval between the acquisition moment and the second moment is less than a preset time length and whose image quality meets the standard can be determined from the first image.

[0232] It can be determined whether the static instruction characterization value is greater than a preset first quality threshold. If so, it is determined that the image quality of the first image meets the standard. Otherwise, it is determined that the image quality of the first image does not meet the standard.

[0233] The first quality threshold mentioned above can be set by the staff according to experience or actual needs.

[0234] In another case, based on the obtained static quality characterization value, a reference image whose acquisition time is closest to the second time and whose image quality meets the standard can be determined from the first image.

[0235] For example, starting from the first moment, the static quality characterization values ​​of the first images can be traversed in order of acquisition time from late to early, so as to determine a reference image among the first images whose acquisition time is closest to the second moment and whose image quality meets the standard.

[0236] In this way, under the premise that the image quality meets the standard, the reference image that is closest to the acquisition moment and the second moment can be determined. Since the second moment is the moment when the user actually clicks the snapshot button, the reference image that is most similar to the picture seen by the user when he actually clicks the snapshot button can be determined as much as possible. The above image is close to the picture seen by the user when he clicks the snapshot button, that is, "what you see is what you get". In this way, for the user, the above snapshot image is a better image, which improves the user experience.

[0237] Since the second moment is the moment when the user actually clicks the snapshot button, based on the second moment, a reference image close to the second moment can be selected from the first image, that is, an image that is relatively similar to the picture seen at the moment when the user actually clicks the snapshot button is selected, which is conducive to generating a snapshot image based on the reference image that is relatively similar to the picture seen at the moment when the user actually clicks the snapshot button. The above image is closer to the picture seen when the user clicks the snapshot button. In this way, for the user, the above snapshot image is a better image, which improves the user experience.

[0238] Step S506: If the scene type is a motion scene, a reference image is selected from the images captured by the image sensor based on the motion information of the object in the image captured by the image sensor.

[0239] Step S507: Generate a captured image based on the reference image.

[0240] The above steps S506-S507 are the same as the above Figure 3 In the illustrated embodiment, steps S304 to S305 are the same and will not be described in detail here.

[0241] As can be seen from the above, when the solution provided in the embodiment of the present application is applied to select a reference image, firstly, based on the first moment, a first image is selected from the images captured by the image sensor, and then, based on the static quality characterization value of the first image, a reference image is selected from the first image. Since the first moment is the moment when the image acquisition instruction is received, based on the first moment, a first image associated with the moment when the image acquisition instruction is received can be selected from the images captured by the image sensor. The first image associated with the moment when the image acquisition instruction is received is generally similar to the picture the user actually sees when clicking to take a photo. Therefore, by selecting a reference image from the first image, the subsequent captured image generated based on the reference image can be closer to the picture the user sees when clicking the capture button, thereby obtaining a better captured image.

[0242] The following two examples provide a more intuitive explanation of the solution provided in the embodiment of the present application for image capture in non-motion scenes and action scenes.

[0243] See also Figure 6 , which is a schematic diagram of the first image capture effect provided in an embodiment of the present application.

[0244] Figure 6 It is a schematic diagram of the image capture effect in a non-motion scene, wherein the images captured by the image sensor are P1-P4, the acquisition times of P1-P4 are t1-t4 respectively, and t1-t4 are arranged in order from early to late.

[0245] It should be noted that, since the shooting scene is a non-motion scene, P1-P4 can all be similar images with little difference in image content. The difference is only in the static quality of the image. Figure 6 The difference in the image content of P1-P4 is not shown.

[0246] Among them, the acquisition time t1 of P1 corresponds to the second moment when the user clicks the capture button. After obtaining the static quality characterization values ​​of P1-P4, P2 can be determined as a reference image based on the static quality characterization values ​​of P1-P4, and a captured image can be generated based on P2. For example, P2 is determined to be an image whose acquisition time is closest to the second moment and whose image quality meets the standard, so P2 can be determined as a reference image, and a captured image can be generated based on P2.

[0247] It can be seen that when the scheme provided by the embodiment of the present application is used to capture an image, when the scene type of the shooting scene is a non-motion scene, after obtaining the image captured by the image sensor, the second moment when the user clicks the capture button and the image quality are comprehensively considered, and the reference image closest to the second moment when the user clicks the capture button can be determined on the premise that the image quality meets the standard. This can ensure that the reference image has a high quality on the one hand, and on the other hand, it is conducive to determining a reference image that is close to the picture the user sees when clicking the capture button, so that the subsequent capture image generated based on the reference image can be closer to the picture the user sees when clicking the capture button, and a better capture image can be obtained.

[0248] See also Figure 7 , which is a schematic diagram of the second image capture effect provided in an embodiment of the present application.

[0249] Figure 7 It is a schematic diagram of the image capture effect in a motion scene, wherein the images captured by the image sensor are P5-P8, the acquisition times of P5-P8 are t5-t8 respectively, and t5-t8 are arranged in order from early to late.

[0250] Among them, the acquisition time t6 of P6 corresponds to the second time when the user clicks the snapshot button. After obtaining the motion information of the objects in P5-P8, P7 can be determined as a reference image based on the motion information of the objects in P5-P8, and a snapshot image can be generated based on P7. For example, P7 can be determined to be the image with the largest dynamic quality representation value, so P7 can be determined as a reference image, and a snapshot image can be generated based on P7.

[0251] It can be seen that when the solution provided in the embodiment of the present application is used to capture images, when the scene type of the shooting scene is a motion scene, the images collected before and after the second moment when the user clicks the capture button are taken into consideration, and based on the motion information of these images, the image with the largest dynamic quality characterization value can be selected as the reference image. In this way, a reference image with the most exciting movement of the object in the image can be captured from the obtained image, so that a more exciting captured image can be generated based on the reference image, that is, a better captured image can be obtained.

[0252] The user information involved in the embodiments of the present application is information that has been authorized by the user. The acquisition, storage, use, processing, transmission, provision and disclosure of the user information comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0253] In a specific implementation, the present application further provides a computer storage medium, wherein the computer storage medium may store a program, wherein when the program is executed, the device where the computer readable storage medium is located is controlled to perform some or all of the steps in the above embodiment. The above storage medium may be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0254] In a specific implementation, an embodiment of the present application further provides a computer program product, which includes executable instructions. When the executable instructions are executed on a terminal, the terminal executes some or all of the steps in the above method embodiment.

[0255] In a specific implementation, the embodiment of the present application further provides a terminal, including:

[0256] one or more processors, image sensors, and memory;

[0257] The memory is coupled to the one or more processors, and the memory is used to store computer program codes, wherein the computer program codes include computer instructions, and the one or more processors call the computer instructions to enable the terminal to execute the aforementioned image capture method.

[0258] like Figure 8As shown, the present application also provides a chip system, which is applied to a terminal 100. The chip system includes one or more processors 801. The processor 801 is used to call computer instructions so that the terminal 100 inputs the data to be processed into the chip system. The chip system processes the data based on the image capture method provided in an embodiment of the present application and outputs the processing result.

[0259] In a possible implementation, the chip system also includes input and output interfaces for inputting and outputting data.

[0260] The various embodiments of the mechanism disclosed in the present application can be implemented in hardware, software, firmware or a combination of these implementation methods. The embodiments of the present application can be implemented as a computer program or program code executed on a programmable system, which includes at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device and at least one output device.

[0261] Program code can be applied to input instructions to perform the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, a processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application specific integrated circuit (ASIC), or a microprocessor.

[0262] Program code can be implemented with high-level programming language or object-oriented programming language to communicate with the processing system. When necessary, program code can also be implemented with assembly language or machine language. In fact, the mechanism described in this application is not limited to the scope of any specific programming language. In either case, the language can be a compiled language or an interpreted language.

[0263] In some cases, the disclosed embodiments can be implemented in hardware, firmware, software or any combination thereof. The disclosed embodiments can also be implemented as instructions carried or stored on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which can be read and executed by one or more processors. For example, instructions can be distributed over a network or through other computer-readable media. Therefore, machine-readable media can include any mechanism for storing or transmitting information in a machine (e.g., computer) readable form, including but not limited to, floppy disks, optical disks, optical disks, compact disc read-only memories (Compact Disc Read Only Memory, CD-ROMs), magneto-optical disks, read-only memories, random access memories, erasable programmable read-only memories (Erasable Programmable Read Only Memory, EPROM), electrically erasable programmable read-only memories (Electrically Erasable Programmable Read Only Memory, EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable memories for transmitting information (e.g., carrier waves, infrared signal digital signals, etc.) using the Internet in electrical, optical, acoustic or other forms of propagation signals. Accordingly, machine-readable media include any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).

[0264] In the accompanying drawings, some structural or method features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be required. Instead, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the accompanying drawings of the specification. In addition, the inclusion of structural or method features in a particular figure does not mean that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.

[0265] It should be noted that the units / modules mentioned in the various device embodiments of the present application are all logical units / modules. Physically, a logical unit / module can be a physical unit / module, or a part of a physical unit / module, or can be implemented as a combination of multiple physical units / modules. The physical implementation method of these logical units / modules themselves is not the most important. The combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed by the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed by the present application, which does not mean that there are no other units / modules in the above-mentioned device embodiments.

[0266] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0267] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.

Claims

1. An image capture method, characterized in that: The method comprises: In response to the user clicking a capture button, obtaining an image captured by the image sensor; Determining a scene type of a shooting scene based on motion information of an object in an image captured by the image sensor; If the scene type is a non-motion scene, a reference image is selected from the images captured by the image sensor based on a first moment and a static quality characterization value of the image captured by the image sensor, wherein the first moment is: the moment when the image sensor receives an image capture instruction triggered by the user clicking a capture button; If the scene type is a motion scene, selecting a reference image from the images captured by the image sensor based on motion information of the object in the image captured by the image sensor; Based on the reference image, generating a captured image; The acquisition time of the image acquired by the image sensor belongs to the target interval, wherein the target interval is determined in the following manner: If the acquisition start time of the image sensor is the start time of the camera application, and the length of the time interval between the acquisition start time and the second time is greater than the first preset length of the previous interval, then the interval start time of the target interval is: the difference between the second time and the first preset length, and the interval end time of the target interval is: the sum of the second time and the second preset length of the subsequent interval, wherein the second time is: the time when the user clicks the snapshot button, the previous interval is: the sub-interval of the target interval before the second time, and the subsequent interval is: the sub-interval of the target interval after the second time; If the collection start time is the program start time, and the time interval length between the collection start time and the second time is less than the first preset length, then the interval start time is: the collection start time, and the interval end time is: the sum of the second time, the second preset length and the target length, wherein the target length is the difference between the first preset length and the time interval length; If the acquisition start time is the first time, then the interval start time is: the first time, and the interval end time is: the sum of the first time and the third preset length; The determining the scene type of the shooting scene based on the motion information of the object in the image collected by the image sensor includes: Determining the scene type based on the motion information of each single image: determining whether there is an image with motion information greater than a motion threshold among the images captured by the image acquisition device, and if so, determining that the scene type of the shooting scene is a motion type, otherwise determining that the scene type of the shooting scene is a non-motion type; The step of selecting a reference image from the images captured by the image sensor based on the first moment and the static quality characterization value of the image captured by the image sensor comprises: A preset number of target images whose acquisition time is closest to a target time are determined from the images acquired by the image sensor, and a reference image is selected from the target images based on a static quality characterization value of the target image, wherein the target time is obtained based on a difference between the first time and the estimated instruction delay.

2. The method according to claim 1, characterized in that The step of selecting a reference image from the images captured by the image sensor based on the first moment and the static quality characterization value of the image captured by the image sensor comprises: Based on a first moment, selecting a first image from images acquired by the image sensor; Obtaining a static quality representation value of the first image; A reference image is selected from the first images according to the obtained static quality representation value.

3. The method according to claim 2, characterized in that The selecting, based on the first moment, a first image from images acquired by the image sensor comprises: The first image is selected in at least one of the following ways: Determine a first image captured before the first moment among the images captured by the image sensor; A first image acquired after the first moment among the images acquired by the image sensor is determined.

4. The method according to claim 2, characterized in that: The step of selecting a reference image from the first image according to the obtained static quality representation value comprises: Determine the second moment when the user clicks the snapshot button; Based on the second moment and the obtained static quality characterization value, a reference image is selected from the first images.

5. The method according to claim 4, characterized in that The selecting a reference image from the first image based on the second moment and the obtained static quality characterization value comprises: Based on the obtained static quality characterization value, a reference image whose acquisition time is closest to the second time and whose image quality meets the standard is determined from the first image.

6. The method according to any one of claims 1 to 5, characterized in that The static quality characterization value includes at least one of the following information: Automatic exposure AE convergence of the image captured by the image sensor; Auto-focus AF convergence degree of the image captured by the image sensor; Automatic white balance (AWB) convergence of the image captured by the image sensor; The clarity of the image captured by the image sensor; The contrast of the image captured by the image sensor; Color saturation of the image captured by the image sensor; Color uniformity of the image captured by the image sensor.

7. The method according to any one of claims 1 to 5, characterized in that The action information includes at least one of the following information: the motion amplitude of the object in the image captured by the image sensor; The difference between the motion amplitude of the object in the image captured by the image sensor and the motion amplitude of the object in the adjacent image.

8. The method according to any one of claims 1 to 5, characterized in that The selecting a reference image from the image acquired by the image sensor based on the motion information of the object in the image acquired by the image sensor comprises: A reference image is selected from the images captured by the image sensor based on the motion information of the object in the image captured by the image sensor and the static quality representation value of the image.

9. The method according to any one of claims 1 to 5, characterized in that The step of generating a captured image based on the reference image comprises: Determine a second difference between the motion amplitude of the object in the reference image and the motion amplitude of the object in the second image, wherein the second image is: an image adjacent to the reference image among the images acquired by the image sensor; If the second difference is greater than the difference threshold, generating a captured image based on the reference image; Otherwise, a captured image is generated based on the reference image and the second image.

10. A terminal, characterized in that: include: one or more processors, image sensors, and memory; The memory is coupled to the one or more processors, and the memory is used to store computer program codes, where the computer program codes include computer instructions. The one or more processors call the computer instructions to enable the terminal to execute the method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: The method comprises a computer program, and when the computer program is run on a terminal, the terminal is caused to execute the method according to any one of claims 1 to 9.

12. A computer program product, characterized in that The computer program product comprises executable instructions, and when the executable instructions are executed on a terminal, the terminal is caused to execute the method according to any one of claims 1 to 9.

13. A chip system, characterized in that: The chip system is applied to a terminal, and the chip system includes one or more processors, and the processor is used to call computer instructions so that the terminal inputs data into the chip system, and executes the method described in any one of claims 1 to 9 to process the data and then output the processing results.

Images