Image processing methods and apparatus
By acquiring a reference frame in the image preview interface and performing image fusion processing in real time, the problem of excessively long shooting time caused by multi-frame fusion technology is solved, thus improving the user experience.
Patent Information
- Application Number
- CN202110693408.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-22
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2041-06-22
AI Technical Summary
Multi-frame fusion technology increases shooting time and affects user experience.
A reference frame is obtained from the image preview interface, and image fusion processing is performed in real time based on the reference frame, thereby shortening the image processing time.
While ensuring image quality, the shooting time was reduced, thus improving the user experience.
Smart Images

Figure CN115514875B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to image processing technology, and more particularly to an image processing method and apparatus for multi-frame fusion scenarios. Background Technology
[0002] With the rapid development of digital imaging technology, multi-frame fusion technology has emerged to provide users with a better shooting experience and achieve better shooting results.
[0003] However, this approach, due to the introduction of multi-frame acquisition and subsequent processing, leads to longer shooting times and affects the user experience. Summary of the Invention
[0004] To overcome the problems existing in related technologies, this disclosure provides an image processing method and apparatus. By acquiring the reference frame in the image preview interface, the efficiency of image fusion processing is greatly improved, achieving an imaging and shooting solution with a better user experience.
[0005] According to a first aspect of the present disclosure, an image processing method is provided, comprising:
[0006] Obtain the reference frame based on the preview frames captured during the image preview process;
[0007] In response to the shooting command, the multiple image frames acquired are fused based on the reference frame to obtain the output image.
[0008] Optionally, the step of obtaining the reference frame based on the preview frames captured during the image preview process includes:
[0009] The image preview process is analyzed frame by frame based on preset image metrics to obtain the analysis results of each preview frame.
[0010] Preview frames whose analysis results meet the preset criteria for selecting reference frames are identified as the reference frames.
[0011] Optionally, the step of determining the preview frame whose analysis result meets the preset reference frame selection criteria as the reference frame includes:
[0012] Determine the shutter trigger time;
[0013] The preview frame with the smallest difference between the preview display time and the shutter trigger time, and whose analysis result meets the preset reference frame selection criteria, is determined as the reference frame.
[0014] Optionally, the image metrics include at least one or more of the following parameters:
[0015] Clarity, dynamic range, and imaging effect.
[0016] Optionally, the step of fusing multiple acquired image frames based on the reference frame to obtain an output image includes:
[0017] Based on the aforementioned reference frame, image fusion processing is performed on the captured image frames in real time.
[0018] After the last frame of the image obtained during the shooting process is fused, the output image is obtained.
[0019] According to a second aspect of the present disclosure, an image processing apparatus is provided, comprising:
[0020] The reference frame determination module is used to obtain a reference frame based on the preview frames collected during the image preview process;
[0021] The image fusion module is used to respond to the shooting command, and fuse multiple acquired image frames based on the reference frame to obtain an output image.
[0022] Optionally, the reference frame determination module includes:
[0023] The image analysis submodule is used to perform frame-by-frame analysis on each preview frame acquired during the image preview process based on preset image metrics, and to obtain the analysis results of each preview frame.
[0024] The reference frame selection submodule is used to determine the preview frames whose analysis results meet the preset reference frame selection conditions as the reference frames.
[0025] Optionally, the reference frame selection submodule includes:
[0026] The shutter detection submodule is used to determine the shutter trigger time;
[0027] The frame selection submodule is used to determine the preview frame with the smallest difference between the preview display time and the shutter trigger time, and whose analysis result meets the preset benchmark frame selection conditions, as the benchmark frame.
[0028] Optionally, the image metrics include at least one or more of the following parameters:
[0029] Clarity, dynamic range, and imaging effect.
[0030] Optionally, the image fusion module includes:
[0031] The real-time fusion submodule is used to perform image fusion processing on the captured image frames in real time based on the reference frame;
[0032] The output submodule is used to obtain the output image after the last image frame obtained during the shooting process has been fused.
[0033] According to a third aspect of the present disclosure, a computer apparatus is provided, comprising:
[0034] processor;
[0035] Memory used to store processor-executable instructions;
[0036] The processor is configured as follows:
[0037] Obtain the reference frame based on the preview frames captured during the image preview process;
[0038] In response to the shooting command, the multiple image frames acquired are fused based on the reference frame to obtain the output image.
[0039] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of a mobile terminal, the mobile terminal is enabled to perform an image processing method, the method comprising:
[0040] Obtain the reference frame based on the preview frames captured during the image preview process;
[0041] In response to the shooting command, the multiple image frames acquired are fused based on the reference frame to obtain the output image.
[0042] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: A reference frame is obtained based on the preview frames acquired during the image preview process; then, in response to a shooting command, multiple acquired image frames are fused based on the reference frame to obtain an output image. That is, obtaining the reference frame during the image preview process provides a foundation for the rapid initiation of subsequent image fusion, shortens image processing time, solves the problem of long shooting times affecting user experience, and reduces shooting time while ensuring image quality.
[0043] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0044] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0045] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment.
[0046] Figure 2 This is a flowchart illustrating yet another image processing method according to an exemplary embodiment.
[0047] Figure 3 This is a schematic diagram illustrating a reference frame acquisition process according to an exemplary embodiment.
[0048] Figure 4 This is a flowchart illustrating yet another image processing method according to an exemplary embodiment.
[0049] Figure 5 This is a flowchart illustrating yet another image processing method according to an exemplary embodiment.
[0050] Figure 6 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.
[0051] Figure 7 This is a schematic diagram of the structure of a reference frame determination module 601 according to an exemplary embodiment.
[0052] Figure 8 This is a schematic diagram of the structure of the reference frame selection submodule 702 according to an exemplary embodiment.
[0053] Figure 9 This is a schematic diagram of the structure of an image fusion module 602 according to an exemplary embodiment.
[0054] Figure 10 This is a block diagram illustrating an apparatus according to an exemplary embodiment.
[0055] Figure 11 This is a block diagram illustrating an apparatus according to an exemplary embodiment. Detailed Implementation
[0056] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.
[0057] Multi-frame fusion involves acquiring multiple image frames consecutively, typically selecting the clearest and best-looking frame as the reference frame (also known as the anchor frame). Based on the reference frame, an algorithm is used to synthesize the acquired image frames into a single image to improve the quality of the captured photograph.
[0058] Image multi-frame fusion involves selecting a reference frame from the captured images after shooting and then integrating other image frames based on this reference frame to obtain a higher quality image. However, this method increases shooting time and negatively impacts user experience.
[0059] To address the aforementioned problems, embodiments of this disclosure provide an image processing method and apparatus. By acquiring the reference frame directly in the image preview interface, the efficiency of image fusion processing is significantly improved, resulting in an imaging and shooting solution with a better user experience.
[0060] An exemplary embodiment of this disclosure provides an image processing method, the process of obtaining a high-quality image using this method is as follows: Figure 1 As shown, it may include:
[0061] Step 101: Obtain the reference frame based on the preview frames captured during the image preview process.
[0062] In this step, after opening the camera and entering the image preview interface, the image preview process begins, allowing for the acquisition of a reference frame. During the image preview, each preview frame is analyzed frame by frame, and the most suitable preview frame is selected as the reference frame.
[0063] Step 102: In response to the shooting command, based on the reference frame, fuse the acquired multiple image frames to obtain the output image.
[0064] In this step, a reference frame has been selected before shooting. Therefore, when a shooting command is received and shooting is performed in response to the shooting command, the multiple image frames obtained can be fused into the reference frame to obtain a higher quality final image.
[0065] The shooting command can be a real-time user operation, such as obtaining it based on the user touching the shutter button on the screen or pressing a physical button; or it can be a shooting command automatically generated based on the judgment that the shutter trigger condition is met in automatic shooting scenarios such as time-lapse shooting and automatic recognition shooting.
[0066] Acquiring a reference frame during the image preview process provides a foundation for the rapid initiation of subsequent image fusion, shortens image processing time, solves the problem of long shooting time affecting user experience, and achieves reduced shooting time while ensuring image quality.
[0067] An exemplary embodiment of this disclosure also provides an image processing method, the process of which for obtaining a reference frame is as follows: Figure 2 As shown, it may include:
[0068] Step 201: Analyze each preview frame acquired during the image preview process based on preset image metrics to obtain the analysis results of each preview frame.
[0069] The image metrics include at least one or more of the following parameters:
[0070] Clarity, dynamic range, and imaging effect.
[0071] Image metrics describe the degree to which an image matches the shooting scene, and can include parameters such as image quality and shooting range. By selecting the reference frame that best matches the shooting scene and has the highest quality, higher quality imaging can be achieved. For example, imaging effects can include at least one of the following: image sharpness, contrast, distortion, chromatic aberration, tonal gradation, and color gradation.
[0072] It should be noted that the parameters used to evaluate the standardization of preview frames are not limited to those listed above, and can be adjusted according to factors such as the hardware and software configuration of the shooting equipment and the needs of the application scenario.
[0073] In this step, the analysis results can be expressed as fractions.
[0074] For example, this can be implemented using hardware such as an Image Front End (IFE) module and an Image Processing Engine (IPE) module. Figure 3 As shown, after the camera sensor detects that the lens has started and entered the preview interface, the hardware IFE / IPE for analysis is initiated. Then, in preview mode, each preview frame output by the camera sensor is analyzed frame by frame through IFE or IPE, and a "baseline score" is assigned to each preview frame (Frame n to Frame n+6) in real time to indicate the analysis result. Among them, Frame n to Frame n+4 are the preview frames for which the anchor score has been obtained after analysis, and Frame n+5 to Frame n+6 are the preview frames to be analyzed by IFE or IPE.
[0075] Step 202: Select the preview frames whose analysis results meet the preset reference frame selection conditions as the reference frames.
[0076] In this step, if there are one or more preview frames that have reached the standard threshold, one of the preview frames is selected as the reference frame.
[0077] For example, such as Figure 4 As shown, step 202 may include:
[0078] Step 401: Determine the shutter trigger time.
[0079] Step 402: The preview frame with the smallest difference between the preview display time and the shutter trigger time, and whose analysis result meets the preset reference frame selection conditions, is determined as the reference frame.
[0080] After detecting a shutter trigger operation, the preview frame whose preview display time is closest to the shutter trigger time and whose analysis result meets the preset reference frame selection criteria is selected as the reference frame. In some possible embodiments, since image processing is required during image acquisition before being output and displayed to the user for preview, the preview frame currently displayed on the image preview interface may have a certain delay compared to the already processed preview frame. Here, by selecting the preview frame whose preview display time is closest to the shutter trigger time, it is ensured that the selected preview frame better meets the user's current shooting needs.
[0081] For example, using an anchor score of 7 as the baseline frame selection criterion, preview frames with an anchor score greater than 7 are considered to meet the baseline frame selection criteria based on their analysis. This reduces the time spent acquiring anchor frames, optimizes the speed of image capture, and improves the user experience. For example, combining a reference... Figure 3 The scenario shown is as follows: Figure 3 In this context, Frame n represents the currently displayed preview frame in the image preview interface, and Frame n+1 to Frame n+4 represent preview frames that have undergone IFE or IPE preprocessing and are ready for preview display. Assuming that the shutter is triggered when the current preview frame Frame n is displayed in the image preview interface, if the anchor score is 7 as the reference frame selection condition, Frame n, which has the closest preview display time and shutter trigger time and an anchor score greater than 7, will be selected as the reference frame; if the anchor score is 8 as the reference frame selection condition, Frame n+1, which has the closest preview display time and shutter trigger time and an anchor score greater than 8, will be selected as the reference frame (assuming that the anchor scores of all preview frames before Frame n are less than 8).
[0082] It should be noted that the criteria for selecting the reference frame can be customized or the rules can be defined and learned by the device itself.
[0083] In this embodiment, a reference frame is selected from the preview frames acquired during the preview process. After the shutter is triggered, image fusion processing based on the selected reference frame can be performed simultaneously with image acquisition, which greatly shortens the processing time for imaging using image fusion technology.
[0084] An exemplary embodiment of this disclosure also provides an image processing method, the process of which includes image integration processing as follows: Figure 5 As shown, it may include:
[0085] Step 501: Based on the reference frame, perform image fusion processing on the captured image frames in real time.
[0086] In this step, after detecting trigger commands such as shutter activation, the system officially enters the image capture state, continuously capturing multiple image frames. Simultaneously, the acquired image frames are fused with the reference frame in real time.
[0087] In some possible embodiments, one or more preview frames whose preview display time is before the reference frame can also be fused into the reference frame. That is, the images participating in the image fusion process include those before and after the shutter trigger, resulting in richer image samples and the acquisition of higher quality output images. For example, preview frames whose preview display time is before the reference frame and whose analysis results meet the preset reference frame selection criteria can be fused into the reference frame to ensure the quality of the fused preview frames.
[0088] Step 502: After the last frame of the image obtained during the shooting process is fused, the output image is obtained.
[0089] In this step, all image frames acquired during the shooting process are fused with the reference frame to obtain the output image. Because the reference frame is acquired in the preview stage, the fusion operation and the shooting process can be performed in real time, and the image fusion processing is completed earlier, ultimately providing the user with the output image faster. This significantly reduces the waiting time caused by the fusion processing and improves the user experience.
[0090] An exemplary embodiment of this disclosure also provides an image processing apparatus, the structure of which is as follows: Figure 6 As shown, it may include:
[0091] The reference frame determination module 601 is used to obtain a reference frame based on the preview frame collected during the image preview process;
[0092] The image fusion module 602 is used to fuse multiple acquired image frames based on the reference frame in response to the shooting command, and obtain an output image.
[0093] Optionally, the structure of the reference frame determination module 601 is as follows: Figure 7 As shown, it may include:
[0094] The image analysis submodule 701 is used to perform frame-by-frame analysis on each preview frame acquired during the image preview process based on preset image indicators, and obtain the analysis results of each preview frame.
[0095] Optionally, the image metrics include at least one or more of the following parameters:
[0096] Sharpness, dynamic range, and imaging quality;
[0097] The reference frame selection submodule 702 is used to determine the preview frame that meets the preset reference frame selection conditions as the reference frame.
[0098] Optionally, the structure of the reference frame selection submodule 702 is as follows: Figure 8 As shown, it may include:
[0099] Shutter detection submodule 801 is used to determine the shutter trigger time;
[0100] The frame selection submodule 802 is used to determine the preview frame with the smallest difference between the preview display time and the shutter trigger time, and whose analysis result meets the preset benchmark frame selection conditions, as the benchmark frame.
[0101] Optionally, the structure of the image fusion module 602 is as follows: Figure 9 As shown, it may include:
[0102] The real-time fusion submodule 901 is used to perform image fusion processing on the captured image frames in real time based on the reference frame;
[0103] The output submodule 902 is used to obtain the output image after the last image frame obtained during the shooting process has been fused.
[0104] The aforementioned devices can be integrated into mobile phones, cameras, computers with webcams, and other devices that have shooting capabilities, allowing the devices to perform the corresponding functions. Alternatively, some functional modules can be integrated into devices with shooting capabilities, while others can be integrated into a remote processing platform, with the devices and the remote processing platform working together to achieve the corresponding functions.
[0105] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0106] Figure 10 This is a block diagram illustrating an image processing apparatus 1000 according to an exemplary embodiment. For example, apparatus 1000 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.
[0107] Reference Figure 10 The device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.
[0108] Processing component 1002 typically controls the overall operation of device 1000, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1002 may include one or more processors 1020 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1002 may include one or more modules to facilitate interaction between processing component 1002 and other components. For example, processing component 1002 may include a multimedia module to facilitate interaction between multimedia component 1008 and processing component 1002.
[0109] Memory 1004 is configured to store various types of data to support the operation of device 1000. Examples of this data include instructions for any application or method operating on device 1000, contact data, phonebook data, messages, pictures, videos, etc. Memory 1004 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0110] The power supply component 1006 provides power to the various components of the device 1000. The power supply component 1006 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 1000.
[0111] Multimedia component 1008 includes a screen that provides an output interface between the device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1008 includes a front-facing camera and / or a rear-facing camera. When the device 1000 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0112] Audio component 1010 is configured to output and / or input audio signals. For example, audio component 1010 includes a microphone (MIC) configured to receive external audio signals when device 1000 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1004 or transmitted via communication component 1016. In some embodiments, audio component 1010 also includes a speaker for outputting audio signals.
[0113] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.
[0114] Sensor assembly 1014 includes one or more sensors for providing state assessments of various aspects of device 1000. For example, sensor assembly 1014 may detect the on / off state of device 1000, the relative positioning of components such as the display and keypad of device 1000, changes in the position of device 1000 or a component of device 1000, the presence or absence of user contact with device 1000, the orientation or acceleration / deceleration of device 1000, and temperature changes of device 1000. Sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1014 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.
[0115] Communication component 1016 is configured to facilitate wired or wireless communication between device 1000 and other devices. Device 1000 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 1016 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1016 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0116] In an exemplary embodiment, the apparatus 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.
[0117] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, which can be executed by a processor 1020 of the device 1000 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0118] A computer device comprising:
[0119] processor;
[0120] Memory used to store processor-executable instructions;
[0121] The processor is configured as follows:
[0122] Obtain the baseline frame in the image preview interface;
[0123] Multiple captured image frames are fused into the reference frame to obtain the output image.
[0124] A non-transitory computer-readable storage medium, when instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform an image processing method, the method comprising:
[0125] Obtain the baseline frame in the image preview interface;
[0126] Multiple captured image frames are fused into the reference frame to obtain the output image.
[0127] Figure 11 This is a block diagram illustrating an apparatus 1100 for image processing according to an exemplary embodiment. For example, apparatus 1100 may be provided as a server. (Refer to...) Figure 11 The apparatus 1100 includes a processing component 1122, which further includes one or more processors, and memory resources represented by memory 1132 for storing instructions, such as application programs, that can be executed by the processing component 1122. The application programs stored in memory 1132 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1122 is configured to execute instructions to perform the methods described above.
[0128] Device 1100 may also include a power supply component 1126 configured to perform power management of device 1100, a wired or wireless network interface 1150 configured to connect device 1100 to a network, and an input / output (I / O) interface 1158. Device 1100 may operate on an operating system stored in memory 1132, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or similar.
[0129] This disclosure provides an image processing method and apparatus. A reference frame is acquired in an image preview interface, and then multiple captured image frames are fused into the reference frame to obtain an output image. Acquiring the reference frame in the image preview interface before triggering the shutter provides a foundation for rapid initiation of subsequent image fusion, shortens image processing time, solves the problem of long shooting times affecting user experience, and reduces shooting time while ensuring image quality.
[0130] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.
[0131] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.
Claims
1. An image processing method, characterized in that, include: After entering the image preview interface, each preview frame is analyzed frame by frame based on the preview frames captured during the image preview process to obtain the baseline frame; In response to a shooting command, while acquiring multiple image frames, the acquired multiple image frames are fused to the reference frame in parallel. Preview frames whose preview display time is before the reference frame and whose analysis results meet the preset reference frame selection conditions are fused into the reference frame to obtain an output image. The shooting command is obtained based on the user touching the shutter button on the screen or pressing a physical button, or it is automatically generated based on the judgment that the shutter trigger condition is met. After entering the image preview interface, the steps of analyzing each preview frame frame by frame to obtain the reference frame based on the preview frames captured during the image preview process include: After entering the image preview interface, each preview frame acquired during the image preview process is analyzed frame by frame based on preset image metrics to obtain the analysis results of each preview frame. Preview frames whose analysis results meet the preset criteria for selecting reference frames are identified as the reference frames. The step of determining the preview frame whose analysis results meet the preset reference frame selection criteria as the reference frame includes: Determine the shutter trigger time; The preview frame with the smallest difference between the preview display time and the shutter trigger time, and whose analysis result meets the preset reference frame selection criteria, is determined as the reference frame.
2. The image processing method according to claim 1, characterized in that, The image metrics include at least one or more of the following parameters: Clarity, dynamic range, and imaging effect.
3. The image processing method according to claim 1, characterized in that, The step of fusing multiple acquired image frames in parallel to the reference frame, and fusing preview frames whose preview display time is before the reference frame and whose analysis results meet the preset reference frame selection criteria into the reference frame to obtain the output image, includes: Based on the aforementioned reference frame, image fusion processing is performed on the captured image frames in real time. After the last frame of the image obtained during the shooting process is fused, the output image is obtained.
4. An image processing apparatus, characterized in that, include: The reference frame determination module is used to analyze each preview frame one by one to obtain the reference frame after entering the image preview interface, based on the preview frames collected during the image preview process. The image fusion module is used to respond to a shooting command, and while acquiring multiple image frames, it simultaneously fuses the acquired multiple image frames to the reference frame. Preview frames whose preview display time is before the reference frame and whose analysis results meet the preset reference frame selection conditions are fused into the reference frame to obtain an output image. The shooting command is obtained based on the user touching the shutter button on the screen or pressing a physical button, or it is automatically generated based on the judgment that the shutter trigger condition is met. The reference frame determination module includes: The image analysis submodule is used to analyze each preview frame acquired during the image preview process based on preset image indicators after entering the image preview interface, and obtain the analysis results of each preview frame. The reference frame selection submodule is used to determine the preview frames whose analysis results meet the preset reference frame selection conditions as the reference frames; The reference frame selection submodule includes: The shutter detection submodule is used to determine the shutter trigger time; The frame selection submodule is used to determine the preview frame with the smallest difference between the preview display time and the shutter trigger time, and whose analysis result meets the preset benchmark frame selection conditions, as the benchmark frame.
5. The image processing apparatus according to claim 4, characterized in that, The image metrics include at least one or more of the following parameters: Clarity, dynamic range, and imaging effect.
6. The image processing apparatus according to claim 4, characterized in that, The image fusion module includes: The real-time fusion submodule is used to perform image fusion processing on the captured image frames in real time based on the reference frame; The output submodule is used to obtain the output image after the last image frame obtained during the shooting process has been fused.
7. A computer device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: After entering the image preview interface, each preview frame is analyzed frame by frame based on the preview frames captured during the image preview process to obtain the baseline frame; In response to a shooting command, while acquiring multiple image frames, the acquired multiple image frames are fused to the reference frame in parallel. Preview frames whose preview display time is before the reference frame and whose analysis results meet the preset reference frame selection conditions are fused into the reference frame to obtain an output image. The shooting command is obtained based on the user touching the shutter button on the screen or pressing a physical button, or it is automatically generated based on the judgment that the shutter trigger condition is met. After entering the image preview interface, the steps of analyzing each preview frame frame by frame to obtain the reference frame based on the preview frames captured during the image preview process include: After entering the image preview interface, each preview frame acquired during the image preview process is analyzed frame by frame based on preset image metrics to obtain the analysis results of each preview frame. Preview frames whose analysis results meet the preset criteria for selecting reference frames are identified as the reference frames. The step of determining the preview frame whose analysis results meet the preset reference frame selection criteria as the reference frame includes: Determine the shutter trigger time; The preview frame with the smallest difference between the preview display time and the shutter trigger time, and whose analysis result meets the preset reference frame selection criteria, is determined as the reference frame.
8. A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of a mobile terminal, enable the mobile terminal to perform an image processing method, the method comprising: After entering the image preview interface, each preview frame is analyzed frame by frame based on the preview frames captured during the image preview process to obtain the baseline frame; In response to a shooting command, while acquiring multiple image frames, the acquired multiple image frames are fused to the reference frame in parallel. Preview frames whose preview display time is before the reference frame and whose analysis results meet the preset reference frame selection conditions are fused into the reference frame to obtain an output image. The shooting command is obtained based on the user touching the shutter button on the screen or pressing a physical button, or it is automatically generated based on the judgment that the shutter trigger condition is met. After entering the image preview interface, the steps of analyzing each preview frame frame by frame to obtain the reference frame based on the preview frames captured during the image preview process include: After entering the image preview interface, each preview frame acquired during the image preview process is analyzed frame by frame based on preset image metrics to obtain the analysis results of each preview frame. Preview frames whose analysis results meet the preset criteria for selecting reference frames are identified as the reference frames. The step of determining the preview frame whose analysis results meet the preset reference frame selection criteria as the reference frame includes: Determine the shutter trigger time; The preview frame with the smallest difference between the preview display time and the shutter trigger time, and whose analysis result meets the preset reference frame selection criteria, is determined as the reference frame.
Citation Information
Patent Citations
Control method and device and electronic device
CN105657289A
Shooting method and mobile terminal
CN107222669A
Interactive shooting method, mobile terminal and storage medium
CN110072061A
Image processing method and device, computer readable medium and electronic equipment
CN111131698A
