Image processing method, photographing display method, image processor and electronic equipment

Abstract

The invention discloses an image processing method, a photographing display method, an image processor and electronic equipment, and the image processing method comprises the steps: obtaining first image data with an initial resolution in real time; in the first mode, executing a first processing process on each frame of first image data to form second image data, and displaying the second image data; in response to the target triggering operation, switching to a second mode, and obtaining target first image data in the multiple frames of first image data; executing a second processing process on the target first image data to form third image data, and storing the third image data; wherein the resolution of the image data of the first processing process is smaller than that of the image data of the second processing process; the resolution of the third image data is greater than the resolution of the second image data.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, a photographic display method, an image processor, and an electronic device. Background Technology

[0002] Many electronic devices nowadays can take photos via camera applications. After launching the camera application, the camera module of the electronic device can output image data in real time. At the same time, the processor of the electronic device processes the output image data and displays the corresponding image on the screen of the electronic device in real time. Thus, the user can see the scene captured by the camera module in real time on the screen and then press the shutter at the appropriate time to complete the photo.

[0003] From the moment the camera module starts outputting image data in real time until the user presses the shutter button, the electronic device's processor needs to process each frame of image data output by the camera module in real time, increasing the power consumption of the electronic device. Summary of the Invention

[0004] Therefore, this application discloses the following technical solution:

[0005] The first aspect of this application provides an image processing method, comprising:

[0006] Acquire first image data with initial resolution in real time;

[0007] In the first mode, a first processing procedure is performed on each frame of the first image data to form second image data, which is used for display.

[0008] In response to a target triggering operation, switch to the second mode to obtain the target first image data in multiple frames of the first image data;

[0009] A second processing procedure is performed on the target first image data to form third image data, which is used for storage;

[0010] The resolution of the image data in the first processing step is lower than that in the second processing step; the resolution of the third image data is higher than that in the second image data.

[0011] Optionally, a first processing procedure is performed on the first image data, including:

[0012] A down-resolution process is performed on the first image data to obtain adjusted image data with a second resolution, which is smaller than the initial resolution;

[0013] An image enhancement process is performed on the adjusted image data to obtain second image data, the resolution of which is less than or equal to the second resolution.

[0014] Optionally, the step of performing a down-resolution process on the first image data to obtain adjusted image data with a second resolution includes:

[0015] Perform a resolution down-resolution process on the first image data to obtain image data with the first resolution;

[0016] A format conversion process is performed on the image data with the first resolution to obtain converted image data, wherein the data format of the converted image data is different from the data format of the first image data.

[0017] A second down-resolution process is performed on the converted image data to obtain adjusted image data with a second resolution.

[0018] Optionally, the resolution of the second image data is determined according to the resolution of the display screen;

[0019] The first resolution and the second resolution are determined based on the processing capabilities of the first processing module and the second processing module, as well as the initial resolution and the resolution of the second image data;

[0020] The first processing module is used to perform the resolution downscaling process, and the second processing module is used to perform the image enhancement process.

[0021] Optional, also includes:

[0022] In the first mode, the operating frequency of the image processing module is set to a first frequency, which is matched with the resolution of the second image data;

[0023] In the second mode, the operating frequency of the image processing module is set to a second frequency, which matches the resolution of the third image data, and the first frequency is less than the second frequency;

[0024] The image processing module is used to perform a first processing procedure in a first mode and a second processing procedure in a second mode.

[0025] Optionally, in the first mode, a first processing procedure is performed on each frame of the first image data to form second image data, including:

[0026] In the first mode, if the remaining power of the electronic device meets the first condition, the first processing procedure is performed on each frame of the first image data to form the second image data;

[0027] Also includes:

[0028] In the first mode, if the remaining battery power of the electronic device meets the second condition, the second processing procedure is performed on each frame of the first image data to form the fourth image data, which is used for display. The resolution of the fourth image data is the same as that of the third image data.

[0029] A second aspect of this application provides an image processor, including an image processing module and a resolution configuration module;

[0030] The image processing module is used for:

[0031] Acquire first image data with initial resolution in real time;

[0032] In the first mode, a first processing procedure is performed on each frame of the first image data to form second image data, which is used for display.

[0033] In response to a target triggering operation, switch to the second mode to obtain the target first image data in multiple frames of the first image data;

[0034] A second processing procedure is performed on the target first image data to form third image data, which is used for storage;

[0035] In this process, the resolution of the image data in the first processing step is lower than that of the image data in the second processing step; the resolution of the third image data is higher than that of the second image data.

[0036] The resolution configuration module is used for:

[0037] In the first mode, the resolution of the image data for which the image processing module performs the first processing procedure is configured;

[0038] In the second mode, the resolution of the image data for which the image processing module performs the second processing procedure is configured.

[0039] Optionally, the image processing module includes a first processing module for reducing resolution and a second processing module for image enhancement;

[0040] In the first mode, the resolution configuration module is used for:

[0041] Configure the resolution of the second image data according to the resolution of the display screen;

[0042] Based on the processing capabilities of the first and second processing modules, as well as the initial resolution and the resolution of the second image data, the resolution of the image data during the resolution reduction process of the first processing module is configured.

[0043] A third aspect of this application provides an electronic device, including a camera module, an image processor, a display module, and a memory;

[0044] The camera module is used to receive image signals in real time and output image data;

[0045] The image processor is used for:

[0046] The camera module acquires first image data with an initial resolution in real time.

[0047] In the first mode, a first processing procedure is performed on each frame of the first image data to form second image data, and the second image data is output to the display module;

[0048] In response to a target triggering operation, switch to the second mode to obtain the target first image data in multiple frames of the first image data;

[0049] A second processing procedure is performed on the target first image data to form third image data, and the third image data is output to the memory;

[0050] In this process, the resolution of the image data in the first processing step is lower than that of the image data in the second processing step; the resolution of the third image data is higher than that of the second image data.

[0051] The display module is used to receive and display the second image data;

[0052] The memory is used to store the third image data in the second mode.

[0053] The fourth aspect of this application provides a method for displaying photographed images, including:

[0054] In response to the electronic device being in a first mode, second image data is displayed in real time on the screen of the electronic device, the second image data being obtained by processing the first image data acquired in real time by the electronic device;

[0055] In response to a target trigger operation, the system switches to the second mode, acquires and stores third image data, which is obtained by processing a frame of first image data acquired by the electronic device.

[0056] The resolution of the third image data is greater than that of the second image data. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0058] Figure 1 This is a flowchart of an image processing method provided in an embodiment of this application;

[0059] Figure 2 This is a schematic diagram illustrating the working principle of an image processing module performing a first processing step, as provided in an embodiment of this application.

[0060] Figure 3 This is a schematic diagram illustrating the working principle of an image processing module performing a second processing step, as provided in an embodiment of this application.

[0061] Figure 4 This is a schematic diagram of the structure of an image processing module provided in an embodiment of this application;

[0062] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0063] Figure 6 This is a flowchart of a photograph display method provided in an embodiment of this application;

[0064] Figure 7 This is a schematic diagram of an interface for a photo display method provided in an embodiment of this application. Detailed Implementation

[0065] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0066] This application provides an image processing method. Please refer to [link to relevant documentation]. Figure 1 The method may include the following steps.

[0067] The image processing method of this embodiment can be executed by any electronic device with a camera module. As examples, the execution subject can be an electronic device such as a mobile phone, tablet computer, or laptop computer. Having a camera module here can include the camera module itself integrated into the electronic device, or it can include the camera module being a separate device that communicates with the electronic device, enabling the electronic device to use the camera module to take photos.

[0068] S101, acquires first image data with initial resolution in real time.

[0069] S102, in the first mode, a first processing procedure is performed on each frame of first image data to form second image data, which is used for display.

[0070] S103, in response to the target triggering operation, switch to the second mode and obtain the target first image data in the multi-frame first image data.

[0071] S104, perform a second processing procedure on the target first image data to form third image data. The third image data is used for storage. The resolution of the image data in the first processing procedure is less than the resolution of the image data in the second processing procedure, and the resolution of the third image data is greater than the resolution of the second image data.

[0072] The beneficial effect of this embodiment is that, in the first mode where only the second image data needs to be acquired and displayed, the electronic device processes the first image data with a first processing procedure that has a lower resolution than the image data during the processing, so as to form the second image data with a lower resolution. In this way, the amount of data that needs to be processed when processing each frame of the first image data can be reduced, thereby reducing the processing resources occupied in the first mode and reducing the power consumption of the electronic device.

[0073] On the other hand, when it is necessary to obtain third image data to be saved as a photo, the electronic device switches to the second mode. In the second mode, the target first image data is processed by a second processing procedure with higher resolution image data during the processing to form high-resolution third image data. This can save high-resolution third image data, meet the user's need to obtain high-quality photos, and since the second processing procedure only needs to process specific target first image data in multiple frames of first image data, without continuously processing each frame of first image data, obtaining third image data by the second processing procedure will not significantly increase power consumption and processing resources occupied.

[0074] In summary, the method of this embodiment can reduce the processing resources and power consumption of the image data processing process in the process of electronic devices acquiring and storing third image data, while ensuring that the user's demand for high-quality images is met and that the highest possible resolution third image data is saved.

[0075] In contrast, in related technologies, electronic devices typically acquire first image data in real time and then process each frame of the first image data in real time using a high-resolution processing procedure supported by the electronic device, thereby forming each frame of high-resolution image data in real time. Obviously, the image processing method of this embodiment can reduce the amount of data that needs to be processed in the process of processing each frame of the first image data in real time, thereby achieving the above-mentioned beneficial effects.

[0076] In step S101, the first image data with initial resolution can be acquired in real time by the camera module of the electronic device. Generally, the electronic device can activate the camera module when the camera application is launched in response to a user's operation. After the camera module is activated, it acquires image signals in real time, that is, it acquires the light signals incident on the camera module, and periodically outputs the image data formed by converting the image signals to the processor located at the back end of the camera module in the electronic device. Each frame of image data output by the camera module is a frame of first image data with initial resolution. The initial resolution is determined by the image sensor of the camera module.

[0077] The frame rate (representing the time interval for obtaining the first image data) and the initial resolution of the first image data are not limited. As some examples, the first image data in this embodiment can be 16M@30fps image data, that is, the initial resolution of the first image data is 4656*3496 pixels, the frame rate is 30 frames per second, and 30 frames of first image data can be obtained in one second.

[0078] The first mode in S102 can be the mode in which the camera module outputs the first image data in real time after the camera application is launched, and the electronic device does not detect the target trigger operation. In some optional application scenarios, the first mode can also be called the preview mode or the photo preview mode.

[0079] A target-triggered operation is an operation used to trigger an electronic device to take and save a photo. Specifically, it can be the user triggering the shutter of the electronic device. The shutter of the electronic device can be a physical shutter button, and pressing the shutter button is equivalent to a target-triggered operation. Alternatively, the shutter of the electronic device can be a virtual shutter control displayed in the human-computer interaction interface, and clicking the virtual shutter control is equivalent to a target-triggered operation.

[0080] In the first mode, each time the processor of the electronic device obtains a frame of first image data, it performs a first processing procedure on the frame of first image data to obtain a frame of second image data corresponding to the frame of first image data. At the same time, each time a frame of second image data is obtained, the processor can transmit the frame of second image data to the display module of the electronic device, so that the display module displays an image based on the frame of second image data.

[0081] The resolution of the second image data is lower than the initial resolution of the first image data, and the frame rate of the second image data can be the same as that of the first image data. For example, the second image data can be 5M@30fps image data, that is, the resolution of the second image data is 2500*1876 pixels, and the frame rate is 30 frames per second.

[0082] In step S103, the electronic device may switch to the second mode in response to receiving a user's target trigger operation while displaying the second image data in real time.

[0083] The way to obtain the first image data of the target in response to a target trigger operation can be:

[0084] The moment when the target trigger operation is detected is determined as the trigger moment. The moment before the trigger moment, which is a preset delay time away from the trigger moment, is obtained as the target moment. The first image data obtained from the camera module at the target moment is used as the target first image data.

[0085] There is a delay in the hardware and software of electronic devices when responding to the user's target trigger operation. Therefore, the time when the electronic device detects the target trigger operation is obviously later than the time when the user actually performs the target trigger operation. The image seen on the screen at the moment when the user actually performs the target trigger operation is the image that the user actually needs to save. Furthermore, there is a certain delay in the process of processing the first image data. Therefore, the image seen on the screen at the moment when the user actually performs the target trigger operation corresponds to the first image data obtained by the camera module some time ago.

[0086] Therefore, in order to ensure that the image content of the third image data saved in response to the target trigger operation is as consistent as possible with the image content seen by the user when the target trigger operation is actually performed, a delay duration that can just cover the delays mentioned above can be pre-configured in the electronic device to address the delays in both aspects: the delay in hardware and software when responding to the user's target trigger operation and the delay in processing the first image data. At the trigger moment when the target trigger operation is detected (i.e., responding to the target trigger operation), the first image data at the target time interval before the trigger moment is obtained as the image that the user actually needs to save, so that the target first image data can be processed and saved.

[0087] The specific value of the delay duration can be determined based on the processing performance of the electronic device's hardware and software. For specific configuration methods, please refer to relevant technologies, which will not be elaborated here.

[0088] As explained above, the first image data is saved when the first processing is executed. Therefore, the first image data of the target can also be obtained by responding to the target trigger operation and using the most recently saved frame of the first image data as the target first image data.

[0089] In order to obtain the target's first image data after responding to a target trigger operation, the electronic device can save each frame of first image data in its own storage space after obtaining it. Thus, when responding to a target trigger operation, the target's first image data can be retrieved from the storage space based on the aforementioned delay duration. This storage space can be the electronic device's RAM or its hard drive.

[0090] Furthermore, in order to save its own storage space, the electronic device can be pre-configured with a storage duration, saving only the first image data obtained within the storage duration prior to the current moment, and deleting the first image data whose duration exceeds the storage duration prior to the current moment.

[0091] Optionally, the strategy for deleting the saved first image data can also be based on the amount of stored data, that is, when the total amount of stored first image data reaches a certain threshold, the earliest stored first image data is deleted; or, it can be based on the number of images, that is, when the number of frames of stored first image data reaches a certain threshold, the earliest stored first image data is deleted.

[0092] The storage duration can be set based on the aforementioned delay duration, for example, it can be equal to or slightly greater than the delay duration, and the specific value is not limited.

[0093] As an example, the storage duration can be set to 1 second. Based on this, the electronic device only saves the first image data obtained within the most recent 1 second. First image data obtained more than 1 second ago, such as the first image data obtained 1.1 seconds ago, and the first image data obtained earlier, are deleted. This ensures that the target first image data can be found from the saved first image data after responding to the target trigger operation, while also saving the storage space occupied by storing the first image data.

[0094] Generally, each time a target trigger operation is responded to, only one frame of the target's first image data is determined. Furthermore, after each response to a target trigger operation and the acquisition of the corresponding frame of the target's first image data, the electronic device can delete the first image data acquired before the target's first image data to save storage space.

[0095] In step S104, the electronic device processes the target first image data based on the second processing procedure to form third image data.

[0096] After the third image data is generated, the electronic device can save it as a frame of photograph on its own disk or other storage space. Furthermore, the electronic device can also display a thumbnail of the third image data on the display interface currently used to display the second image data, to indicate to the user that a frame of third image data has been saved.

[0097] After acquiring the third image data, the electronic device can exit the second mode, but will not immediately execute the first processing procedure. Instead, it will display the third image data on the screen. After receiving the operation to trigger continued shooting, the electronic device will switch back to the first mode and continue executing the first processing procedure to generate the second image data in real time. During the display of the third image data, the camera module of the electronic device may or may not be working. If the camera module remains working, the acquired first image data can be discarded directly.

[0098] Alternatively, after obtaining the third image data, if the electronic device does not receive the operation to close the camera application or close the camera module, the electronic device can exit the second mode and directly switch back to the first mode, and continue to execute the first processing procedure in accordance with S101 and S102 to form the second image data in real time.

[0099] The resolution of the third image data is greater than that of the second image data. The resolution of the third image data can be equal to or less than the initial resolution, depending on the user's configuration. For example, when the user configures to obtain high-quality photos, the resolution of the resulting third image data can be equal to the initial resolution. When the user configures to obtain low-quality photos with a small data size, the resolution of the third image data can be less than the initial resolution.

[0100] For a frame of first image data, intermediate image data is generated during both the execution of the first processing step and the execution of the second processing step. The resolution of the image data in the first processing step may include the resolution of the intermediate image data during the execution of the first processing step and the resolution of the second image data formed by the execution of the first processing step; the resolution of the image data in the second processing step may include the resolution of the intermediate image data during the execution of the second processing step and the resolution of the third image data formed by the execution of the second processing step.

[0101] Optionally, a first processing procedure is performed on the first image data, including:

[0102] A down-resolution process is performed on the first image data to obtain adjusted image data with a second resolution, which is smaller than the initial resolution;

[0103] An image enhancement process is performed on the adjusted image data to obtain second image data, the resolution of which is less than or equal to the second resolution.

[0104] In this embodiment, the first processing procedure can be divided into a resolution reduction process and an image enhancement process executed sequentially. The resolution reduction process is mainly used to reduce the resolution of the first image data to obtain adjusted image data with a resolution lower than the initial resolution. The image enhancement process is mainly used to perform image enhancement processing on the adjusted image data formed after the resolution reduction process, so that the obtained second image data better meets the user's visual needs.

[0105] During image enhancement, any one or more image enhancement processes can be performed on the adjusted image data. These can include color correction matrix processing, gamma correction processing, color space conversion processing, sharpening processing, noise reduction processing, contrast enhancement processing, saturation enhancement processing, image smoothing processing, etc. The color correction matrix is ​​used to accurately adjust the color of the image to restore color accuracy, and the gamma correction processing is used to make non-linear adjustments to the image brightness to adapt to the display characteristics of the electronic device's screen. For the specific processing methods of the above image enhancement processes, please refer to the relevant technologies, which will not be elaborated here.

[0106] Optionally, the resolution reduction process may be omitted during the image enhancement process. In this case, the resolution of the final second image data may be equal to the second resolution.

[0107] The frame rate of the adjusted image data can be the same as that of the first image data. The second resolution of the adjusted image data can be pre-configured according to the performance of the electronic device's processor; there are no specific limitations, as long as it is lower than the initial resolution. As some examples, the adjusted image data can be 5M@30fps image data, that is, the resolution of the adjusted image data is 2500*1876 pixels, and the frame rate is 30 frames per second.

[0108] Performing the down-resolution process first, followed by the image enhancement process, helps reduce the amount of image data that needs to be processed during the image enhancement process, thereby reducing the power consumption of electronic devices during the first processing step.

[0109] Optionally, during the image enhancement process, or after the image enhancement process but before forming the second image data, a resolution reduction process can be performed on the adjusted image data to make the resolution of the second image data smaller than the second resolution of the adjusted image data, thereby minimizing the resolution of the second image data.

[0110] After the second image data is generated, it is displayed on the screen. The processor of the electronic device also needs to perform corresponding image rendering processing based on the second image data. The smaller the resolution of the second image data, the less data needs to be processed during image rendering. Therefore, reducing the resolution of the second image data during or after the image enhancement process can help reduce the power consumption and processing resource consumption of the electronic device.

[0111] Optionally, a down-resolution process is performed on the first image data to obtain adjusted image data with a second resolution, including:

[0112] Perform a resolution down-resolution process on the first image data to obtain image data with the first resolution;

[0113] A format conversion process is performed on the image data with a first resolution to obtain converted image data. The data format of the converted image data is different from that of the first image data.

[0114] A second down-resolution process is performed on the converted image data to obtain adjusted image data with a second resolution.

[0115] Electronic devices typically perform a first processing procedure through a first processing module and a second processing module. The first processing module is used to perform a resolution reduction process, and the second processing module is used to perform an image enhancement process.

[0116] Both the first processing module and the second processing module can be regarded as sub-modules of the image processing module. That is, the image processing module includes the first processing module and the second processing module. The image processing module is equivalent to a software module run by the image signal processor or image signal processor (ISP) of the electronic device. The first processing module is mainly used for front-end resolution reduction and format conversion processing, so it can be regarded as the front-end processing module, or ISP_FE. The second processing module is mainly used for back-end enhancement processing, so it can be regarded as the back-end processing module, or ISP_BE.

[0117] The image data obtained after performing a down-resolution process on the first image data can have the same frame rate as the first image data, and the first resolution can be less than the initial resolution but greater than the second resolution. In some examples, the image data obtained after performing a down-resolution process can be 7M@30fps image data, that is, image data with a frame rate of 30 frames per second and a first resolution of 3104*2332 pixels.

[0118] The beneficial effects of this embodiment are as follows:

[0119] On the one hand, before performing the format conversion process, the first image data is first down-resolutiond to obtain image data with the first resolution. This can reduce the amount of image data to be processed during the format conversion process, thereby shortening the processing time and reducing the power consumption of the format conversion process.

[0120] On the other hand, each time the first processing module obtains a frame of adjusted image data, it can write the adjusted image data into the storage space pre-allocated by the electronic device. Then, the second processing module can read the adjusted image data from the storage space to perform the subsequent image enhancement process. After performing the format conversion process, the second resolution reduction process is performed, which helps to reduce the amount of adjusted image data written into the storage space and saves the storage space occupied by the first processing module when transmitting the adjusted image data to the second processing module.

[0121] The purpose of executing the resolution reduction process in two steps is that the image processing module has certain performance limitations when reducing resolution. The amount of resolution reduction that can be achieved in each processing step is limited. If the resolution reduction is too large at once, it may lead to serious image distortion. Therefore, by executing the resolution reduction process once and then twice, the second resolution of the adjusted image data can be minimized within the limitation of the resolution reduction range, thereby improving the power consumption reduction effect of the image processing method in this embodiment.

[0122] The image data with the first resolution and the adjusted image data obtained by performing the down-resolution process can both be regarded as intermediate image data for performing the first processing process.

[0123] The following is combined with Figure 2 and Figure 3 The schematic diagram of the image processing module shown illustrates the process by which the electronic device executes the image processing method of this embodiment based on the image processing module.

[0124] The image processing module may include Figure 2 The preprocessing module 201 (also known as the Pre_ISP module), the first processing module 202, and the second processing module 203 are mentioned.

[0125] After the camera module starts outputting the first image data in real time, the preprocessing module 201 receives the first image data with an initial resolution based on the Mobile Industry Processor Interface (MIPI), and then transmits the first image data to the first processing module 202 through the Camera Serial Interface (CSI) and the Video Interface (VI).

[0126] The first processing module 202 includes a correction module 221, a primary processing module (raw_scale) 222, a format conversion module 223, and a secondary processing module (MP_scale) 224, wherein the correction module 221 is used to perform a microlens color mixing (MCM) process on the input image data.

[0127] The first image data with an initial resolution of 4656*3496 output by the preprocessing module 201 first enters the correction module 221. The correction module 221 writes the first image data into the storage space pre-allocated by the electronic device for storage, and performs a microlens color mixing correction process on the first image data to form corrected image data and transmits it to the primary processing module 222.

[0128] The primary processing module 222 performs a resolution downscaling process on the input image data with an initial resolution of 4656*3496 after correction, to obtain image data with a first resolution of 3104*2332, and outputs the image data with the first resolution to the format conversion module 223.

[0129] The image data input to the format conversion module 223 has the same data format as the first image data, which is the original sensor data format output by the camera module's sensor (generally referred to as RAW format). This image data needs to be converted to a common image data format before it can be used for subsequent processing. The format conversion module 223 can convert the input image data in the original sensor data format to RGB or YUV format and output RGB or YUV format, which also has the same converted image data with the first resolution.

[0130] The format conversion module outputs the converted image data to the secondary processing module 224. The secondary processing module 224 performs a second resolution reduction process on the converted image data to form adjusted image data with a second resolution of 2500*1876. At this point, the resolution reduction process performed by the first processing module ends.

[0131] It can be seen that in the first mode, the resolution of the image data obtained by the format conversion module 223 and the secondary processing module 224 in the first processing module is a first resolution that is lower than the initial resolution. Compared with directly processing the image data at the initial resolution, power consumption can be effectively reduced.

[0132] Each time the first processing module 202 obtains a frame of adjusted image data, it writes the adjusted image data into a pre-allocated storage space, so that the second processing module 203 can read the adjusted image data from the storage space.

[0133] The second processing module 203 may include an enhancement module 231 and a scaling module 232. The read adjusted image data is first input to the enhancement module 231. The enhancement module 231 can perform any one or more of the aforementioned image enhancement processes on the input image data and output the enhanced image data to the scaling module 232. The scaling module 232 can perform a resolution reduction process on the image data output by the enhancement module 231 to form second image data. In this case, the resolution of the second image data is less than the second resolution. Alternatively, the image data output by the enhancement module 231 can be directly output as the second image data. In this case, the resolution of the second image data is equal to the second resolution.

[0134] Please see Figure 3 After switching to the second mode in response to a target trigger operation, the primary processing module 222 can be deactivated, and then re-enabled when switching back to the first mode to perform a resolution downscaling process.

[0135] The correction module 221 reads the target first image data from the previously stored multi-frame first image data, performs a microlens color mixing correction process on the target first image data, and outputs the corrected target first image data. Since the primary processing module 222 is disabled, the corrected target first image data is directly input into the format conversion module 223. The format conversion module 223 converts the format of the corrected target first image data to obtain the converted target image data and outputs it to the secondary processing module 224. In the second mode, the secondary processing module 224 does not perform a secondary resolution reduction process, but directly outputs the converted target image data to the second processing module 203 by writing it into the storage space. In the second processing module 203, the enhancement module 231 performs image enhancement processing on the converted target image data to obtain the enhanced target image data and outputs it to the scaling module 232. The scaling module 232 directly outputs the enhanced target image data as the third image data. At this point, the second processing process ends.

[0136] It can be seen that no resolution reduction operation is performed in the entire second processing process. The first processing module 202 only performs the format conversion process. Therefore, the image data processed in each step and the final third image data have the initial resolution of the first image data, 4656*3496.

[0137] It should be noted that the first and second processing modules can also obtain the second and third image data through time-division multiplexing. Figure 3 After each submodule within the first and second processing modules has executed the corresponding steps in the second processing procedure according to the second mode, it can immediately switch back to the previous mode. Figure 2The first mode shown continues to process the real-time acquired first image data in the manner shown in the first mode. In this way, after outputting a frame of third image data, the first processing module and the second processing module can immediately output the subsequent second image data in real time, thus presenting the effect of simultaneously saving the third image data and displaying the second image data.

[0138] For example, after the correction module 221 performs the microlens color mixing correction process on the target first image data, it can immediately switch back to the working mode of the first mode, write the real-time obtained first image data into the storage space and perform the microlens color mixing correction process. In this way, while the format conversion module 223 performs format conversion on the corrected target first image data, the correction module 221 is preparing the next frame of corrected image data. And while the format conversion module 223 performs format conversion on the corrected target first image data, the primary processing module 222 can switch from the disabled state in the second mode back to the enabled state in the first mode and continue to perform a resolution reduction process.

[0139] The target first image data can be determined by the correction module 221, or it can be determined by other modules and then notified to the correction module 221 to read it out.

[0140] It needs to be explained that, Figure 2 and Figure 3 The processing modules involved are the same set of processing modules. The first processing process and the second processing process in this embodiment are two processing processes implemented by reusing the same set of processing modules. Figure 2 and Figure 3 This is equivalent to a schematic diagram illustrating the principle of the same processing module executing different processing procedures.

[0141] Furthermore, the electronic device may also include a resolution configuration module 204, which is used to configure the first processing module and the second processing module to perform the first processing procedure or the second processing procedure in response to the electronic device being in a first mode or a second mode.

[0142] For example, when the electronic device is in the first mode, the resolution configuration module 204 configures the primary processing module 222 to enable, and configures the primary processing module 222 to reduce the resolution of the input image data to the first resolution, and configures the secondary processing module 224 to reduce the resolution of the input image data to the second resolution, so that the first processing module can perform the resolution reduction process to obtain the second image data.

[0143] When the electronic device is in the second mode, the resolution configuration module 204 configures the primary processing module 222 to be disabled, and configures the secondary processing module 224 not to perform a secondary resolution reduction process on the input image data, so that the primary processing module 202 only performs the format conversion process, thereby obtaining the third image data.

[0144] If the scaling module 232 is used to perform the resolution reduction process in the first mode, the resolution configuration module 204 can also respond to the first mode by configuring the scaling module 232 to reduce the resolution of the input image data to the resolution of the specified second image data, and respond to the second mode by configuring the scaling module 232 not to reduce the resolution of the input image data.

[0145] Optionally, the resolution of the second image data is determined based on the resolution of the display screen;

[0146] The first resolution and the second resolution are determined based on the processing capabilities of the first processing module and the second processing module, as well as the initial resolution and the resolution of the second image data.

[0147] The resolution of the second image data (denoted as the preview resolution) can be the same as or close to the resolution of the electronic device's display screen.

[0148] When the preview resolution is lower than the initial resolution, the first and second resolutions can be determined as follows:

[0149] If the scaling capability, or resolution reduction capability, of the primary processing module 222 and the secondary processing module 224 is sufficient to reduce the image data at the initial resolution to the preview resolution, and the primary processing module 222 cannot directly reduce the image data to the preview resolution, then the second resolution can be configured to be equal to the preview resolution. Based on the scaling capability of the primary processing module 222 and the initial resolution, the minimum resolution that the primary processing module 222 can output is determined as the first resolution.

[0150] If the scaling capabilities of the primary processing module 222 and the secondary processing module 224 are insufficient to reduce the image data at the initial resolution to the preview resolution, then based on the scaling capability of the primary processing module 222 and the initial resolution, the minimum resolution that the primary processing module 222 can output is determined as the first resolution. Then, based on the first resolution and the scaling capability of the secondary processing module 224, the minimum resolution that the secondary processing module 224 can output is determined as the second resolution. The process of reducing the second resolution to the preview resolution is then performed by the scaling module 232.

[0151] The above preview resolution, first resolution, and second resolution can all be determined by the aforementioned resolution configuration module 204.

[0152] In some optional embodiments, if the display resolution is large, the preview resolution may be greater than or equal to the initial resolution. In this case, regardless of whether it is in the first mode or the second mode, the primary processing module 222 can be configured to be disabled, and the secondary processing module can be configured not to perform the secondary resolution reduction process, so that the primary processing module 202 outputs image data with the initial resolution regardless of whether it is in the first mode or the second mode. If the preview resolution is equal to the initial resolution, the scaling module 232 can be configured to output the input image data directly as the second image data or the third image data. If the preview resolution is greater than the initial resolution, the scaling module 232 can be configured to perform a resolution scaling process on the input image data to input the second image data or the third image data with the preview resolution.

[0153] Optionally, the method in this embodiment may further include:

[0154] In the first mode, the operating frequency of the image processing module is set to a first frequency, which is matched with the resolution of the second image data;

[0155] In the second mode, the operating frequency of the image processing module is set to the second frequency, which matches the resolution of the third image data, and the first frequency is lower than the second frequency;

[0156] The image processing module is used to perform a first processing procedure in the first mode and a second processing procedure in the second mode.

[0157] The resolution matching of the first frequency and the second image data means that when the image processing module performs the first processing procedure on a frame of first image data to form the second image data at the first frequency, the time taken is equal to or slightly less than the time interval for obtaining the first image data.

[0158] For example, if the first image data is obtained in real time at a frame rate of 30 frames per second, it means that the camera module provides one frame of the first image data to the image processing module every 1 / 30th of a second. Then, the frequency at which the image processing module performs the first processing on one frame of the first image data takes 1 / 20th of a second can be determined, and this frequency is determined as the first frequency.

[0159] The resolution matching between the second frequency and the third image data means that, when operating at the second frequency, the time taken by the image processing module to perform the second processing procedure on a frame of first image data to form the third image data is equal to or slightly less than the time interval for obtaining the first image data. The specific method for determining the second frequency can be found in the first frequency description, and will not be repeated here.

[0160] It can be seen that the amount of data that the image processing module needs to process when executing the second processing step is larger than the amount of data when executing the first processing step, while the requirement for execution time remains basically unchanged. Therefore, when executing the second processing step in the second mode, the operating frequency of the image processing module should be set to a higher second frequency, while when executing the first processing step in the first mode, the operating frequency of the image processing module can be set to a lower first frequency.

[0161] By setting the operating frequency of the image processing module according to the different modes as described above, it is possible to reduce the power consumption of electronic devices in the first mode, and to obtain third image data in a timely manner by increasing the operating frequency in the second mode.

[0162] The operating frequency of the image processing module can be set by the aforementioned resolution configuration module 204, or by other processors in the electronic device besides the image processing module, such as the central processing unit (CPU) or the system-on-a-chip (SoC).

[0163] Optionally, in the first mode, a first processing procedure is performed on each frame of first image data to form second image data, including:

[0164] In the first mode, if the remaining power of the electronic device meets the first condition, the first processing procedure is performed on each frame of the first image data to form the second image data;

[0165] Also includes:

[0166] In the first mode, if the remaining power of the electronic device meets the second condition, the second processing procedure is performed on each frame of the first image data to form the fourth image data. The fourth image data is used for display, and the resolution of the fourth image data is the same as that of the third image data.

[0167] The fourth image data serves the same purpose as the second image data, namely, to be displayed in real time on the screen of the electronic device.

[0168] The method of performing the second processing procedure on the first image data to form the fourth image data is the same as the method of performing the second processing procedure on the target first image data to form the third image data, and will not be described in detail.

[0169] When the first condition is met, the remaining battery power of the electronic device is less than when the second condition is met. For example, meeting the first condition can be less than a first battery power threshold, and meeting the second condition can be greater than or equal to the first battery power threshold.

[0170] The advantages of applying this embodiment are:

[0171] Even when the remaining battery power is high, the electronic device obtains high-resolution fourth image data based on the second processing and displays the fourth image data on the screen in real time, allowing the user to preview a high-quality image, thus providing a good visual experience in the first mode.

[0172] When there is a lot of power left, if the device is in the first mode, the electronic device will obtain low-resolution second image data based on the first processing, thereby reducing the power consumption of the electronic device and preventing the remaining power from being quickly depleted and affecting the operation of the electronic device.

[0173] Therefore, by applying the above method, a good visual experience can be prioritized when there is a lot of battery power remaining, and power consumption can be reduced when there is little battery power remaining, thus taking into account the user's needs for both visual experience and extending the usage time of electronic devices to a certain extent.

[0174] This application also provides an image processor; please refer to [link to relevant documentation]. Figure 4 It includes an image processing module 300 and a resolution configuration module 304;

[0175] Image processing module 300 is used for:

[0176] Acquire first image data with initial resolution in real time;

[0177] In the first mode, a first processing procedure is performed on each frame of first image data to form second image data, which is used for display.

[0178] In response to the target trigger operation, switch to the second mode to obtain the target first image data in multiple frames of first image data;

[0179] A second processing procedure is performed on the first image data of the target to form third image data, which is used for storage.

[0180] In this process, the resolution of the image data in the first processing step is lower than that of the image data in the second processing step; the resolution of the third image data is higher than that of the second image data.

[0181] Resolution configuration module 304 is used for:

[0182] In the first mode, the resolution of the image data for which the image processing module performs the first processing step is configured;

[0183] In the second mode, the resolution of the image data is configured for the image processing module to perform the second processing procedure.

[0184] Optionally, the image processing module 300 may include a first processing module 302 for reducing resolution and a second processing module 303 for image enhancement, and may also include a preprocessing module 301 for transmitting first image data from the camera module to the first processing module;

[0185] In the first mode, the resolution configuration module 304 is used for:

[0186] Configure the resolution of the second image data according to the resolution of the display screen;

[0187] Based on the processing capabilities of the first processing module 302 and the second processing module 303, as well as the initial resolution and the resolution of the second image data, the resolution of the image data during the resolution reduction process of the first processing module is configured.

[0188] The specific working principle of the above image processor can be found in the relevant steps of the image processing method in the foregoing embodiments. Figure 2 The working principle of the image processing module is not elaborated here.

[0189] This application provides an electronic device, see [link to relevant documentation] Figure 5 It includes a camera module 401, an image signal processing (ISP) 402, a display module 403, and a memory 404;

[0190] The camera module 401 is used to receive image signals in real time and output image data;

[0191] Image processor 402, used for:

[0192] The camera module acquires first image data with initial resolution in real time.

[0193] In the first mode, a first processing procedure is performed on each frame of first image data to form second image data, and the second image data is output to the display module;

[0194] In response to the target trigger operation, switch to the second mode to obtain the target first image data in multiple frames of first image data;

[0195] The second processing procedure is performed on the first image data of the target to form the third image data, and the third image data is output to the memory;

[0196] In this process, the resolution of the image data in the first processing step is lower than that of the image data in the second processing step; the resolution of the third image data is higher than that of the second image data.

[0197] Display module 403 is used to receive and display the second image data;

[0198] Memory 404 is used to store third image data in the second mode.

[0199] For the specific working principle of the above electronic devices, please refer to the relevant steps of the image processing method in the foregoing embodiments. Figure 2 and Figure 3 The working principle of the image processing module is not elaborated here.

[0200] This application also provides a method for displaying photographed images. Please refer to [link to relevant documentation]. Figure 6 The method may include the following steps.

[0201] S501, in response to the electronic device being in the first mode, displays second image data on the display screen of the electronic device in real time. The second image data is obtained by processing the first image data acquired by the electronic device in real time.

[0202] S502, in response to the target trigger operation, switches to the second mode, acquires and stores the third image data, which is obtained by processing a frame of the first image data acquired by the electronic device, and the resolution of the third image data is greater than the resolution of the second image data.

[0203] The method for performing the first processing procedure to form the second image data and the method for performing the second processing procedure to form the third image data can be found in the relevant steps of the aforementioned image processing method, and will not be repeated here.

[0204] The following examples of the display interface illustrate the application scenarios of the above-mentioned photo display method.

[0205] like Figure 7 As shown in (1), the electronic device can respond to the user's operation of launching the camera application and display... Figure 7 The interface shown in (1) indicates that the electronic device is in the first mode, acquiring first image data in real time through the camera module and displaying second image data obtained by performing the first processing on the first image data in real time, such as displaying... Figure 7 The second image data shown in image 601 in (1) is a frame of image data.

[0206] like Figure 7 As shown in (2), after the camera application is launched, at a certain moment the user presses the shutter control 602. The electronic device then recognizes the pressing of the shutter control as a target trigger operation, responds to the target trigger operation, performs a second processing procedure on the first image data of the target, forms third image data, saves the third image data, and... Figure 7 The lower left corner of the interface shown in (2) displays a thumbnail corresponding to the third image data, namely thumbnail 703.

[0207] like Figure 7As shown in (3), after acquiring and saving the third image data, the electronic device responds to the operation of exiting the camera application and opening the album application, launches the album application, and responds to the user's operation of viewing the third image data in the album application, displaying as shown in (3). Figure 7 The interface shown in (3) displays the third image data shown in image 604.

[0208] By using the above-described image display method, electronic devices can display lower-resolution second image data during image preview to reduce the power consumption of the electronic device, and save high-resolution third image data after the user presses the shutter to meet the user's need for high-quality photos.

[0209] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0210] For ease of description, the above systems or devices are described separately as various modules or units based on their functions. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components.

[0211] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0212] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0213] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image processing method comprising: acquiring first image data with an initial resolution in real time; in a first mode, performing a first processing procedure on each frame of the first image data to form second image data, the second image data being used for display; in response to a target trigger operation, switching to a second mode, and obtaining target first image data from a plurality of frames of the first image data; performing a second processing procedure on the target first image data to form third image data, the third image data being used for storage; wherein a resolution of image data of the first processing procedure is less than a resolution of image data of the second processing procedure, and a resolution of the third image data is greater than a resolution of the second image data.

2. The method of claim 1, wherein performing the first processing procedure on the first image data comprises: performing a resolution reduction procedure on the first image data to obtain adjusted image data with a second resolution, the second resolution being less than the initial resolution; performing an image enhancement procedure on the adjusted image data to obtain the second image data, the second image data having a resolution less than or equal to the second resolution.

3. The method of claim 2, wherein performing the resolution reduction procedure on the first image data to obtain adjusted image data with a second resolution comprises: performing a first resolution reduction procedure on the first image data to obtain image data with a first resolution; performing a format conversion procedure on the image data with the first resolution to obtain converted image data, the converted image data having a different data format than a data format of the first image data; performing a second resolution reduction procedure on the converted image data to obtain the adjusted image data with the second resolution.

4. The method of claim 3, wherein the resolution of the second image data is determined according to a resolution of a display screen; the first resolution and the second resolution are determined according to processing capabilities of a first processing module and a second processing module, and the initial resolution and the resolution of the second image data; wherein the first processing module is configured to perform the resolution reduction procedure, and the second processing module is configured to perform the image enhancement procedure.

5. The method of claim 1, further comprising: setting a running frequency of an image processing module to a first frequency in the first mode, the first frequency matching the resolution of the second image data; setting the running frequency of the image processing module to a second frequency in the second mode, the second frequency matching the resolution of the third image data, the first frequency being less than the second frequency; wherein the image processing module is configured to perform the first processing procedure in the first mode and perform the second processing procedure in the second mode.

6. The method of claim 1, wherein performing the first processing procedure on each frame of the first image data to form the second image data in the first mode comprises: in the first mode, performing the first processing procedure on each frame of the first image data to form the second image data if a remaining power of an electronic device satisfies a first condition; further comprising: In the first mode, if the remaining power of the electronic device satisfies a second condition, a second processing procedure is performed on each frame of the first image data to form fourth image data, the fourth image data being used for display, the resolution of the fourth image data being the same as the resolution of the third image data.

7. An image processor comprising an image processing module and a resolution configuration module; the image processing module is configured to: obtain first image data with an initial resolution in real time; in a first mode, perform a first processing procedure on each frame of the first image data to form second image data, the second image data being used for display; in response to a target trigger operation, switch to a second mode, and obtain target first image data from a plurality of frames of the first image data; perform a second processing procedure on the target first image data to form third image data, the third image data being used for storage; wherein the resolution of the image data processed by the first processing procedure is less than the resolution of the image data processed by the second processing procedure, and the resolution of the third image data is greater than the resolution of the second image data; the resolution configuration module is configured to: in the first mode, configure the resolution of the image data processed by the image processing module in the first processing procedure; in the second mode, configure the resolution of the image data processed by the image processing module in the second processing procedure.

8. The image processor of claim 7, wherein the image processing module comprises a first processing module for resolution reduction and a second processing module for image enhancement; in the first mode, the resolution configuration module is configured to: configure the resolution of the second image data according to the resolution of the display screen; and configure the resolution of the image data in the resolution reduction procedure of the first processing module according to the processing capability of the first processing module and the second processing module, and the initial resolution and the resolution of the second image data.

9. An electronic device comprising a camera module, an image processor, a display module, and a memory; the camera module is configured to receive image signals in real time to output image data; the image processor is configured to: obtain first image data with an initial resolution in real time from the camera module; in a first mode, perform a first processing procedure on each frame of the first image data to form second image data, and output the second image data to the display module; in response to a target trigger operation, switch to a second mode, and obtain target first image data from a plurality of frames of the first image data; perform a second processing procedure on the target first image data to form third image data, and output the third image data to the memory; wherein the resolution of the image data processed by the first processing procedure is less than the resolution of the image data processed by the second processing procedure, and the resolution of the third image data is greater than the resolution of the second image data; the display module is configured to receive the second image data and display; the memory is configured to store the third image data in the second mode.

10. A photographing and displaying method comprising: In response to the electronic device being in the first mode, real-time display of second image data on a display screen of the electronic device, the second image data being processed from first image data collected by the electronic device in real time; In response to a target trigger operation, switching to a second mode, obtaining and storing third image data, the third image data being processed from a frame of first image data collected by the electronic device; The resolution of the third image data is greater than the resolution of the second image data.

Images