Image processing method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention relates to the field of image processing, and discloses an image processing method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: obtaining a plurality of frames of first images collected by a camera in response to a triggered shooting operation; and performing first image processing on a basic frame first image in the multiple frames of first images to obtain a fast image, and displaying the fast image, and carrying out image fusion processing on the multiple frames of first images to obtain a second image, and switching the fast image into the second image for display, and under the condition that the image fusion processing fails, keeping displaying the fast image. By implementing the embodiment of the invention, the storage space occupied in the image shooting and processing process can be saved under the condition of improving the image quality of the shot image.

Description

Technical Field

[0001] The present application relates to the field of image processing, and specifically to an image processing method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the rapid development of image processing technology, current electronic devices can perform image processing on the captured images after capturing them to improve the image quality. In this regard, how to optimize the image capture and processing process is a continuous optimization goal in this field. Summary of the Invention

[0003] The embodiments of the present application disclose an image processing method and device, an electronic device, and a computer-readable storage medium, which can save storage space occupied during image capture and processing while improving the image quality of captured images.

[0004] A first aspect of an embodiment of the present application discloses an image processing method, the method comprising:

[0005] In response to a triggered shooting operation, acquiring a plurality of frames of first images captured by a camera;

[0006] Performing first image processing on a basic frame first image among the multiple frames of first images to obtain a quick image, and displaying the quick image;

[0007] Performing image fusion processing on the multiple frames of first image to obtain a second image, and switching the fast image to display the second image;

[0008] In the case that the image fusion process fails, the quick image is kept displayed.

[0009] A second aspect of an embodiment of the present application discloses an image processing device, comprising:

[0010] an acquisition unit, configured to acquire, in response to a triggered shooting operation, a plurality of frames of first images captured by a camera;

[0011] a first processing unit, configured to perform first image processing on a first basic frame image among the multiple first image frames to obtain a quick image, and display the quick image;

[0012] a second processing unit, configured to perform image fusion processing on the multiple frames of first image to obtain a second image, and switch the fast image to display the second image;

[0013] The display unit is configured to keep displaying the quick image when the image fusion process fails.

[0014] The third aspect of an embodiment of the present application discloses an electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; the processor calls the executable program code stored in the memory to execute the image processing method disclosed in the first aspect of the embodiment of the present application.

[0015] A fourth aspect of an embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the image processing method disclosed in the first aspect of the embodiment of the present application.

[0016] The fifth aspect of the embodiments of the present application discloses a computer program product. When the computer program product is run on a computer, the computer is caused to execute part or all of the steps of any one of the methods of the first aspect of the embodiments of the present application.

[0017] The sixth aspect of the embodiments of the present application discloses an application publishing platform, which is used to publish a computer program product. When the computer program product runs on a computer, the computer executes part or all of the steps of any one of the methods of the first aspect of the embodiments of the present application.

[0018] Compared with the related art, the embodiments of the present application have the following beneficial effects:

[0019] In an embodiment of the present application, in response to a triggered shooting operation, multiple frames of first images captured by the camera can be obtained; then, the first image processing is performed on the basic frame first image in the multiple frames of first images to obtain a quick image, and the quick image is displayed. Among them, since the quick image only needs to be processed on one of the basic frames to obtain, the processing speed is fast, so that the quick image can be displayed more quickly for preview, thereby improving the user experience; and, image fusion processing can be performed on the multiple frames of first images to obtain a second image, and the quick image can be switched to the second image for display; it can be understood that since the second image is obtained by fusion of multiple frames of first images, more image details are retained, so the image quality of the second image is higher than that of the quick image, and replacing it with the quick image can provide the user with an image with better image quality; in addition, if the image fusion processing fails, the quick image can be kept displayed, and there is no need to generate other images to replace the quick image for preview, thereby saving the storage space occupied during the image shooting and processing process while avoiding image loss. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 This is a flowchart of an image processing method disclosed in an embodiment of the present application;

[0022] Figure 2 is a flowchart of another image processing method disclosed in an embodiment of the present application;

[0023] Figure 3 This is a flowchart of another image processing method disclosed in an embodiment of the present application;

[0024] Figure 4 This is a flowchart of another image processing method disclosed in an embodiment of the present application;

[0025] Figure 5 is a flowchart of another image processing method disclosed in an embodiment of the present application;

[0026] Figure 6 is a structural diagram of an image processing device disclosed in an embodiment of the present application;

[0027] Figure 7 This is a structural diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0029] It should be noted that the terms "first," "second," "third," and "fourth" in the specification and claims of this application are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having," as well as any variations thereof, in the embodiments of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or are inherent to these processes, methods, products, or apparatuses.

[0030] The embodiments of the present application disclose an image processing method and device, an electronic device, and a computer-readable storage medium, which can save storage space occupied during image capture and processing while improving the image quality of captured images.

[0031] The technical solution of this application will be described in detail below with reference to specific embodiments.

[0032] In order to more clearly introduce the image processing method disclosed in the embodiment of the present application, the image capturing and processing flow in the related art is first introduced.

[0033] In related technologies, after an electronic device captures a first image through a camera, it usually performs a series of relatively simple image processing on the first image (for example, adjusting brightness, adjusting contrast, etc.) to quickly obtain a preview image, and preferentially displays the preview image in the album for user reference; in addition, the electronic device can also perform more complex image processing on the first image (for example, high dynamic range imaging (HDR) multi-frame fusion processing, turboRaw multi-frame fusion processing, etc.) to obtain a finished image, and then subsequently use a finished image with higher image quality to replace the preview image display, and store the finished image in the album, so that a smoother image feedback process can be achieved without the user noticing, thereby improving the user experience.

[0034] However, it has been found in practice that if the electronic device fails to perform complex image processing on the first image, for example, the multi-frame image fusion processing fails, the electronic device cannot output the finished image, and the preview image is usually only displayed temporarily and will not be stored in the album. In this case, a "lost image" situation will occur in the album, that is, the electronic device has taken the image but cannot find it in the album, thereby reducing the user experience.

[0035] An image processing method disclosed in an embodiment of the present application can be applied to various electronic devices with image processing capabilities, such as portable electronic devices such as mobile phones and tablets, wearable devices such as smart watches and head-mounted devices, or shooting devices such as cameras, without limitation here.

[0036] See also Figure 1 , Figure 1 Optionally, the electronic device may include: an album module 102 , a camera module 104 , a data transmission module 106 , an interface module 108 , and an image processing module 110 .

[0037] The album module 102 is used to display images. After the user triggers the capture of an image, the user can view the captured image in the album supported by the album module 102.

[0038] The camera module 104 is used to detect a triggered shooting operation and perform a corresponding shooting function;

[0039] The data transmission module 106 can be a submodule of the camera module 104, including but not limited to: cameraUnit (camera service module); used to realize data transmission between the interface module 108 and the image processing module 110; optionally, the data transmission module 106 may include a data generation unit 1061 and a data consumption unit 1062, wherein: the data generation unit 1061 is used to generate and push image data, and the data consumption unit 1062 is used to pull and process image data.

[0040] The interface module 108 may include, but is not limited to, a hardware abstraction layer (HAL), which implements the interface and control between the operating system and the camera hardware. It abstracts the hardware details and encapsulates the camera control, image processing, and hardware interaction into a standardized interface so that the operating system and applications can easily use the camera to implement functions such as image sensor control, image data acquisition and processing.

[0041] The image processing module 110 may be a native service layer of the camera module 104. The image processing module 110 may be used to perform various image processing operations on the captured images. Optionally, the data transfer module 106 may call image processing methods in the image processing module 110 via a target operation interface to implement image processing functions. The target operation interface may include a Java Native Interface.

[0042] Optionally, in response to a triggered shooting operation, the camera module 104 can send a shooting request to the data generation unit 1061 in the data transmission module 106 to collect multiple frames of first images and send them to the interface module 108; then the interface module 108 can feed back multiple frames of first images from the "first image queue" to the data consumption unit 1062 in the data transmission module 106, and the data consumption unit 1062 sends the multiple frames of first images to the image processing module 110 for processing, thereby realizing the transmission of multiple frames of first image data from the hardware to the operating system.

[0043] Furthermore, the image processing module 110 may perform frame selection and / or frame removal processing on the multiple first image frames to obtain a base frame first image. Optionally, the base frame first image may be the first image in the multiple first image frames that satisfies a first condition; the first condition may include only one of the following: (1) the widest dynamic range coverage and the exposure within the target exposure range; (2) the highest image quality; (3) coverage of the target scene and / or target person, etc., which are not limited here.

[0044] It should be noted that when performing HDR multi-frame fusion processing, a base frame is required as the basis for fusion to improve the accuracy, efficiency and final effect of the fusion. The first image of the base frame can be a base frame obtained by frame selection during the HDR multi-frame fusion process. The frame selection processing method can include the base frame automatically determined by the algorithm, or the user selecting the base frame according to usage requirements, which is not limited here.

[0045] Furthermore, the image processing module 110 can obtain the first base frame image and perform one or more single-frame image processing and encoding processing on the first base frame image through the first image processing pipeline 1101 to obtain a quick image (quickJpeg). The quick image is then fed back to the camera module 104, which can then send the quick image to the album module 102 for display. Optionally, the single-frame image processing may include one or more of: image format conversion, rotation of the luminance component and / or chrominance component in the target color space, image beautification processing, and adding a target pattern, which are not limited here.

[0046] For example, Figure 1 The first image of the basic frame can be converted from RAW format to YUV format, and then the brightness component and / or chrominance component can be rotated in the YUV domain, face beautification processing can be performed, and watermarks can be added, etc. Finally, the Jpeg format encoding processing is performed to obtain a quick image.

[0047] In addition, the image processing module 110 can obtain the first image of the basic frame, and perform image fusion processing (for example: turboRaw multi-frame fusion processing) on ​​multiple frames of the first image through the second image processing pipeline 1102 to obtain a fused image, and perform single-frame image processing on the fused image, and then encode the processed fused image to obtain a second image, and feed the second image back to the camera module 104, and then the camera module 104 can send the second image to the album module 102 to replace the quick image for display.

[0048] If the image fusion processing of the multiple frames of first images fails, the basic frame first image in the multiple frames of first images can be obtained as a backup first image, and the backup first image can be processed using the single-frame image processing method of the target fusion algorithm (for example: turboRaw single-frame processing), and then single-frame image processing and encoding processing are performed to obtain a second image, and the second image is fed back to the camera module 104; then the camera module 104 can send the second image to the album module 102 to replace the quick image for display.

[0049] Among them, turboRaw single-frame processing is a method that can optimize the processing of single-frame images. Although its optimization effect is worse than that of turboRaw multi-frame fusion processing, it can process single-frame images and has lower implementation requirements.

[0050] It can be seen that by implementing the above method, when the image fusion processing fails, the electronic device can obtain the basic frame first image from multiple frames of first images as a backup first image, and generate the second image through processing processes such as turboRaw single frame processing, single frame image processing, and programming processing, thereby ensuring that the second image can be successfully generated, thereby avoiding the situation of lost images in the album and improving the user experience.

[0051] However, in the above method, if the image fusion processing of multiple frames of first images fails, it is necessary to add an additional storage space (for example, a hardware buffer) to store the backup first image. In practice, it is found that the backup first image requires approximately 24MB of storage space, which will increase the storage pressure of the electronic device. In addition, generating the second image by backing up the first image also increases the computational complexity of the electronic device.

[0052] See also Figure 2 , Figure 2 1 is a flow chart of another image processing method disclosed in an embodiment of the present application. Optionally, the method can be applied to the electronic device described above, without limitation herein.

[0053] Optionally, in response to a triggered shooting operation, the camera module 104 can send a shooting request to the data generation unit 1061 in the data transmission module 106 to collect multiple frames of first images and send them to the interface module 108; then the interface module 108 can feed back multiple frames of first images from the "first image queue" to the data consumption unit 1062 in the data transmission module 106, and the data consumption unit 1062 sends the multiple frames of first images to the image processing module 110 for processing, thereby realizing the transmission of multiple frames of first image data from the hardware to the operating system.

[0054] Furthermore, the image processing module 110 can perform frame selection and / or frame removal on the multiple frames of the first image to obtain a base frame first image. The image processing module 110 can obtain the base frame first image and perform one or more single-frame image processing and encoding processes on the base frame first image through the first image processing pipeline 1101 to obtain a quick image, and then feed the quick image back to the camera module 104. The camera module 104 can then send the quick image to the album module 102 for display. Since the quick image only needs to be processed on one of the base frames, the processing speed is fast, so the quick image can be displayed for preview more quickly, improving the user experience.

[0055] Furthermore, the image processing module 110 can obtain a first image as a base frame, and perform image fusion processing (e.g., turboRaw multi-frame fusion processing) on ​​multiple first image frames through the second image processing pipeline 1102 to obtain a fused image. The fused image can then be processed as a single frame, and the processed fused image can be encoded to obtain a second image. The second image can then be fed back to the camera module 104. The camera module 104 can then send the second image to the album module 102 to replace the quick image for display. It can be understood that because the second image is obtained by fusing multiple first image frames, it retains more image details, so the image quality of the second image is higher than that of the quick image. Replacing the quick image with the second image can provide users with images of better image quality.

[0056] In addition, in the case of image fusion processing failure, since there is no second image available for display, the electronic device can continue to display the quick image to avoid the situation where the image is lost in the album; in addition, the electronic device can no longer perform Figure 1 The steps of "obtaining a basic frame first image from multiple frames of first images as a backup first image, and processing the backup first image through a single-frame image processing method of a target fusion algorithm (for example, turboRaw single-frame processing), and then performing single-frame image processing and encoding processing to obtain a second image" are shown in the text, so that there is no need to add additional storage space for storing the backup first image, thereby saving the storage space occupied during the image shooting and processing process, and reducing the computing power of the electronic device.

[0057] In addition, the step of "obtaining the basic frame first image from multiple frames of first images as the backup first image, and processing the backup first image through the single-frame image processing method of the target fusion algorithm (for example: turboRaw single-frame processing), and then performing single-frame image processing and encoding processing to obtain the second image" has been deleted. It also reduces the redundancy of the algorithm code, improves the efficiency of generating the final product image, optimizes the smoothness of shooting, and the optimization effect is more obvious under heavy-load shooting conditions.

[0058] It should be noted that Figure 1 and Figure 2 This is only an optional implementation, and the electronic device may further include other modules and / or units. Figure 1 and Figure 2 This should not be construed as limiting the embodiments of the present application.

[0059] Based on this, the image processing method and device, electronic device, and computer-readable storage medium disclosed in the embodiments of the present application are introduced below.

[0060] See also Figure 3 , Figure 3 This is a flow chart of another image processing method disclosed in an embodiment of the present application. Optionally, the method can be applied to the above-mentioned electronic device (specifically, it can be applied to the processor of the electronic device), or other execution entities, which are not limited here. Optionally, the method can include the following steps:

[0061] 302. In response to a triggered shooting operation, acquire multiple frames of first images captured by a camera.

[0062] In the embodiment of the present application, the capture operation is used to trigger the camera to perform the capture function. Optionally, the capture operation may include: a click operation of clicking a capture button, a call operation of another application calling the camera to capture an image (for example, a shopping application calling the camera to capture an image of an object for identifying object information and price, etc.), etc., which are not limited here.

[0063] In an optional embodiment, the triggered shooting operation may be input by a user, which is not limited here.

[0064] The camera may include components such as an image sensor, which can convert optical signals into corresponding electrical signals. These electrical signals record the captured image data, thereby enabling the camera to capture images. Optionally, in response to a triggered capture operation, the electronic device may send a capture request to the camera, and the camera may capture multiple frames of the first image in response to the capture request.

[0065] Optionally, the multiple frames of first images may be captured continuously or indirectly, which is not limited herein. Optionally, the multiple frames of first images may include two or more frames of first images; or three or more frames of first images, which is not limited herein.

[0066] Because the camera may move during the shooting process, the exposure of the same subject in multiple first image frames may be different. For example, the exposure of the facial area in multiple first image frames may be different. In this case, HDR fusion processing can be performed on the multiple first image frames to obtain an image with a normal exposure of the facial area.

[0067] 304 : Perform first image processing on the basic frame first image among the multiple frames of first images to obtain a quick image, and display the quick image.

[0068] In the embodiment of the present application, the electronic device can obtain a basic frame first image from multiple frames of first images through frame selection processing and / or frame removal processing.

[0069] Optionally, the frame selection process may include: determining a first image that meets a first condition from the plurality of first image frames as the base frame first image. How to determine the base frame first image has been described above and will not be repeated here.

[0070] The frame removal process may include removing abnormal images from the plurality of first image frames. Optionally, the abnormal images may include first images having an image clarity less than a clarity threshold, first images having an abnormal exposure of a target area (e.g., a face area, a foreground area, etc.), or first images having an image noise greater than a noise threshold, without limitation herein.

[0071] Alternatively, the electronic device may first perform frame culling to obtain at least two remaining first image frames, and then perform frame selection to obtain the base first image frame. In yet another optional embodiment, the electronic device may perform frame culling on multiple first image frames until only one first image frame remains, and determine this first image frame as the base first image frame. This is not a limitation herein.

[0072] In the embodiment of the present application, the first image processing may include: the single-frame image processing mentioned above, or a combination of single-frame image processing and encoding processing, which is not limited here. Optionally, the encoding processing may include: encoding processing of the target format. Optionally, the target format may include: JPEG (Joint Photographic Experts Group) format, JPEG format, WebP format, or HEIF / HEIC format, etc., which is not limited here.

[0073] Among them, the basic frame (i.e., base frame) is usually in RAW format. The basic frame in RAW format contains richer color information and higher image quality. The image quality of the quick image obtained by performing the first image processing based on the basic frame (i.e., base frame) is also better, so that the image quality of the quick image previewed by the user in the album is higher, thereby improving the user experience.

[0074] Optionally, the electronic device may display the snapshot on the display screen of the electronic device for user reference. Optionally, the electronic device may display the snapshot in the electronic device's photo album, or display a thumbnail of the snapshot, and display the snapshot upon detecting a viewing operation for the thumbnail of the snapshot, without limitation herein.

[0075] 306. Perform image fusion processing on the multiple frames of the first image to obtain a second image, and switch the quick image to display the second image.

[0076] In the embodiment of the present application, the image fusion processing may include the aforementioned: turboRaw multi-frame fusion processing, HDR multi-frame fusion processing, etc., which are not limited here.

[0077] It can be understood that the second image obtained through image fusion processing fuses the image details of multiple frames of the first image. Therefore, the image quality of the second image is usually higher than the image quality of the quick image. Therefore, the electronic device can replace the quick image with the second image for display to provide an image with better image quality, thereby improving the user experience.

[0078] 308. In the case that the image fusion process fails, the quick image is kept displayed.

[0079] In the embodiments of the present application, the electronic device may fail to perform image fusion processing on multiple frames of the first image, thereby failing to successfully output the second image. To avoid the situation where the second image cannot be displayed in the album and thus results in "image loss", the electronic device does not delete the temporarily displayed quick image as in the related art, but instead maintains the display of the quick image.

[0080] Optionally, before the electronic device determines whether to display the quick image, the quick image may be stored in a temporary storage space, as it is only temporarily displayed, to facilitate deletion at any time. However, if the image fusion process fails, the quick image may need to be displayed to prevent accidental deletion of the quick image and to facilitate searching for the quick image. As an optional embodiment, if the image fusion process fails, the electronic device may store the quick image in the target storage space.

[0081] Optionally, the target storage space may be a local storage space of the electronic device, or a cloud storage space corresponding to the electronic device, which is not limited here.

[0082] In an optional embodiment, the target storage space may be the storage space corresponding to the photo album of the electronic device, so that if a display operation for a quick image is subsequently detected in the photo album, the electronic device can quickly find the quick image from the storage space corresponding to the photo album and display it, thereby improving the display efficiency of the image.

[0083] By implementing the above method, the electronic device can store the quick image in the target storage space when the image fusion processing fails, thereby preventing the quick image from being accidentally deleted and facilitating the search for the quick image, thereby improving the intelligence of the method.

[0084] As an optional implementation, when the electronic device determines that the image fusion processing has failed, it can accumulate a failure count value of one and determine whether the current failure count value is greater than or equal to the count threshold. If it is greater than or equal to the count threshold, a reminder message is output. The reminder message is used to remind the electronic device that the image fusion processing is in a fault state.

[0085] The counting threshold can be set by developers based on extensive development experience and is not limited here.

[0086] It should be noted that, in the embodiment of the present application, even if the image fusion process fails, the quick image will remain displayed, and thus no image will be lost in the album. Since the difference between the quick image and the second image is difficult to detect with the naked eye without comparison, the user will not be able to detect when the image fusion process fails multiple times and is in a faulty state. The electronic device can output a reminder message to alert the user, prompting the user to repair the image fusion process function to improve the image quality of subsequent images.

[0087] By implementing the methods disclosed in the above embodiments, in response to a triggered shooting operation, multiple frames of first images captured by the camera can be obtained; then, the first image processing is performed on the basic frame first image in the multiple frames of first images to obtain a quick image, and the quick image is displayed. In particular, since the quick image only needs to be processed on one of the basic frames to obtain, the processing speed is fast, so that the quick image can be displayed more quickly for preview, thereby improving the user experience; and, image fusion processing can be performed on the multiple frames of first images to obtain a second image, and the quick image can be switched to the second image for display; it can be understood that since the second image is obtained by fusion of multiple frames of first images, more image details are retained, so the image quality of the second image is higher than that of the quick image, and replacing it with the quick image can provide the user with an image with better image quality; in addition, if the image fusion processing fails, the quick image can be kept displayed, and there is no need to generate other images to replace the quick image for preview, thereby saving the storage space occupied during the image capture and processing process while avoiding image loss.

[0088] See also Figure 4 , Figure 4 This is a flow chart of another image processing method disclosed in an embodiment of the present application. Optionally, the method can be applied to the above-mentioned electronic device or other execution entities, which are not limited here. Optionally, the method can include the following steps:

[0089] 402. In response to a triggered shooting operation, obtain multiple frames of first images captured by a camera.

[0090] 404. Perform first image processing on the basic frame first image in the multiple frames of first images through a first image processing pipeline to obtain a quick image.

[0091] 406. Perform image fusion processing on the multiple frames of first images through a second image processing pipeline to obtain a second image.

[0092] Please refer again Figure 2Optionally, the electronic device may perform first image processing on the base frame first image in the multiple frames of first images through the first image processing pipeline 1101; and perform image fusion processing on the multiple frames of first images through the second image processing pipeline 1102.

[0093] Optionally, the first image processing pipeline and the second image processing pipeline may be executed in parallel or in series, which is not limited here.

[0094] By implementing the above method, the electronic device generates the quick image and the second image respectively through two independent image processing pipelines, which can avoid interference between the generation processes of the two, thereby ensuring the image quality of the generated quick image and the second image.

[0095] In an embodiment of the present application, when the electronic device performs image fusion processing on multiple frames of the first image to obtain a second image, the electronic device may perform a cropping operation on the image to obtain a second image of a target image size. Optionally, the target image size may be set by the developer based on a large number of developments or by the user based on actual usage requirements, and is not limited here.

[0096] By implementing the above method, the image size of the subsequently generated second image is kept uniform through the cropping operation, thereby facilitating appropriate display in a display area (eg, an album) to ensure the display effect of the image.

[0097] In practice, it is found that the image size of the generated quick image may be larger than the target image size corresponding to the second image. Therefore, when the quick image is subsequently switched to the second image for display, the image size will jump due to the difference in image sizes between the quick image and the second image, thereby reducing the user's viewing experience.

[0098] In this regard, in an optional embodiment, the electronic device may crop the basic frame first image in the multiple frames of first images according to the target image size to obtain a quick image; wherein the target image size is the image size corresponding to the second image.

[0099] By implementing the above method, when the electronic device generates a quick image based on the basic frame first image in multiple frames of first images, it can crop the generated quick image according to the target image size of the second image, so that the image size of the generated quick image and the second image are consistent, thereby avoiding the situation where the image size jumps when the quick image is subsequently switched to the second image for display, thereby improving the user's viewing experience.

[0100] 408. In the case that the image fusion process fails, the quick image is kept displayed.

[0101] In an optional embodiment, when the image fusion processing fails, the electronic device may close the second image processing pipeline to avoid the electronic device from performing the image fusion processing again, thereby reducing the computational complexity of the electronic device.

[0102] Also, please refer again to Figure 2 The electronic device closes the second image processing pipeline, and can also avoid the electronic device from executing the step of "obtaining the basic frame first image from the multiple frames of first images as the backup first image, and processing the backup first image through the single-frame image processing method of the target fusion algorithm (for example: turboRaw single-frame processing), and then performing single-frame image processing and encoding processing to obtain the second image".

[0103] It can be seen that when the above method is implemented, if the image fusion processing fails, the second image processing pipeline is closed to prohibit the electronic device from executing the steps of "obtaining the basic frame first image from multiple frames of first images as the backup first image, and processing the backup first image through the single-frame image processing method of the target fusion algorithm (for example: turboRaw single-frame processing), and then performing single-frame image processing and encoding processing to obtain the second image". This reduces the computational load of the electronic device, and no longer requires additional storage space to store the backup first image, thereby saving storage space of the electronic device.

[0104] As another optional implementation, in the event that the image fusion processing fails, a target capacity of a storage space available to the electronic device is obtained; if the target capacity is less than a first capacity threshold, the second image processing pipeline is closed to prevent the electronic device from performing the image fusion processing again, thereby reducing the computational workload of the electronic device.

[0105] If the target capacity is greater than or equal to the second capacity threshold, the basic frame first image is obtained from the multiple frames of first images as the backup first image through the second image processing pipeline, and the backup first image is processed by the single-frame image processing method of the target fusion algorithm, and then single-frame image processing and encoding processing are performed to obtain the second image.

[0106] Among them, the second capacity threshold is greater than the first capacity threshold, and the first capacity threshold can be less than the image data volume of the backed-up first image; the second capacity threshold can be greater than the image data volume of the backed-up first image. The specific value can be set by the developer based on a lot of development experience and is not limited here.

[0107] By implementing the above method, when the electronic device's storage space is heavily loaded, the second image processing pipeline can be shut down to avoid the need to store a backup first image, which would further increase the storage pressure on the electronic device. Furthermore, when the electronic device has ample storage space, the second image processing pipeline can be used to obtain a base frame first image from multiple frames of first images as a backup first image. The backup first image can then be processed using the single-frame image processing method of the target fusion algorithm, followed by further single-frame image processing and encoding to obtain a second image with higher image quality, which can be used to replace the quick image. This can improve the quality of the final image provided and enhance the user experience.

[0108] By implementing the methods disclosed in the above embodiments, in response to a triggered shooting operation, multiple frames of first images captured by the camera can be obtained; then, first image processing is performed on the base frame first image in the multiple frames of first images to obtain a quick image, and the quick image is displayed. Since the quick image only requires processing one of the base frames to obtain the quick image, the processing speed is fast, and thus the quick image can be displayed more quickly for preview, thereby improving the user experience; and, image fusion processing can be performed on the multiple frames of first images to obtain a second image, and the quick image can be switched to the second image for display; it can be understood that since the second image is obtained by fusion of multiple frames of first images, more image details are retained, so the image quality of the second image is higher than that of the quick image, and replacing it with the quick image can provide the user with an image of better image quality; in addition, if the image fusion processing fails, the quick image can be displayed without generating other images to replace the quick image for preview, thereby reducing the computational load of the electronic device and saving storage space of the electronic device while avoiding image loss; and, by generating the quick image and the second image respectively through two independent image processing pipelines, interference between the generation processes of the two can be avoided, thereby ensuring that the generated quick image and the second image are the same. image quality; and, in the process of generating a quick image based on the basic frame first image in the multiple frames of first images, the generated quick image can be cropped according to the target image size of the second image so that the image size of the generated quick image and the second image are consistent, thereby avoiding the situation where the image size jumps when the quick image is subsequently switched to the second image for display, thereby improving the user's viewing experience; and, in the case of image fusion processing failure, closing the second image processing pipeline to prohibit the electronic device from executing the step of "obtaining the basic frame first image from the multiple frames of first images as the backup first image, and processing the backup first image through the single-frame image processing method of the target fusion algorithm (for example: turboRaw single-frame processing), and then performing single-frame image processing and encoding processing to obtain the second image", thereby reducing the calculation amount of the electronic device and no longer needing to add additional storage space for storing the backup first image, thereby saving the storage space of the electronic device.

[0109] See also Figure 5 , Figure 5 This is a flow chart of another image processing method disclosed in an embodiment of the present application. Optionally, the method can be applied to the above-mentioned electronic device or other execution entities, which are not limited here. Optionally, the method can include the following steps:

[0110] 502. In response to a triggered shooting operation, obtain multiple frames of first images captured by a camera.

[0111] 504. Convert the base frame first image in the multiple frames of first images into a target color space to obtain a third image. The target color space is a color space including a luminance information channel and a chrominance information channel.

[0112] In an embodiment of the present application, the electronic device can convert the basic frame first image among the multiple frames of first images into a target color space, thereby facilitating subsequent image optimization processing of the image.

[0113] Optionally, the target color space may include a color space in the YUV domain, which is not limited here.

[0114] 506. Perform image optimization processing on the third image to obtain a fourth image.

[0115] As an optional embodiment, the image optimization processing may include: image denoising processing, color calibration processing, rotating the brightness component and / or chrominance component of the third image in the target color space, image beautification processing, and adding one or more of the target pattern.

[0116] Among them, performing image denoising and / or color calibration on the third image can retain more image details, especially details in the transition part of color and brightness, thereby improving the ability of the subsequently generated quick image to retain more image details and improving the image quality of the quick image.

[0117] 508. Encode the fourth image to obtain a quick image in a target format, and display the quick image.

[0118] In an embodiment of the present application, the target format may include Jpeg (Joint Photographic Experts Group) format, Jpg format, WebP format or HEIF / HEIC format, etc., which is not limited here.

[0119] The target format can compress the image to reduce the data volume of the generated target format quick image, thereby saving storage space of the electronic device and facilitating the transmission of the quick image.

[0120] By implementing the above method, the electronic device can first convert the first image of the basic frame into the target color space to facilitate subsequent image optimization processing. Furthermore, the electronic device can also encode the optimized fourth image to compress the image data volume and save the storage space of the electronic device.

[0121] In an optional embodiment, the image format of the first image of the base frame may be a RAW format, and the target format may be a JPEG format.

[0122] Since RAW format images have a higher dynamic range, more accurate color reproduction and better noise control, the image quality of the first image of the base frame in RAW format is higher, and the image quality of the JPEG format quick image generated using RAW format is also higher.

[0123] In addition, the YUV format retains brightness information (Y channel) and a certain degree of chrominance information (UV channel), but the resolution of its chrominance information is lower than that of the brightness part, which means that color details will be lost. If it is only converted from RAW format to YUV format, the color details will be lost, thereby reducing the image quality of the generated quick image.

[0124] 510. Perform image fusion processing on multiple frames of first images to obtain a second image, and switch the quick image to display the second image.

[0125] 512. In the event that the image fusion process fails, the quick image will continue to be displayed.

[0126] By implementing the methods disclosed in the above embodiments, in response to a triggered shooting operation, multiple frames of first images captured by the camera can be obtained; then, the basic frame first image in the multiple frames of first images is subjected to first image processing to obtain a quick image, and the quick image is displayed. Since the quick image only needs to be processed on one frame of the basic frame to obtain, the processing speed is fast, so that the quick image can be displayed more quickly for preview, which improves the user experience; and, image fusion processing can be performed on the multiple frames of first images to obtain a second image, and the quick image can be switched to the second image for display; it can be understood that since the second image is obtained by fusion of multiple frames of first images, more image details are retained, so the image quality of the second image is higher than that of the quick image, and replacing it with the quick image can provide the user with an image with better image quality; in addition, in the case of failure of image fusion processing, the quick image can be kept displayed, and there is no need to generate other image replacements. The first image of the basic frame can be converted to a target color space to facilitate subsequent image optimization processing. Furthermore, the electronic device can encode the optimized fourth image to compress the image data and save the storage space of the electronic device.

[0127] See also Figure 6 , Figure 6 6 is a schematic diagram of the structure of an image processing device disclosed in an embodiment of the present application. Optionally, the device can be applied to the above-mentioned electronic device or other execution entities, which are not limited here. Optionally, the device can include an acquisition unit 602, a first processing unit 604, a second processing unit 606 and a display unit 608, wherein:

[0128] An acquisition unit 602 is configured to acquire a plurality of first image frames captured by a camera in response to a triggered shooting operation;

[0129] A first processing unit 604 is configured to perform first image processing on a base frame first image in the plurality of first image frames to obtain a quick image and display the quick image;

[0130] The second processing unit 606 is configured to perform image fusion processing on the multiple frames of the first image to obtain a second image, and switch the quick image to display the second image;

[0131] The display unit 608 is configured to keep displaying the quick image if the image fusion process fails.

[0132] By implementing the above-mentioned device, in response to a triggered shooting operation, multiple frames of first images captured by the camera can be obtained; the first image processing of the basic frame first image in the multiple frames of first images can be performed to obtain a quick image, and the quick image can be displayed. Since the quick image only needs to be processed by one of the basic frames, the processing speed is fast, so the quick image can be displayed more quickly for preview, improving the user experience; and image fusion processing can be performed on the multiple frames of first images to obtain a second image, and the quick image can be switched to the second image for display; it can be understood that since the second image is obtained by fusing multiple frames of first images, more image details are retained, so the image quality of the second image is higher than that of the quick image, and replacing it with the quick image can provide the user with an image of better image quality; in addition, if the image fusion processing fails, the quick image can be displayed, and there is no need to generate other images to replace the quick image for preview, thereby saving the storage space occupied during the image capture and processing process while avoiding image loss.

[0133] As an optional implementation, the first processing unit 604 is further configured to perform first image processing on the base frame first image in the multiple frames of first images through the first image processing pipeline to obtain a quick image;

[0134] The second processing unit 606 is further configured to perform image fusion processing on the multiple frames of first images through a second image processing pipeline to obtain a second image.

[0135] By implementing the above device, the electronic device generates the quick image and the second image respectively through two independent image processing pipelines, which can avoid interference between the generation processes of the two, thereby ensuring the image quality of the generated quick image and the second image.

[0136] As an optional implementation, Figure 6 The device shown may also include a closing unit (not shown), wherein:

[0137] The closing unit is used to close the second image processing pipeline when the image fusion processing fails.

[0138] By implementing the above-mentioned device, when the image fusion processing fails, the second image processing pipeline is closed to prohibit the electronic device from executing the steps of "obtaining the basic frame first image from multiple frames of first images as the backup first image, and processing the backup first image through the single-frame image processing method of the target fusion algorithm (for example: turboRaw single-frame processing), and then performing single-frame image processing and encoding processing to obtain the second image". This reduces the computational load of the electronic device and eliminates the need to add additional storage space for storing the backup first image, thereby saving storage space of the electronic device.

[0139] As an optional implementation, the first processing unit 604 is further configured to crop the base frame first image in the multiple frames of first images according to a target image size to obtain a quick image, where the target image size is the image size corresponding to the second image.

[0140] By implementing the above-mentioned device, when the electronic device generates a quick image based on the basic frame first image in multiple frames of first images, it can crop the generated quick image according to the target image size of the second image, so that the image size of the generated quick image and the second image are consistent, thereby avoiding the situation where the image size jumps when the quick image is subsequently switched to the second image for display, thereby improving the user's viewing experience.

[0141] As an optional embodiment, the first processing unit 604 is further used to convert the basic frame first image in the multiple frames of first images into a target color space to obtain a third image, where the target color space is a color space including a luminance information channel and a chrominance information channel; and, perform image optimization processing on the third image to obtain a fourth image; and, perform encoding processing on the fourth image to obtain a quick image in a target format.

[0142] By implementing the above device, the electronic device can first convert the first image of the basic frame into the target color space to facilitate subsequent image optimization processing. Furthermore, the electronic device can also encode the optimized fourth image to compress the image data volume and save the storage space of the electronic device.

[0143] As an optional embodiment, the image optimization processing includes: image denoising processing, color calibration processing, rotating the brightness component and / or chromaticity component of the third image in the target color space, image beautification processing, and adding one or more of the target pattern.

[0144] By implementing the above-mentioned device and performing image denoising and / or color calibration on the third image, more image details can be retained, especially details in the transition part of color and brightness, thereby improving the ability of the subsequently generated quick image to retain more image details and improving the image quality of the quick image.

[0145] As an optional implementation, Figure 6 The device shown may further include a storage unit (not shown), wherein:

[0146] The storage unit is used to store the quick image in the target storage space when the image fusion processing fails.

[0147] By implementing the above device, the electronic device can store the quick image in the target storage space when the image fusion processing fails, thereby preventing the quick image from being accidentally deleted and facilitating the search for the quick image, thereby improving the intelligence of the method.

[0148] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. Figure 7 As shown, the electronic device may include:

[0149] A memory 701 storing executable program code;

[0150] a processor 702 coupled to the memory 701;

[0151] The processor 702 calls the executable program code stored in the memory 701 to execute the image processing method disclosed in the above embodiments.

[0152] An embodiment of the present application discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the image processing method disclosed in the above embodiments.

[0153] An embodiment of the present application further discloses an application publishing platform, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product runs on a computer, the computer executes part or all of the steps of the method in the above method embodiments.

[0154] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment" throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required for the present application.

[0155] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the above-mentioned processes does not necessarily mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0156] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of these units may be selected based on actual needs to achieve the objectives of this embodiment.

[0157] In addition, the functional units in the embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0158] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a memory and includes several requests for a computer device (which can be a personal computer, server or network device, etc., specifically a processor in a computer device) to execute some or all of the steps of the above-mentioned methods of various embodiments of the present application.

[0159] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0160] The above is a detailed introduction to the image processing method and device, electronic device, and computer-readable storage medium disclosed in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, based on the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the contents of this specification should not be understood as limiting the present application.

Claims

1. An image processing method, characterized in that: The method comprises: In response to a triggered shooting operation, acquiring a plurality of frames of first images captured by a camera; Performing first image processing on a basic frame first image among the multiple frames of first images to obtain a quick image, and displaying the quick image; Performing image fusion processing on the multiple frames of first image to obtain a second image, and switching the fast image to display the second image; In the case that the image fusion process fails, the quick image is kept displayed.

2. The method according to claim 1, characterized in that The performing first image processing on the base frame first image in the multiple frames of first images to obtain a quick image includes: Performing first image processing on a base frame first image in the plurality of first image frames through a first image processing pipeline to obtain a quick image; Furthermore, performing image fusion processing on the multiple frames of first images to obtain a second image includes: The second image processing pipeline performs image fusion processing on the multiple frames of first image to obtain a second image.

3. The method according to claim 2, characterized in that The method further comprises: If the image fusion process fails, the second image processing pipeline is closed.

4. The method according to any one of claims 1 to 3, characterized in that The performing first image processing on the base frame first image in the multiple frames of first images to obtain a quick image includes: The first image of the basic frame in the multiple first image frames is cropped according to a target image size to obtain a quick image, and the target image size is an image size corresponding to the second image.

5. The method according to any one of claims 1 to 3, characterized in that The performing first image processing on the base frame first image in the multiple frames of first images to obtain a quick image includes: Converting a first image, a base frame, from the plurality of first image frames, to a target color space to obtain a third image, wherein the target color space is a color space including a luminance information channel and a chrominance information channel; performing image optimization processing on the third image to obtain a fourth image; The fourth image is coded to obtain a fast image in a target format.

6. The method according to claim 5, characterized in that The image optimization processing includes: image denoising processing, color calibration processing, rotating the brightness component and / or chrominance component of the third image in the target color space, image beautification processing, and adding a target pattern or more.

7. The method according to any one of claims 1 to 3, characterized in that The method further comprises: In the case that the image fusion process fails, the quick image is stored in the target storage space.

8. An image processing device, characterized in that: The device comprises: an acquisition unit, configured to acquire, in response to a triggered shooting operation, a plurality of frames of first images captured by a camera; a first processing unit, configured to perform first image processing on a first basic frame image among the multiple first image frames to obtain a quick image, and display the quick image; a second processing unit, configured to perform image fusion processing on the multiple frames of first image to obtain a second image, and switch the fast image to display the second image; The display unit is configured to keep displaying the quick image when the image fusion process fails.

9. An electronic device, characterized in that: The method comprises a memory storing executable program code and a processor coupled to the memory; wherein the processor calls the executable program code stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images