Image processing method, device, electronic device and storage medium

Abstract

The present application discloses an image processing method, device, electronic device and storage medium, which relate to the field of image processing technology. The method includes: when an image acquisition operation is detected, acquiring m frames of target images frame by frame, where m is a positive integer. When the m-th frame of target image is acquired, a first processing result is acquired, and the first processing result includes the result obtained by pre-preset processing of at least one frame of target image before the m-th frame. Based on the first processing result, a second preset processing is performed on all m frames of target image to obtain a second processing result. When acquiring the m-th frame of target image, the present application can obtain the result of pre-preset processing of at least one frame of target image before the m-th frame, and then perform the second preset processing on all m frames of image based on this result. In this way, the images acquired before the m-th frame can be processed in advance before all m frames of target image are acquired, which can improve the speed of image processing.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to an image processing method, device, electronic device and storage medium. Background Art

[0002] The image acquisition process can be simplified into three steps: 1. Enable image acquisition (e.g., opening a camera); 2. Capture an image; 3. Disable image acquisition (e.g., closing the camera). This process may be repeated multiple times. To meet increasingly demanding image acquisition requirements, various image processing techniques, such as noise reduction, HDR, and beautification, are often performed during image acquisition. Multi-frame image algorithms typically perform image processing only after all required frames have been acquired, resulting in slower overall processing speeds. Summary of the Invention

[0003] This application proposes an image processing method, device, electronic device and storage medium to improve the above-mentioned defects.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, comprising: when an image acquisition operation is detected, acquiring m frames of target images frame by frame, where m is a positive integer; when the mth frame of target image is acquired, acquiring a first processing result, the first processing result including a result obtained by performing a first preset processing on at least one frame of target image before the mth frame; based on the first processing result, performing a second preset processing on the m frames of target image to obtain a second processing result.

[0005] In a second aspect, embodiments of the present application further provide an image processing device, comprising: a first acquisition unit, a second acquisition unit, and an image processing unit. The first acquisition unit is configured to acquire, upon detecting an image acquisition operation, m frames of target images, frame by frame, where m is a positive integer; the second acquisition unit is configured to acquire, upon acquiring the mth frame of target image, a first processing result, the first processing result comprising a result obtained by performing a first preset processing on at least one frame of target image prior to the mth frame; and the image processing unit is configured to perform a second preset processing on the m frames of target image based on the first processing result to obtain a second processing result.

[0006] In a third aspect, an embodiment of the present application further provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the above method by calling the computer program.

[0007] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded by a processor and executes the above method.

[0008] In a fifth aspect, an embodiment of the present application further provides a computer program product comprising instructions, wherein the computer program product stores instructions that, when executed on a computer, enable the computer to implement the above method.

[0009] The image processing method, device, electronic device and storage medium provided in the embodiments of the present application can acquire m frames of target images frame by frame when an image acquisition operation is detected, where m is a positive integer. When the m-th frame of the target image is acquired, a first processing result is acquired, and the first processing result includes the result obtained by performing a first preset processing on at least one frame of the target image before the m-th frame. Then, based on the first processing result, a second preset processing is performed on all m frames of the target image to obtain a second processing result. When acquiring the m-th frame of the target image, the embodiments of the present application can obtain the result of performing a first preset processing on at least one frame of the target image before the m-th frame, and then perform a second preset processing on all m frames of the image based on this result. In this way, the images acquired before the m-th frame can be processed in advance before all m frames of the target image are acquired, which can improve the speed of image processing.

[0010] Other features and advantages of the embodiments of the present application will be described in the following description and, in part, will become apparent from the description or be understood by practicing the embodiments of the present application. The objectives and other advantages of the embodiments of the present application can be achieved and obtained through the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A schematic diagram showing the timeline of the current image processing;

[0013] Figure 2 A flowchart of an image processing method provided by an embodiment of the present application is shown;

[0014] Figure 3A schematic diagram showing a timeline of image processing provided by an embodiment of the present application is shown;

[0015] Figure 4 A flowchart of an image processing method provided by another embodiment of the present application is shown;

[0016] Figure 5 A schematic diagram showing a timeline of image processing provided by another embodiment of the present application is shown;

[0017] Figure 6 A flowchart of an image processing method provided by another embodiment of the present application is shown;

[0018] Figure 7 A schematic diagram showing a time axis of image processing provided by another embodiment of the present application is shown;

[0019] Figure 8 A schematic diagram showing a timeline of another image processing method provided by yet another embodiment of the present application is shown;

[0020] Figure 9 A flowchart of an image processing method provided by another embodiment of the present application is shown;

[0021] Figure 10 A schematic diagram showing a timeline of image processing provided by yet another embodiment of the present application is shown;

[0022] Figure 11 A schematic diagram showing a timeline of another image processing method provided by yet another embodiment of the present application is shown;

[0023] Figure 12 A flowchart of an image processing method applied to a camera according to an embodiment of the present application is shown;

[0024] Figure 13 A schematic diagram showing a process of image processing by a camera provided in an embodiment of the present application is shown;

[0025] Figure 14 A structural block diagram of an image processing device provided in an embodiment of the present application is shown;

[0026] Figure 15 A structural block diagram of an electronic device provided in an embodiment of the present application is shown;

[0027] Figure 16 A block diagram of a computer-readable storage medium is shown in an embodiment of the present application;

[0028] Figure 17 A structural block diagram of a computer program product provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work fall within the scope of protection of the present application.

[0030] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0031] With current technology, for example, cameras capture all the image data needed for each photo before performing algorithm post-processing. For example, with a multi-frame image algorithm, if four frames are needed, the data for all four frames must be captured and then packaged together for image processing.

[0032] like Figure 1 As shown in the figure, after clicking the photo button, four frames of images are captured. When the fourth frame is captured, all four frames are processed. However, no image processing is performed between the time the photo button is clicked and the fourth frame is received. This results in a period of low load during the initial phase, which underutilizes the phone's hardware resources. All post-processing algorithms are deferred to the latter phase, resulting in slower image generation.

[0033] During their research, the inventors discovered that, before receiving the mth frame, they could pre-process at least one image frame prior to the mth frame. Then, upon receiving the mth frame, all m frames of image processing could be performed based on the results of this pre-processing. To address this issue, embodiments of the present application provide an image processing method, apparatus, electronic device, and storage medium that can pre-process images acquired prior to the mth frame before acquiring all m frames, thereby increasing the speed of image processing.

[0034] The image processing method, device, electronic device and storage medium provided in the embodiments of the present application will be described in detail below through specific embodiments.

[0035] See also Figure 2 , Figure 2 A flowchart of an image processing method provided by an embodiment of the present application is shown. Optionally, the image processing method shown in the present application can be applied to an electronic device having an image acquisition function. Specifically, the method may include: S210 to S230.

[0036] S210: When an image acquisition operation is detected, m frames of target images are acquired frame by frame, where m is a positive integer.

[0037] An image acquisition operation can be used to trigger image acquisition. Optionally, the image acquisition operation can be initiated by the user. In this case, the image acquisition operation can be a user clicking an image acquisition button, such as a user clicking a photo button. Alternatively, the image acquisition operation can be automatically initiated by the electronic device. In this case, the image acquisition operation can be initiated by the electronic device through an acquisition instruction. For example, if a user sets the camera to take a timed photo at a preset time, an acquisition instruction can be issued at the preset time to instruct the camera to begin capturing images, indicating that the image acquisition operation has been detected.

[0038] In an embodiment of the present application, upon detecting an image acquisition operation, m frames of target images can be acquired frame by frame. When an image acquisition operation is detected, the camera of the electronic device is activated and the camera module is called to capture an image. The camera controls the camera module to capture images of the current scene frame by frame. Each frame of the image acquired for the current scene can be used as the target image.

[0039] Optionally, the number m of target images acquired frame by frame can be set by the user, for example, the user sets the number of images to be taken before clicking the photo button. Optionally, the number m of target images acquired frame by frame can also be determined based on the image algorithm required for the current image acquisition operation. The image algorithm can be a multi-frame image algorithm, which can simultaneously process at least one frame of image to obtain an image. It can be seen that the number m of target images acquired frame by frame can be the number of images that can be processed simultaneously by the multi-frame image algorithm required for the current image acquisition operation. Among them, the multi-frame image algorithm can, for example, be a beauty processing algorithm, a blur processing algorithm, a high dynamic range image (HDR) processing algorithm, a filter processing algorithm, a rotation processing algorithm, a watermark processing algorithm, and the like.

[0040] Among them, each frame of target image acquired can be a YUV image. In the current image acquisition method, the image acquired by the current scene is first generated into a RAW image, and then the RAW image is converted into a YUV image. These two operations are performed serially, that is, when the image acquisition operation is detected, if it is necessary to acquire YUV images frame by frame, that is, after acquiring a frame of YUV image, another frame of YUV image is acquired, it is necessary to reacquire a frame of RAW image, and then perform the conversion operation of converting the reacquired frame of RAW image into a YUV image, so that a YUV image can be obtained. At this time, each frame of YUV image acquired can be used as a frame of target image. Then, the subsequent steps in the embodiment of the present application can be used to perform image post-processing on these target images.

[0041] S220: When the m-th frame of the target image is acquired, a first processing result is acquired, where the first processing result includes a result obtained by performing a first preset processing on at least one frame of the target image before the m-th frame.

[0042] It is understandable that when the m-th frame of the target image is acquired, all m-frame target images have been acquired at this time, and when the m-th frame of the target image is acquired, the first processing result acquired may include the result obtained by pre-presetting the at least one frame of the target image before the m-th frame. That is to say, the embodiment of the present application may perform the first preset processing on at least one frame of the target image acquired before the m-th frame of the target image before acquiring all m-frame images. In the embodiment of the present application, if Figure 3 As shown, the time for performing the first preset processing on at least one frame of target image acquired before the mth frame of target image needs to be before the mth frame of target image is acquired, and the specific time before the mth frame of target image is acquired is not limited in the embodiment of the present application.

[0043] Exemplarily, the first preset processing is performed in advance on at least one frame of target image before the mth frame, which can be any of the following situations: Case 1, when the first frame of target image is acquired, the first preset processing can be performed on the first frame of target image; Case 2, when any frame of target image from the 2nd frame to the m-1th frame of target image is acquired, the first preset processing can be performed on the target image acquired this time or at least one frame of target image before the target image acquired this time; Case 3, every time a frame of target image before the mth frame of target image is acquired, the first preset processing can be performed on the currently acquired frame of target image.

[0044] It should be noted that the above-mentioned first preset processing can correspond to the image acquisition operation. For example, if the image acquisition operation is detected and the image algorithm to be used in the current image acquisition operation has been determined, the first preset processing can be a processing operation in the image algorithm that processes each frame of the target image separately, or a processing operation that processes other frame target images other than the last m-th frame target image. Since the first preset processing can process one or more frames of target images other than the m-th frame (i.e., at least one frame of target image) without having to be processed simultaneously with the m-th frame target image, the first preset processing can be advanced to before the acquisition of the m-th frame target image. When the m-th frame target image is acquired, the result obtained by performing the first preset processing on at least one frame of target image before the m-th frame can be acquired.

[0045] Optionally, in addition to the result obtained by performing the first preset processing on at least one target image frame before the m-th frame, the first processing result may also include a result obtained by performing the first preset processing on the m-th target image frame. It is understood that the first preset processing may be performed on the m-th target image frame only after the m-th target image frame is acquired.

[0046] It should be noted that the first preset processing can be used to adjust the image, and the first processing result can be the image obtained after the first preset processing is performed on at least one frame of target image before the mth frame, and the corresponding adjustment; the first preset processing can also be used to extract image data from the image, for recognition results of image recognition, etc., that is, the first processing result is not limited to an image, but can also be image data, recognition results, etc. obtained after the first preset processing is performed on at least one frame of target image before the mth frame.

[0047] S230: Based on the first processing result, perform a second preset processing on the m frames of target image to obtain a second processing result.

[0048] In the embodiment of the present application, in addition to the first preset processing, a second preset processing is also required for all m frames of target images. It is understood that when performing the second preset processing, the result obtained by the first preset processing, i.e., the first processing result, is used. In other words, based on the first processing result, the second preset processing can be performed on all m frames of target images to obtain the second processing result.

[0049] In some embodiments, the second preset processing may also correspond to an image acquisition operation. If an image acquisition operation is detected and an image algorithm is determined to be used for the image acquisition operation, the second preset processing may optionally be a processing operation in the image algorithm that simultaneously processes all m frames of the target image based on the first processing result.

[0050] Similar to the first processing result, the second processing result can be an image obtained by performing a second preset processing on m frames of target images based on the first processing result, or it can be image data, recognition results, etc. obtained by performing a second preset processing on m frames of target images based on the first processing result.

[0051] Taking the multi-frame beautification algorithm as an example, the first preset processing can be facial feature extraction, and the second preset processing can be multi-frame noise reduction and beautification processing, etc. Therefore, the first processing result can include the target facial features obtained by pre-extracting facial features from at least one target frame before the mth frame. Then, when the mth target frame is acquired, multi-frame noise reduction is performed on all m target frames to obtain a single denoised image. This denoised image is then beautified based on the target facial features to obtain a beautified image. This beautified image is then processed as the second processing result.

[0052] In an embodiment of the present application, when an image acquisition operation is detected, m frames of target images can be acquired frame by frame, where m is a positive integer. When the m-th frame of target image is acquired, a first processing result is acquired, and the first processing result includes the result obtained by performing a first preset processing on at least one frame of target image before the m-th frame. Then, based on the first processing result, a second preset processing is performed on all m frames of target image to obtain a second processing result. In an embodiment of the present application, when acquiring the m-th frame of target image, the result of performing a first preset processing on at least one frame of target image before the m-th frame can be obtained, and then based on this result, a second preset processing is performed on all m frames of image. In this way, the images acquired before the m-th frame can be processed in advance before all m frames of target image are acquired, thereby improving the speed of image processing.

[0053] See also Figure 4 , Figure 4 A flowchart of an image processing method provided by another embodiment of the present application is shown. Based on the above embodiment, this embodiment of the present application can, when acquiring a first frame of target image, perform a first preset processing on the first frame of target image to obtain a first processing result. Specifically, the method may include: S410 to S440.

[0054] S410: When an image acquisition operation is detected, m frames of target images are acquired frame by frame, where m is a positive integer.

[0055] S420: When the first frame of the target image is acquired, a first preset processing is performed on the first frame of the target image to obtain a first processing result.

[0056] S430: When the m-th frame of the target image is obtained, the first processing result is obtained.

[0057] S440: Based on the first processing result, perform a second preset processing on the m frames of target image to obtain a second processing result.

[0058] In some embodiments, the first processing result may be a result obtained by performing a first preset processing on the first frame of the target image when acquiring the first frame of the target image. Figure 5 As shown, when a click-to-photograph operation is detected, m target images are acquired frame by frame. When the first target image is acquired, a first preset processing is performed on the first target image to obtain a first processing result. Then, the second target image, the third target image, and so on are acquired until the mth target image is acquired. Based on the first processing result, a second preset processing can be performed on all m target images to obtain a second processing result. For example, if the first preset processing is facial feature extraction, the first processing result can be the target facial features obtained by performing facial feature extraction on the first target image when the first target image is acquired. Then, when the mth target image is acquired, multi-frame denoising is performed on all m target images to obtain a denoised image. This denoised image is then beautified based on the target facial features corresponding to the first target image to obtain a beautified image, which can be the second processing result.

[0059] Based on this, when the first frame of the target image is acquired, the embodiment of the present application performs a first preset processing on the first frame of the target image to obtain a first processing result. Then, when the mth frame of the target image is acquired, the second preset processing is performed on all m frames of the target image based on the first processing result to obtain a second processing result. This allows the images acquired before the mth frame to be processed in advance before all m frames of the target image are acquired. This allows the time when the first frame of the image is acquired, during which resource utilization is relatively low, to perform some algorithm preprocessing, thereby increasing the speed of image processing.

[0060] See also Figure 6 , Figure 6 A flowchart of an image processing method according to another embodiment of the present application is shown. Based on any of the preceding embodiments, when acquiring an nth target image frame, this embodiment of the present application can perform a first preset process on the nth target image frame and at least one target image frame before the nth target image frame to obtain a first processing result, where n is greater than 1 and less than m, and n is a positive integer. Specifically, the method can further include: S610 to S640.

[0061] S610: When an image acquisition operation is detected, m frames of target images are acquired frame by frame, where m is a positive integer.

[0062] S620: When the nth frame of the target image is acquired, perform a first preset processing on the nth frame and at least one frame of the target image before the nth frame to obtain the first processing result, where n is greater than 1 and n is less than m, and n is a positive integer.

[0063] S630: When the m-th frame of the target image is obtained, the first processing result is obtained.

[0064] S640: Based on the first processing result, perform a second preset processing on the m frames of target image to obtain a second processing result.

[0065] In this embodiment, the first processing result may be the result of performing the first preset processing on the nth frame and at least one frame of the target image before the nth frame when the nth frame of the target image is acquired. Here, n can be any positive integer between 2 and m-1. Alternatively, n can be set by the user, for example, by the user enabling pre-processing to be performed when capturing the nth image before clicking the capture button. Alternatively, n can be set by the system, for example, based on the number of images required for the first preset processing.

[0066] Optionally, the at least one target image frame before and including the nth frame may be every target image frame including the nth frame. That is, the first processing result may be the result of performing the first preset processing on every image frame before and including the nth frame when the nth target image frame is acquired. Optionally, the at least one target image frame before and including the nth frame may be any one or two frames before and including the nth frame, and so on.

[0067] For example, if n is 2, that is, when the second frame of the target image is acquired, the first preset processing can be performed on the second frame and at least one frame of the target image before the second frame to obtain the first processing result. Figure 7 As shown; it may also be that when the second frame target image is acquired, the first frame and the second frame target image are subjected to a first preset processing to obtain a first processing result, such as Figure 8 shown.

[0068] Similarly, if n is 3, that is, when the third target image frame is acquired, the first preset processing can be performed on the third frame and at least one target image frame before the third frame to obtain a first processing result. Specifically, when the third target image frame is acquired, the first preset processing can be performed on any one of the first, second, or third frames to obtain the first processing result; or when the third target image frame is acquired, the first preset processing can be performed on any two of the first, second, and third frames to obtain the first processing result; or when the third target image frame is acquired, the first preset processing can be performed on the first, second, and third frames to obtain the first processing result.

[0069] It can be understood that when n is any positive integer between 2 and m-1, the first processing result can be obtained in a manner similar to the above example, and the embodiments of the present application will not be repeated here.

[0070] Optionally, if the first preset processing is performed on two or more target images respectively, multiple processing results can be obtained, and each of the above processing results can be used as the first processing result. Exemplarily, if the first preset processing is facial feature extraction, the first processing result can be the target facial feature obtained by performing facial feature extraction on the nth frame and at least one frame of target image before the nth frame when the nth frame of target image is acquired, wherein n can be any positive integer between 2 and m-1. The first processing result may include multiple target facial features, so after performing multi-frame denoising on m frames of target images to obtain a denoised image, the denoised image can be beautified according to the multiple target facial features to obtain a beautified image, and the beautified image can be the second processing result.

[0071] Based on this, when the nth frame of the target image is acquired, the embodiment of the present application can perform a first preset processing on the nth frame and at least one target frame before the nth frame to obtain a first processing result, where n is greater than 1 and less than m, and n is a positive integer. This allows the images acquired before the mth frame to be processed in advance before all m frames of the target image are acquired. The period between the acquisition of the first and mth frames, when resource utilization is relatively low, can be used to perform some algorithm preprocessing, thereby increasing the speed of image processing.

[0072] See also Figure 9 , Figure 9A flowchart of an image processing method provided by yet another embodiment of the present application is provided. Based on any of the preceding embodiments, each time a target image frame preceding the mth target image frame is acquired, a first preset processing is performed on the currently acquired target image frame to obtain a processing result for each target image frame. The processing result for each target image frame can then be used as the first processing result. Specifically, the method can further include: S910 to S950.

[0073] S910: When an image acquisition operation is detected, m frames of target images are acquired frame by frame, where m is a positive integer.

[0074] S920: Whenever a target image frame before the m-th target image frame is acquired, a first preset process is performed on the currently acquired target image frame to obtain a processing result for each target image frame.

[0075] S930: Using the processing results of each frame of the target image as the first processing result.

[0076] S940: When the m-th frame of the target image is obtained, the first processing result is obtained.

[0077] S950: Based on the first processing result, perform a second preset processing on the m frames of target image to obtain a second processing result.

[0078] In some embodiments, the first processing result may also be each processing result obtained by performing the first preset processing on the currently acquired frame of target image each time a frame of target image before the mth frame of target image is acquired. Figure 10 As shown, when a click-to-photograph operation is detected, m frames of target images are acquired frame by frame. Each time a frame of target image is acquired, the first preset processing is performed on the frame to obtain a processing result for each frame of target image. For example, when the first frame of target image is acquired, the first preset processing is performed on the frame to obtain a processing result for the first frame of target image. Furthermore, when the second frame of target image is acquired, the first preset processing is performed on the second frame to obtain a processing result for the second frame of target image. Similarly, when the m-1th frame of target image is acquired, the first preset processing is performed on the m-1th frame to obtain a processing result for the m-1th frame of target image. This allows the processing result of each frame of target image from the first to the m-1th frame to be obtained, and the processing result of each frame of target image can be further used as the first processing result.

[0079] Optionally, the first processing result may also include a result obtained by performing a first preset processing on the m-th frame target image. Figure 11As shown, it can be seen that the first processing result can also be each processing result obtained by performing the first preset processing on the currently acquired target image frame each time one of the m target image frames is acquired. When the mth target image frame is acquired, the first preset processing has already been performed on the 1st, 2nd, ..., and m-1th target images. At this time, the first preset processing can be performed on the mth target image frame to obtain a first processing result including the processing results of each of the m target image frames. Then, based on the first processing result, the second preset processing can be performed on all m target image frames to obtain a second processing result.

[0080] Exemplarily, if the first preset processing is facial feature extraction, the first processing result may include target facial features obtained by performing facial feature extraction on each target frame acquired. Next, when the mth target image is acquired, facial feature extraction may be performed on the mth target image to obtain target facial features corresponding to the mth target image. In this case, the first processing result may also include target facial features corresponding to the mth target image. Multi-frame denoising is then performed on all m target images to obtain a denoised image. This denoised image is then beautified based on each target facial feature in the first processing result to obtain a beautified image. This beautified image may serve as the second processing result.

[0081] Based on this, the embodiment of the present application can perform a first preset processing on the currently acquired target image frame each time a target image frame before the m-th target image frame is acquired, to obtain a processing result for each target image frame, and then use the processing result of each target image frame as the first processing result. Then, when the m-th target image frame is acquired, a second preset processing is performed on all m target image frames based on the first processing result to obtain a second processing result. In this way, the images acquired before the m-th frame can be processed in advance before all m target image frames are acquired, and the time between the acquisition of the first frame image and the acquisition of the m-th frame image, when resource utilization is relatively low, can be used to perform some algorithm preprocessing, which can improve the speed of image processing.

[0082] In some embodiments of the present application, the methods described in the embodiments of the present application may be applied to a camera, where the camera may include a hardware abstraction layer (HAL), an application program (APP), and an image post-processing module (APS). In this embodiment, upon detecting an image acquisition operation on the camera, the application program module may be used to acquire m frames of target images from the HAL, frame by frame.

[0083] See also Figure 12 , Figure 12 A flowchart of an image processing method for a camera according to an embodiment of the present application is shown. Based on any of the aforementioned embodiments, this method can be applied to a camera, which may include a hardware abstraction module, an application module, and an image post-processing module. Specifically, the method may further include: S1210 to S1230.

[0084] S1210: When the image acquisition operation acting on the camera is detected, use the application module to acquire m frames of target images from the hardware abstraction module frame by frame.

[0085] In an embodiment of the present application, upon detecting an image acquisition operation acting on a camera, the camera can control the image sensor to capture images of the current scene frame by frame and generate RAW images frame by frame based on the currently captured images. The hardware abstraction module can then generate a YUV image frame based on each RAW image. The application module then uses the hardware abstraction module to obtain m frames of target images frame by frame from the hardware abstraction module, where the target image can be the aforementioned YUV image. That is, each time the application module obtains a YUV image frame from the hardware abstraction module, the image sensor first captures a frame of the current scene and generates a RAW image frame. The hardware abstraction module then generates a YUV image frame based on this RAW image frame. The application module then obtains this YUV image frame from the hardware abstraction module and uses it as the target image frame. This process of acquiring YUV images frame by frame can be referred to as a frame buffer, and this frame buffering process can be repeated to obtain m frames of target images.

[0086] S1220: When it is detected that the application module obtains the m-th frame target image, a call instruction is sent to the image post-processing module through the application module, and the call instruction is used to instruct the image post-processing module to obtain the first processing result. The call instruction is also used to instruct the image post-processing module to take out a processing instruction from the thread queue, and the processing instruction is used to determine the second preset processing based on the m-frame target image.

[0087] In an embodiment of the present application, a thread queue (Process Queue Q) may include multiple processing instructions, each corresponding to a frame of a target image. The processing instruction may be used to indicate whether a first preset processing should be performed on the frame of the target image corresponding to the processing instruction, as well as which image processing method is determined as the first preset processing. The thread queue may also include a processing instruction for instructing the performance of a second preset processing on all m frames of the target image. Since the target images are acquired frame by frame, the processing instructions may be stored in the thread queue in the order in which the target images were acquired.

[0088] For example, taking the beautification processing algorithm as an example, if it is necessary to extract facial features of the first frame target image to obtain the target facial features, and then after obtaining all m frames of target images, perform multi-frame noise reduction and beautification processing on all target images based on the target facial features, according to the image processing method shown in the embodiment of the present application, the image processing method indicated by the processing instruction corresponding to the first frame target image in the thread queue can be determined as facial feature extraction, and the image processing method indicated by the processing instructions corresponding to all m frames of target images can be determined as multi-frame noise reduction and beautification processing.

[0089] Therefore, during the image processing process of the camera, when the application module obtains the first frame of the target image, a call instruction is sent to the image post-processing module through the application module. The call instruction at this time can instruct the image post-processing module to take out the processing instruction corresponding to the first frame of the target image from the thread queue, that is, Figure 13 The processing instruction at the head of the thread queue shown enables the image post-processing module to extract facial features from the target image of the first frame according to the first preset processing determined by the processing instruction, that is, the processing method of facial feature extraction, when receiving the processing instruction, to obtain the target facial features. Then, in this embodiment, since facial feature extraction is not required for the second frame, the third frame, and the mth frame, the processing instructions corresponding to the second frame, the third frame, and the mth frame in the thread queue can be empty, that is, they do not point to any image processing method. Therefore, when the target images of the second frame, the third frame, and the mth frame are obtained, the image post-processing algorithm will not extract facial features from these target images. When the target image of the mth frame is obtained, a call instruction can be sent to the image post-processing module through the application module. At this time, the call instruction can not only instruct the image post-processing module to take out the processing instruction from the thread queue to instruct multi-frame noise reduction and beauty processing for all m frames of target images, that is, Figure 13 The processing instruction at the tail (end) of the thread queue in the processing instruction is received, so that when the image post-processing module receives the processing instruction, it can first perform multi-frame denoising on all m frames of target images to obtain one frame of denoised image according to the second preset processing determined by the processing instruction, that is, the processing mode of multi-frame denoising and beautification processing, and then perform beautification processing on the one frame of denoised image based on the target facial features to obtain a beautified image.

[0090] It will be understood that the aforementioned method of performing the first preset processing on the first target image frame only upon acquisition of the first target image frame is merely exemplary. In the embodiments of the present application, since the first processing result includes the result obtained by pre-processing at least one target image frame prior to the mth frame, the specific frame or frames prior to the mth frame to which the first preset processing is performed, as well as the time point prior to acquisition of the mth target image frame, can be determined by setting the image processing method indicated by the processing instruction corresponding to each target image frame in the thread queue according to specific needs. The image processing method can indicate the time before acquisition of the mth target image frame and the type of first preset processing that must be performed on the target images prior to the mth frame in order to obtain the first processing result. Upon sequentially receiving the processing instruction corresponding to each target image frame, the image post-processing module can execute the operation corresponding to the processing instruction through a work routine, namely, pre-processing the at least one target image frame prior to acquisition of the mth target image frame to obtain the first processing result.

[0091] S1230: When it is detected that the image post-processing module receives the processing instruction, the image post-processing module performs the second preset processing on the m frames of target images based on the first processing result to obtain the second processing result.

[0092] It is understood that the processing instruction received by the image post-processing module at this time is an instruction for instructing the image post-processing module to perform the second preset processing on all m frames of target images. Therefore, when it is detected that the image post-processing module receives the processing instruction, please refer to Figure 13 , the operation corresponding to the processing instruction can be executed by the working routine in the image post-processing module, that is, the second preset processing is performed on all m frames of target images based on the first processing result to obtain a second processing result.

[0093] It can be seen that when clicking to take a photo, obtaining each frame of the target image, and obtaining all m frames of the target image, the application module will send a call instruction to the image post-processing module, and then the work routine in the image post-processing module will take out the processing instructions from the thread queue in turn for execution.

[0094] Based on this, when the application module detects that the mth frame of the target image has been acquired, the present solution can send a call instruction to the image post-processing module through the application module. The call instruction is used to instruct the image post-processing module to obtain the first processing result, and the first processing result includes the result of performing a first preset processing on at least one frame of the target image before the mth frame. The call instruction can also be used to instruct the image post-processing module to retrieve a processing instruction from the thread queue, and the processing instruction is used to determine the second preset processing based on the mth frame of the target image. Therefore, when the image post-processing module receives the processing instruction, the image post-processing module can perform the second preset processing on all m frames of the target image based on the first processing result to obtain the second processing result. In this embodiment of the present application, before the camera application module acquires all m frames of the target image, the image post-processing module can pre-process the images acquired before the mth frame. Therefore, the period between the acquisition of the first frame and the mth frame, when resource utilization is relatively low, can be used to perform some algorithm pre-processing, thereby rationally utilizing hardware resources, reducing the risks brought about by concentrated resource use, and effectively reducing the time of a single photo.

[0095] See also Figure 14 , which shows a structural block diagram of an image processing device provided in an embodiment of the present application. The device may include: a first acquisition unit 1410, a second acquisition unit 1420 and an image processing unit 1430.

[0096] The first acquisition unit 1410 is configured to acquire m frames of target images frame by frame when an image acquisition operation is detected, where m is a positive integer.

[0097] The second acquiring unit 1420 is configured to acquire a first processing result when acquiring the m-th frame of the target image, where the first processing result includes a result obtained by performing a first preset processing on at least one frame of the target image before the m-th frame;

[0098] The image processing unit 1430 is configured to perform a second preset processing on the m frames of target image based on the first processing result to obtain a second processing result.

[0099] Optionally, the device further includes: a first processing unit, configured to perform a first preset processing on the first frame of target image when the first frame of target image is acquired, to obtain the first processing result.

[0100] Optionally, the device also includes: a second processing unit, used to perform a first preset processing on the nth frame and at least one frame of target image before the nth frame when the nth frame target image is acquired, to obtain the first processing result, wherein n is greater than 1 and n is less than m, and n is a positive integer.

[0101] Optionally, the device also includes: a third processing unit, which is used to perform a first preset processing on a currently acquired frame of target image each time a frame of target image before the mth frame of target image is acquired, to obtain a processing result for each frame of target image, and to use the processing result of each frame of target image as the first processing result.

[0102] In some implementations, the first processing result further includes a result obtained by performing the first preset processing on the m-th frame target image.

[0103] Optionally, the above-mentioned device can be applied to a camera, which includes a hardware abstraction module, an application module and an image post-processing module. The above-mentioned device may also include: a frame-by-frame acquisition unit, which is used to use the application module to acquire m frames of target images frame by frame from the hardware abstraction module when the image acquisition operation acting on the camera is detected.

[0104] Furthermore, based on the above embodiments, the above-mentioned device may also include: a fourth processing unit, which is used to send a call instruction to the image post-processing module through the application module when it is detected that the application module obtains the m-th frame target image, and the call instruction is used to instruct the image post-processing module to obtain the first processing result. The call instruction is also used to instruct the image post-processing module to take out a processing instruction from the thread queue, and the processing instruction is used to determine the second preset processing based on the m-frame target image; a fifth processing unit, which is used to perform the second preset processing on the m-frame target image based on the first processing result through the image post-processing module when it is detected that the image post-processing module receives the processing instruction, to obtain the second processing result.

[0105] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0106] In several embodiments provided in this application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0107] In addition, the functional modules in the various embodiments of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into a single module. The above-mentioned integrated modules may be implemented in the form of hardware or software functional modules.

[0108] Please refer to Figure 15, which shows a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device 1500 can be an electronic device such as a smart phone, a tablet computer, a laptop computer, a camera, a video recorder, etc. that can run applications and has an image acquisition function. Specifically, the electronic device 1500 may include one or more of the following components: a processor 1510, a memory 1520, and one or more applications, wherein the one or more applications can be stored in the memory 1520 and configured to be executed by one or more processors 1510, and the one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0109] The electronic device may be any of various types of computer system devices that are mobile, portable, and capable of wireless communication. Specifically, the electronic device may include a mobile phone or smartphone (e.g., an iPhone™-based or Android™-based phone), a portable gaming device (e.g., a Nintendo DS™, a PlayStation Portable™, a Gameboy Advance™, an iPhone™), a laptop computer, a PDA, a portable internet device, a music player, a data storage device, other handheld devices, and devices such as smart watches, smart bracelets, headphones, and pendants. The electronic device may also include other wearable devices (e.g., electronic glasses, electronic clothing, electronic bracelets, electronic necklaces, electronic tattoos, electronic devices, or head-mounted devices (HMDs)), smart home devices, and in-vehicle devices.

[0110] The electronic device may also be any one of a plurality of electronic devices, including but not limited to cellular phones, smart phones, smart watches, smart bracelets, other wireless communication devices, personal digital assistants, audio players, other media players, music recorders, video recorders, cameras, other media recorders, radios, medical devices, vehicle transportation instruments, calculators, programmable remote controls, pagers, laptop computers, desktop computers, printers, netbook computers, personal digital assistants (PDAs), portable multimedia players (PMPs), Moving Picture Experts Group (MPEG-1 or MPEG-2) Audio Layer 3 (MP3) players, portable medical devices, and digital cameras, and combinations thereof.

[0111] In some cases, the electronic device can also perform multiple functions (e.g., play music, display video, store pictures, and receive and send phone calls). If desired, the electronic device can be, for example, a cellular phone, a media player, other handheld device, a wristwatch device, a pendant device, an earpiece device, or other compact portable device.

[0112] The processor 1510 may include one or more processing cores. The processor 1510 utilizes various interfaces and circuits to connect various components within the electronic device 1500. It executes instructions, programs, code sets, or instruction sets stored in the memory 1520, and accesses data stored in the memory 1520 to perform various functions and process data within the electronic device 1500. Optionally, the processor 1510 may be implemented in hardware using at least one of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 1510 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem handles wireless communications. It is understood that the modem may also be implemented independently of the processor 1510 via a separate communications chip.

[0113] Memory 1520 may include random access memory (RAM) or read-only memory (ROM). Memory 1520 may be used to store instructions, programs, code, code sets, or instruction sets. Memory 1520 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), and instructions for implementing the various method embodiments described below. The data storage area may also store data created by the electronic device during use (such as a phone book, audio and video data, and chat history data).

[0114] Please refer to Figure 16 , which shows a block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 1600 stores program code, which can be called by a processor to execute the method described in the above method embodiment.

[0115] Computer-readable storage medium 1600 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, a hard disk, or ROM. Alternatively, computer-readable storage medium 1600 may include a non-transitory computer-readable storage medium. Computer-readable storage medium 1600 has storage space for program code 1610 for executing any of the method steps described above. This program code can be read from or written to one or more computer program products. Program code 1610 may be compressed, for example, in a suitable format.

[0116] Please refer to Figure 17 , which shows a structural block diagram of a computer program product provided in an embodiment of the present application. The computer program product 1700 stores a computer program / instruction 1710. When it is run on a computer, the computer program / instruction 1710 performs any method steps in the above method.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: When an image acquisition operation is detected, m frames of target images are acquired frame by frame, where m is a positive integer, and the target image is each frame of the image acquired by the camera module for the current scene frame by frame; When the m-th frame of the target image is acquired, a first processing result is acquired, wherein the first processing result includes a result obtained by performing a first preset processing on at least one frame of the target image before the m-th frame of the target image is acquired; Based on the first processing result, a second preset processing is performed on all m frames of target images to obtain a second processing result.

2. The method according to claim 1, characterized in that When the m-th frame of the target image is acquired, before acquiring the first processing result, the method further includes: When the first frame of the target image is acquired, the first preset processing is performed on the first frame of the target image to obtain the first processing result.

3. The method according to claim 1, characterized in that When the m-th frame of the target image is acquired, before acquiring the first processing result, the method further includes: When the nth frame target image is acquired, the first preset processing is performed on the nth frame and at least one frame target image before the nth frame to obtain the first processing result, where n is greater than 1 and n is less than m, and n is a positive integer.

4. The method according to claim 1, wherein When the m-th frame of the target image is acquired, before acquiring the first processing result, the method further includes: Whenever a target image frame before the m-th target image frame is acquired, a first preset processing is performed on the currently acquired target image frame to obtain a processing result of each target image frame; The processing result of each frame of the target image is used as the first processing result.

5. The method according to claim 1, wherein The first processing result also includes a result obtained by performing the first preset processing on the m-th frame target image.

6. The method according to claim 1, characterized in that The method is applied to a camera, which includes a hardware abstraction module, an application module, and an image post-processing module. When an image acquisition operation is detected, m frames of target images are acquired frame by frame, including: When the image acquisition operation acting on the camera is detected, the application module is used to acquire m frames of target images from the hardware abstraction module frame by frame.

7. The method according to claim 6, characterized in that When the mth frame of the target image is acquired, obtaining a first processing result includes: When it is detected that the application module has acquired the m-th frame of the target image, a calling instruction is sent to the image post-processing module through the application module, wherein the calling instruction is used to instruct the image post-processing module to acquire the first processing result; The calling instruction is further used to instruct the image post-processing module to take out a processing instruction from a thread queue, wherein the processing instruction is used to determine the second preset processing according to the m frames of target images; The step of performing a second preset processing on all m frames of target images based on the first processing result to obtain a second processing result includes: When it is detected that the image post-processing module receives the processing instruction, the image post-processing module performs the second preset processing on all m frames of target images based on the first processing result to obtain the second processing result.

8. An image processing device, characterized in that: The device comprises: A first acquisition unit is configured to acquire, when an image acquisition operation is detected, m frames of target images frame by frame, where m is a positive integer and the target image is each frame of image acquired by the camera module for the current scene frame by frame; a second acquiring unit, configured to acquire a first processing result when the m-th frame of the target image is acquired, wherein the first processing result includes a result obtained by performing a first preset processing on at least one frame of the target image before the m-th frame of the target image is acquired; The image processing unit is used to perform a second preset processing on all m frames of target images based on the first processing result to obtain a second processing result.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the method according to any one of claims 1 to 7 by calling the computer program.

10. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the method according to any one of claims 1 to 7.

11. A computer program product comprising instructions, characterized in that The computer program product stores instructions, and when the computer program product is executed, the computer is caused to implement the method according to any one of claims 1 to 7.

Images