Image processing method and device, electronic equipment and readable storage medium

Abstract

The invention discloses an image processing method and device, electronic equipment and a readable storage medium, and belongs to the technical field of image processing. The method comprises the steps that image quality detection is conducted on N frames of first image data, the N frames of first image data are N frames of preview image data or N frames of video image data corresponding to N frames of image data obtained in the video recording process, and N is a positive integer; and under the condition that the N frames of first image data have the image quality problem, performing optimization processing on the N frames of video image data by adopting an optimization algorithm corresponding to the image quality problem.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to an image processing method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] With the development of image processing technology, users have increasingly higher requirements for the image quality of video recordings. Therefore, after electronic devices acquire multiple consecutive frames of image data through sensors, they also need to use optimization algorithms to optimize these frames of image data in order to obtain video recordings with higher image quality.

[0003] In current optimization processes, preset optimization algorithms are typically used to optimize image quality, such as color correction or improved sharpness. This results in electronic devices processing the recorded image data regardless of whether there are corresponding image quality issues, thus increasing the power consumption of the electronic device and causing wasted power. Summary of the Invention

[0004] The purpose of this application is to provide an image processing method, apparatus, electronic device, and readable storage medium that can reduce the power consumption of electronic devices when processing image data.

[0005] In a first aspect, embodiments of this application provide an image processing method, which includes: performing image quality detection on N frames of first image data, wherein the N frames of first image data are: N frames of preview image data corresponding to the N frames of image data obtained during recording, or N frames of recorded image data, where N is a positive integer; and when there are image quality problems in the N frames of first image data, using an optimization algorithm corresponding to the image quality problem to optimize the N frames of recorded image data.

[0006] Secondly, embodiments of this application provide an image processing apparatus, which includes a detection module and a processing module. The detection module is used to perform image quality detection on N frames of first image data, wherein the N frames of first image data are: N frames of preview image data corresponding to the N frames of image data acquired during recording, or N frames of recorded image data, where N is a positive integer. The processing module is used to optimize the N frames of recorded image data by employing an optimization algorithm corresponding to the image quality problem when there is an image quality problem in the N frames of first image data.

[0007] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0008] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0009] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0010] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0011] In this embodiment, the electronic device performs image quality detection on N frames of first image data. The N frames of first image data are either N frames of preview image data corresponding to the N frames of image data acquired during recording, or N frames of recorded image data, where N is a positive integer. If there are image quality issues with the N frames of first image data, an optimization algorithm corresponding to the image quality issue is used to optimize the N frames of recorded image data. In this solution, the electronic device performs pre-detection on the preview image data or recorded image data to identify image quality issues in the recorded image data. Then, it uses a targeted optimization algorithm corresponding to the image quality issue to optimize the recorded image data. This improves the image quality of the optimized recorded image data and reduces the power consumption of processing the recorded image data. Attached Figure Description

[0012] Figure 1 This is a schematic flowchart of a conventional image processing method provided in an embodiment of this application;

[0013] Figure 2 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0014] Figure 3 This is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0015] Figure 4 This is a schematic diagram of another image processing method provided in an embodiment of this application;

[0016] Figure 5 This is a schematic diagram of zoom magnification provided in an embodiment of this application;

[0017] Figure 6 This is a schematic diagram of a color gain correction and optimization process provided in an embodiment of this application;

[0018] Figure 7This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application;

[0019] Figure 8 This is one of the hardware structure diagrams of an electronic device provided in the embodiments of this application;

[0020] Figure 9 This is a second schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0022] The following explains some terms and nouns used in the embodiments of this application.

[0023] 1) Preview data stream: refers to the continuous output of image data captured by the camera sensor to the screen with low latency and (usually) low resolution / quality.

[0024] 2) Video data stream: refers to the data stream in which the camera compresses and writes the image data captured by the sensor into a storage device (such as the memory of an electronic device or an SD card) after a series of complex processing and encoding.

[0025] 3) How the preview data stream and video recording data stream work:

[0026] The preview stream and the recording stream are not completely independent; they work closely together.

[0027] Shared sensor: Both originate from data captured by the same image sensor.

[0028] Parameter synchronization: The effects seen in the preview (exposure, white balance, focus) are ultimately applied directly to the recorded video stream. The electronic device receives user input adjusting parameters based on the preview and uses these parameters to adjust the recorded video accordingly.

[0029] Simultaneous: such as Figure 1 As shown, when the electronic device receives a recording input, it starts recording video. At this time, the camera of the electronic device is actually processing two data streams simultaneously, namely the preview data stream and the recording data stream mentioned above.

[0030] One of the image data streams uses a fast channel to ensure smooth real-time preview on the screen.

[0031] Another image data stream goes through a fine-processing channel of the video data stream to generate a high-quality video file.

[0032] It should be noted that the original image data acquired in both data streams is exactly the same.

[0033] 4) The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0034] The terms "at least one," "at least one of," etc., used in the specification and claims of this application refer to any one, any two, or a combination of two or more of the included items. For example, at least one of a, b, and c can mean: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more items, and its meaning is similar to that of "at least one."

[0035] The identifiers in this application are text, symbols, images, etc. used to indicate information, and may be used as carriers for displaying information in the form of identifiers or other containers, including but not limited to text identifiers, image identifiers, symbol identifiers, etc.

[0036] It should be noted that the image processing method provided in this application can be executed by electronic devices such as mobile phones, tablets, laptops, PDAs, and in-vehicle electronic devices. Some embodiments of this application use electronic devices as the executing entity to illustrate the image processing method provided in this application.

[0037] The image processing method, apparatus, electronic device, and readable storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0038] In related technologies, camera applications on electronic devices generate two data streams during video recording: a preview stream and a recording stream. The preview stream is used for real-time display and requires low latency, while the recording stream is cached in the background and does not require strict real-time performance. Typically, during recording, the electronic device configures processing algorithms of varying complexity based on the different real-time requirements of the two data streams. However, this method does not fully utilize the prior knowledge of the preview stream.

[0039] Furthermore, with the development of image processing technology, users have increasingly higher requirements for the image quality of video recordings. Therefore, after electronic devices acquire multiple consecutive frames of image data through sensors, they also need to use optimization algorithms to optimize these frames of image data in order to obtain video recordings with higher image quality.

[0040] In current optimization processes, preset optimization algorithms are typically used to optimize image quality, such as color correction or improved sharpness. This results in electronic devices processing the recorded image data regardless of whether there are corresponding image quality issues, thus increasing the power consumption of the electronic device and causing wasted power.

[0041] In the image processing method provided in this application embodiment, the electronic device performs image quality detection on N frames of first image data. The N frames of first image data are either N frames of preview image data corresponding to the N frames of image data acquired during recording, or N frames of recorded image data, where N is a positive integer. If there are image quality issues with the N frames of first image data, an optimization algorithm corresponding to the image quality issue is used to optimize the N frames of recorded image data. In this solution, the electronic device performs pre-detection on the preview image data or recorded image data to identify image quality issues in the recorded image data. Then, it uses a targeted optimization algorithm corresponding to the image quality issue to optimize the recorded image data. This improves the image quality of the optimized recorded image data and reduces the power consumption of processing the recorded image data.

[0042] The image processing method provided in this application can be executed by an image processing device. Exemplarily, the image processing device can be an electronic device, or a component within that electronic device, such as an integrated circuit or a chip. The image processing method provided in this application will be described executively below using an electronic device as an example.

[0043] This application provides an image processing method. Figure 2 A flowchart illustrating an image processing method provided in an embodiment of this application is shown, which can be applied to electronic devices. Figure 2As shown, the image processing method provided in this application embodiment may include the following steps 201 and 202.

[0044] Step 201: The electronic device performs image quality detection on the N frames of the first image data.

[0045] In some embodiments of this application, the aforementioned N-frame first image data is: N-frame preview image data corresponding to the N-frame image data obtained during the recording process, or N-frame recorded image data, where N is a positive integer.

[0046] In some embodiments of this application, the aforementioned N frames of preview image data are image data that needs to be processed in the preview data stream.

[0047] In some embodiments of this application, the aforementioned N frames of video image data are image data processed in the video data stream.

[0048] In some embodiments of this application, the electronic device acquires N frames of image signals through a sensor, converts the N frames of image data into N frames of image data through an image signal processor (ISP), and copies the N frames of image data to obtain N frames of preview image data and N frames of video recording image data, respectively. Then, the N frames of preview image data are input into the preview data stream, and the N frames of video recording image data are input into the video recording data stream for data processing.

[0049] It is understandable that the image data content of N frames of preview image data and N frames of recorded image data are exactly the same.

[0050] In some embodiments of this application, the electronic device determines the imaging quality of the N frames of first image data by detecting the smoothness and 3A consistency of the N frames of first image data.

[0051] For example, the aforementioned smoothness can be manifested by, but is not limited to, at least one of the following: frame interval uniformity, scaling uniformity, motion uniformity.

[0052] For example, the above 3A consistency can be reflected by at least one of the following: color consistency, focus sharpness.

[0053] It should be noted that the specific detection method for the above-mentioned electronic device to detect N frames of the first image data can be implemented through the following steps 201a to 201e, which will not be described here.

[0054] Step 202: If there is an image quality problem in the first N frames of image data, the electronic device uses an optimization algorithm corresponding to the image quality problem to optimize the N frames of recorded image data.

[0055] In some embodiments of this application, the above-mentioned image quality problems include, but are not limited to, at least one of the following: uneven frame intervals, uneven scaling, uneven motion, inconsistent colors, and unclear focus.

[0056] In some embodiments of this application, an image quality problem corresponds to at least one optimization algorithm.

[0057] For example, the optimization algorithm corresponding to uneven frame intervals is a frame interpolation algorithm, the optimization algorithm corresponding to uniform scaling is a frame interpolation algorithm, the optimization algorithm corresponding to uneven motion is a stabilization optimization algorithm, the optimization algorithm corresponding to inconsistent colors is a color correction optimization algorithm, and the optimization algorithm corresponding to unclear focus is a sharpening optimization algorithm.

[0058] In some embodiments of this application, the electronic device determines the optimization algorithm corresponding to each image quality problem from at least one stored optimization algorithm, based on different image quality problems.

[0059] It is understood that the aforementioned optimization algorithm may be stored in an electronic device, generated based on different image quality issues, or obtained in other ways; this application does not impose any restrictions.

[0060] In some embodiments of this application, the electronic device detects the first image data of the Nth frame through a detection module to evaluate the imaging quality, and then optimizes the N frames of video image data through an optimization module to improve the imaging quality.

[0061] In one possible embodiment, such as Figure 3 As shown, the electronic device performs image quality detection on N frames of preview image data through the detection module, and determines the corresponding optimization algorithm based on the detected image quality problems through the optimization module. Then, the determined optimization algorithm is used to process the N frames of video recording image data to generate a video recording, which is then compressed and stored in the storage space of the electronic device.

[0062] In the method of this application embodiment, the generation of video recording based on video image data does not require strict real-time performance. The preview stream processing is performed several frames (m frames) before the video stream. By detecting the imaging quality of N frames of preview image data, the processing method of the video stream is selected in a targeted manner, thereby ensuring both the real-time performance of the preview stream and maximizing the imaging quality of the video stream.

[0063] In another possible embodiment, such as Figure 4As shown, the electronic device performs image quality detection on N video image data through the detection module, and determines the corresponding optimization algorithm based on the detected image quality problems through the optimization module. Then, the determined optimization algorithm is used to further process the N video image data to generate a video, which is then compressed and stored in the storage space of the electronic device.

[0064] In the image processing method provided in this application embodiment, the electronic device performs image quality detection on N frames of first image data. The N frames of first image data are either N frames of preview image data corresponding to the N frames of image data acquired during recording, or N frames of recorded image data, where N is a positive integer. If there are image quality problems in the N frames of first image data, an optimization algorithm corresponding to the image quality problem is used to optimize the N frames of recorded image data. In this solution, the electronic device performs pre-detection on the preview image data or recorded image data to identify image quality problems in the recorded image data. Then, it uses a targeted optimization algorithm corresponding to the image quality problem to optimize the recorded image data. This improves the image quality of the optimized recorded image data and reduces the power consumption of processing the recorded image data.

[0065] The following five embodiments describe in detail the process of an electronic device performing image quality detection on N frames of first image data.

[0066] Example 1: Since the uniformity of frame interval is a crucial factor affecting image smoothness, a smaller frame interval means more frames are displayed per second (i.e., a higher frame rate), resulting in a smoother and more continuous image. Conversely, a larger frame interval means fewer frames are displayed per second (i.e., a lower frame rate), leading to stuttering or jerking in the image. Therefore, in this example, the electronic device determines whether there are any first image quality issues, i.e., uneven frame intervals, in the N frames of first image data by detecting the frame interval of the first N frames.

[0067] Optionally, in some embodiments of this application, step 201 can be implemented by steps 201a1 and 201a2, and step 202 can be implemented by step 202a.

[0068] Step 201a1: The electronic device acquires each frame interval of the N frames of first image data.

[0069] In some embodiments of this application, the frame interval is the time difference between every two adjacent frames of first image data in N frames of first image data.

[0070] For example, taking N as 4, that is, 4 frames of first image data including image frame 1 to image frame 4, these 4 frames of first image data have three frame intervals, namely the frame interval between image frame 1 and image frame 2, the frame interval between image frame 2 and image frame 3, and the frame interval between image frame 3 and image frame 4.

[0071] Step 201a2: If the first frame interval is greater than the first preset threshold, determine that there is a first image quality problem in the N frames of first image data.

[0072] In some embodiments of this application, when the electronic device detects the existence of a frame interval, that is, when the first frame interval is greater than a first preset threshold, it is determined that there is a first image quality problem in N frames of first image data.

[0073] In some embodiments of this application, the first frame interval is one frame interval in each frame interval of the N frames of first image data.

[0074] In some embodiments of this application, the first preset threshold is determined based on the recording frame rate corresponding to N frames of first image data, or the first preset threshold is preset by the electronic device, or the first preset threshold is set by the user.

[0075] For example, different frame rates correspond to different frame intervals. For instance, 30 FPS corresponds to a frame interval of approximately 33 ms, so the first preset threshold can be set to 33 ms.

[0076] In some embodiments of this application, the first image quality problem mentioned above is uneven frame spacing.

[0077] In some embodiments of this application, the electronic device obtains the frame interval between every two adjacent image data in N frames of first image data, and then determines whether there is a frame interval greater than a first preset frame rate. If there is a frame interval greater than the first preset frame rate, it indicates that the frame interval of the N frames of first image data is not uniform; if there is no frame interval, it indicates that the frame interval between the N frames of first image data is uniform.

[0078] Step 202a: The electronic device uses a frame interpolation optimization algorithm to optimize the N frames of video image data.

[0079] In some embodiments of this application, the electronic device employs a frame interpolation optimization algorithm corresponding to the first image quality problem to optimize N frames of video image data.

[0080] In some embodiments of this application, the electronic device inserts at least one frame of image data into N frames of first image data to adjust the frame interval between every two adjacent frames of image data in the N frames of first image data.

[0081] For example, electronic devices may employ, but are not limited to, any of the following frame interpolation algorithms: frame repetition algorithm, frame averaging algorithm, motion-based frame interpolation algorithm, and artificial intelligence (AI) frame interpolation algorithm.

[0082] In this way, electronic devices can detect image quality problems in the first N frames of image data and then select optimization algorithms to optimize the N frames of video recording data, thereby improving the image quality of the video generated from the N frames of video recording data.

[0083] Example 2: During video recording on an electronic device, if the focal length changes—that is, when the user adjusts the focal length parameter—the uniformity of the zoom is also a crucial factor affecting image smoothness. For example... Figure 5 As shown, a non-uniform change in ratio or magnification during zooming is more consistent with human visual perception. Therefore, in this embodiment, scaling uniformity detection can directly detect changes in ratio during zooming. If a uniform change is detected in the first N frames of image data, appropriate cropping and frame interpolation can be performed during the processing of the N frames of recorded image data to make the transition of the zoom process smoother. In other words, the change in ratio during zooming should be fast at first and then slow down, which is more consistent with visual perception.

[0084] Optionally, in some embodiments of this application, step 201 can be implemented by steps 201b1 to 201b3, or steps 201b1, 201b2 and 201b4, and step 202 can be implemented by step 202b.

[0085] Step 201b1: The electronic device acquires the zoom ratio change of each frame of the N frames of the first image data.

[0086] In some embodiments of this application, the zoom ratio is the zoom factor of the camera of the electronic device, or the focal length ratio.

[0087] For example, the electronic device obtains the zoom ratio change of each frame by acquiring the zoom ratio corresponding to each frame of the first image data in N frames of first image data.

[0088] Step 201b2: The electronic device calculates the scaling smoothness value corresponding to the first image data of N frames based on the zoom ratio change.

[0089] In some embodiments of this application, the above-mentioned scaling smoothness value is used to characterize whether the image is smooth during the scaling process.

[0090] In some embodiments of this application, the electronic device calculates the scaling smoothness value corresponding to each pair of adjacent first image data based on the zoom ratio corresponding to the first image data of N frames using formula (1).

[0091] For example, let the values ​​of ratio during the zoom process be (r1, r2...r...). N Then, the scaling smoothness R during zooming can be denoted as:

[0092] R = (r n - r0) / (n-1) * ({n / 2}–1) + r0 – r {n / 2} (1)

[0093] Here, {.} indicates rounding up the elements within it.

[0094] Step 201b3: When the zoom ratio is the magnification ratio, if the zoom smoothness value is greater than 0, the electronic device determines that there is a second image quality problem in the N frames of the first image data.

[0095] Step 201b4: When the zoom ratio is the reduction ratio, if the zoom smoothness value is less than 0, the electronic device determines that there is a second image quality problem in the N frames of the first image data.

[0096] In some embodiments of this application, the aforementioned second image quality problem is uniform scaling.

[0097] In some embodiments of this application, when the zoom ratio is 2X, the scene is enlarged from small to large, and in this case, the zoom ratio is the magnification ratio.

[0098] In some embodiments of this application, when the zoom ratio is 0.5X, the scene is reduced from large to small, and in this case, the zoom ratio is the reduction ratio.

[0099] For example, when R is 0, it indicates that the zoom trend is average scaling, which is considered unacceptable and requires optimization. When the zoom scene is a magnification, R less than 0 indicates that the requirement is met, and greater than 0 indicates that the requirement is not met; that is, the electronic device determines that the first N frames of image data have a second image quality problem. Conversely, in a reduction magnification scene, R greater than 0 indicates that the requirement is met, and less than 0 indicates that the requirement is not met; that is, the electronic device determines that the first N frames of image data have a second image quality problem. For the first image data that meets the requirements, no adjustment is required in subsequent recording streams; otherwise, appropriate adjustments need to be made in the recording stream.

[0100] Step 202b: The electronic device uses a pruning and interpolation optimization algorithm to optimize the N frames of video image data.

[0101] In some embodiments of this application, the electronic device employs a cropping and interpolation optimization algorithm corresponding to the second image quality problem to optimize N frames of video image data.

[0102] In some embodiments of this application, the electronic device inserts at least one cropped frame of image data into N frames of first image data to adjust the frame interval between every two adjacent frames of image data in the N frames of first image data.

[0103] For example, if you want to insert a frame of image data at a magnified scale, you crop the original image data; if you want to insert a frame of image data at a reduced scale, you fill the original image data with pixels.

[0104] For example, electronic devices may employ, but are not limited to, any of the following frame interpolation algorithms: frame repetition algorithm, frame averaging algorithm, motion-based frame interpolation algorithm, and artificial intelligence (AI) frame interpolation algorithm.

[0105] In this way, electronic devices can detect image quality problems in the first N frames of image data and then select optimization algorithms to optimize the N frames of video recording data, thereby improving the image quality of the video generated from the N frames of video recording data.

[0106] Example 3: When a user records a moving scene using an electronic device, or records while in motion, the image is prone to shaking. In this case, the movement of the main subject in the captured image is usually "discontinuous." Therefore, the electronic device can determine whether there is severe shaking by detecting the light flow direction of the main subject in the first N frames of image data. If it is determined that there is uneven motion in the first N frames of image data, a corresponding optimization algorithm is used to optimize the N frames of recorded image data.

[0107] Optionally, in some embodiments of this application, step 201 can be implemented by steps 201c1 to 201c3, and step 202 can be implemented by step 202c.

[0108] Step 201c1: The electronic device acquires the optical flow diagram corresponding to each two adjacent frames of the first image data in N frames of the first image data.

[0109] In some embodiments of this application, the optical flow graph described above is a graph used to visualize the apparent motion patterns of an object, surface, or edge between two consecutive frames of images.

[0110] In some embodiments of this application, the optical flow graph described above describes the motion vector of a pixel from one frame to the next.

[0111] In some embodiments of this application, the electronic device employs an optical flow calculation method to calculate and generate an optical flow map based on two adjacent frames of first image data.

[0112] For example, the electronic device acquires N images corresponding to N frames of first image data, denoted as {N1, N2, N3...N...} n The optical flow graphs of two adjacent frames are calculated using existing optical flow calculation methods, denoted as {F1, F2, F3 … F}. K}

[0113] It should be noted that since the electronic device records moving scenes, there may be appearances and disappearances of objects. That is, the number of scenes in the preview image differs between N1 and N2. Therefore, the electronic device needs to process each frame {N1, N2, N3...N...} n} Perform scene segmentation; if the number of scenes in N1 is greater than N. n Then N n The scenes identified in N2 are used as the baseline. Let M scenes be identified in N2, denoted as {R... 21 R 22 ...R 2m}

[0114] Step 201c2: The electronic device obtains the variance of the mean optical flow motion of each captured object in the N frames of the first image data based on the optical flow map.

[0115] In some embodiments of this application, the aforementioned mean light of optical flow motion is used to characterize the average amplitude of all pixel optical flow vectors calculated in a continuous video corresponding to N frames of first image data.

[0116] In some embodiments of this application, the electronic device calculates the motion vector of each pixel in the pixel region where a scene is located in each frame of first image data, and then calculates the average motion vector of each pixel to obtain the average optical flow motion value corresponding to that scene. This process is repeated to obtain the average optical flow motion value corresponding to all scenes.

[0117] For example, the electronic device calculates each scenario { R 11 R 22 ...R 1m}……{ R k1 R k2 ...R km The mean optical flow in the corresponding region is denoted as {E}. 11 E 22 ...E 1m}……{ E k1 E k2 ...E km Next, the mean optical flow motion value for each scene is calculated, i.e., {E}11 E 22 ...E 1m}……{ E k1 E k2 ...E km The variance of}.

[0118] Step 201c3: When the variance of the mean of optical flow motion is greater than the second preset threshold, the electronic device determines that there is a third image quality problem in the N frames of the first image data.

[0119] In some embodiments of this application, when the electronic device detects that the variance of the mean of optical flow motion is greater than a second preset threshold, the electronic device determines that there is a third image quality problem in N frames of first image data.

[0120] In some embodiments of this application, the aforementioned third image quality problem is motion non-uniformity.

[0121] In some embodiments of this application, the aforementioned second preset threshold is either a default setting of the electronic device or a user-defined setting.

[0122] In some embodiments of this application, the variance of the mean optical flow motion is close to 0: this means there is almost no motion in the image. The scene is static, or there is only very slight, negligible jitter. A small variance of the mean optical flow motion means there is slow or slight motion in the image. A large variance of the mean optical flow motion means there is fast, intense, and large-scale motion in the image.

[0123] Step 202c: The electronic device uses an anti-shake optimization algorithm to optimize the N frames of video image data.

[0124] In some embodiments of this application, the electronic device employs a stabilization optimization algorithm corresponding to the third image quality problem to optimize the N frames of video image data.

[0125] In some embodiments of this application, the electronic device reduces image jitter by increasing the strength of the anti-shake threshold, thereby ensuring the smoothness of the video corresponding to the N frames of the first image data.

[0126] In this way, electronic devices can detect image quality problems in the first N frames of image data and then select optimization algorithms to optimize the N frames of video recording data, thereby improving the image quality of the video generated from the N frames of video recording data.

[0127] Example 4: When a user records using an electronic device, the camera may be switched. Different cameras have different hardware characteristics, which may lead to color inconsistencies. Therefore, the electronic device can detect the color consistency of the first N frames of image data. If color inconsistencies are found in the first N frames of image data, a corresponding optimization algorithm is used to optimize the recorded N frames of image data.

[0128] Optionally, in some embodiments of this application, step 201 can be implemented by steps 201d1 and 201d2, and step 202 can be implemented by step 202d.

[0129] Step 201d1: When the first image data of N frames changes scene, the electronic device performs color difference calculation on the target pixel region in the second image data where scene change occurs.

[0130] In some embodiments of this application, the electronic device detects whether there is a scene change in the N frames of first image data by detecting the image content in the N frames of first image data.

[0131] In some embodiments of this application, the electronic device determines whether there is a scene switch in the N frames of first image data by judging whether the scenes in different image data are the same.

[0132] For example, an electronic device can determine whether there is a scene change in the N-frame first image data by detecting whether the camera has switched, or it can determine whether there is a scene change in the N-frame first image data by detecting the captured content in the N-frame first image data.

[0133] In some embodiments of this application, the second image data mentioned above are two adjacent frames of image data where a scene switch occurs.

[0134] In some embodiments of this application, the target pixel region is the pixel region where a scene switching occurs.

[0135] In some embodiments of this application, the electronic device uses a color difference calculation formula to calculate the color variance of the target pixel region.

[0136] For example, the electronic device detects whether the camera has switched. If a switch has occurred, it performs scene segmentation on the first image data of the two frames before and after the switch, and denots the resulting M scenes as {C}. 11 C 22 ...C 1m}……{ C n1 C n2 ...C nm}; where C 11C represents the first scene of the first frame of the first image data. 1m This represents the m-th scene in the first frame of the first image data. Then, the electronic device performs color difference calculations on the pixel regions of the scene before and after the switch, for example, using the CIE 1976 color difference formula to calculate the color difference between corresponding scenes in two adjacent frames {{ C 11 C 21}、 { C 12 C 22}、…{ C 1m CC nm The color differences between}} are obtained to get {C D1 C D2 ...C DN}, then calculate {C D1 C D2 ...C DN The variance of} represents the fluctuation of color changes. The larger the variance, the greater the color change, which means that the switching of cameras is more obvious and the zoom is less smooth.

[0137] Step 201d3: If the color difference variance corresponding to the second image data is greater than the third preset threshold, the electronic device determines that there is a fourth image quality problem in the N frames of the first image data.

[0138] In some embodiments of this application, the aforementioned fourth image quality problem is color inconsistency.

[0139] In some embodiments of this application, the aforementioned third preset threshold is either a default setting of the electronic device or a user-defined setting.

[0140] Step 202d: The electronic device uses a color gain correction optimization algorithm to optimize the N frames of video image data.

[0141] In some embodiments of this application, the electronic device employs a color gain correction optimization algorithm corresponding to the fourth image quality problem to optimize the N frames of video image data.

[0142] In some embodiments of this application, the electronic device performs color space conversion processing on the second image data, namely the image data with inconsistent colors in the above-mentioned N frames of video image data; then calculates the color gain value of the second image data after color space conversion processing; then, based on the color gain value, performs color correction on the second image data after color space conversion processing, and performs color space conversion processing again on the color-corrected second image data, finally obtaining the optimized video image data.

[0143] For example, the above color gain correction optimization method is specifically as follows, such as... Figure 6 As shown:

[0144] Step 1: Perform color space conversion on the two frames before and after the camera switch, i.e., the second image data mentioned above, from RGB color space to HSV color space.

[0145] Step 2: The electronic device calculates the mean hue E of each segmented scene in these two frames. f (H), E b (H), mean saturation E f (S), E b (S), and mean brightness E f (V), E b (V), taking the calculation of the mean value of the switching foreground color as an example, the calculation formula (2) is as follows:

[0146] E f (H) = mean({C f 1(H), C f 2(H),…C f N(H)}) (2)

[0147] Step 3: The electronic device calculates the gain of each color mean to obtain G(H), G(S), and G(S). The calculation formula (3) is as follows:

[0148] G(H) = (E) f (H) + E b (H)) / E b (H) (3)

[0149] Step 4: After performing gain correction on the image after switching cameras in the HSV color space, convert it to the RGB color space.

[0150] In this way, electronic devices can detect image quality problems in the first N frames of image data and then select optimization algorithms to optimize the N frames of video recording data, thereby improving the image quality of the video generated from the N frames of video recording data.

[0151] Example 5: During video recording by an electronic device, blurry focusing may occur due to movement of objects or switching of cameras. The main difference between clear and blurry images lies in whether the structural information of the main subject is clearly represented. Therefore, to address this image quality issue, the electronic device can detect the scene complexity of the first N frames of image data. If scene complexity is determined to exist in the first N frames of image data, a corresponding optimization algorithm is used to optimize the recorded N frames of image data.

[0152] Optionally, in some embodiments of this application, step 201 can be implemented by steps 201e1 and 201e2, and step 202 can be implemented by step 202e.

[0153] Step 201e1: The electronic device acquires the complexity of the scene being captured in the first N frames of image data.

[0154] In some embodiments of this application, the above-mentioned complexity is used to characterize the structural complexity of the scene being captured in the image data.

[0155] In some embodiments of this application, the electronic device performs scene segmentation on each frame of image data in N frames of first image data to obtain the pixel region of the main scene, and then performs structural texture decomposition on the pixel region of the main scene to obtain the main structure image, so as to statistically analyze the complexity of the structure image.

[0156] It should be noted that the aforementioned main scene can be the scene that occupies the largest proportion of the image data, or the clearest scene, or the scene that is in the foreground; this application does not impose any restrictions.

[0157] Step 201e2: If the complexity is greater than the fourth preset threshold, the electronic device determines that there is a fifth image quality problem in the N frames of the first image data.

[0158] In some embodiments of this application, the aforementioned fifth image quality problem is poor focus.

[0159] In some embodiments of this application, the aforementioned fourth preset threshold is either a default setting of the electronic device or a user-defined setting.

[0160] Understandably, higher complexity indicates higher clarity, and vice versa.

[0161] Step 202e: The electronic device uses a sharpening optimization algorithm to optimize the N frames of video image data.

[0162] In some embodiments of this application, the electronic device employs a sharpening optimization algorithm corresponding to the fifth image quality problem to optimize N frames of video image data.

[0163] In some embodiments of this application, electronic devices enhance the sharpness of edges, details, and textures of image data by suppressing low-frequency components (i.e., details and edges) of the image.

[0164] In this way, electronic devices can detect image quality problems in the first N frames of image data and then select optimization algorithms to optimize the N frames of video recording data, thereby improving the image quality of the video generated from the N frames of video recording data.

[0165] Each of the above-described method embodiments, or various possible implementations of each method embodiment, can be executed individually or in combination of any two or more. The specific implementation can be determined according to actual usage requirements, and this application does not impose any restrictions on this.

[0166] The image processing method provided in this application can be executed by an electronic device or an image processing apparatus. This application uses an image processing apparatus to execute the image processing method as an example to illustrate the image processing apparatus provided in this application.

[0167] Figure 7 A schematic diagram of a possible structure of the image processing apparatus involved in an embodiment of this application is shown. For example... Figure 7 As shown, the image processing device 700 may include a detection module 701 and a processing module 702.

[0168] The detection module 701 is used to perform image quality detection on N frames of first image data. The N frames of first image data are: N frames of preview image data corresponding to the N frames of image data acquired during recording, or N frames of recorded image data, where N is a positive integer. The processing module 702 is used to optimize the N frames of recorded image data by using an optimization algorithm corresponding to the image quality problem when the detection module 701 detects that there is an image quality problem in the N frames of first image data.

[0169] Optionally, in some embodiments of this application, the detection module 701 described above is specifically used for:

[0170] Obtain the frame interval of each of the above N frames of the first image data;

[0171] If the first frame interval is greater than the first preset threshold, it is determined that there is a first image quality problem in the N frames of first image data. The first frame interval is one frame interval in each frame interval of the N frames of first image data.

[0172] The aforementioned processing module 702 is specifically used to optimize the aforementioned N frames of video image data using a frame interpolation optimization algorithm.

[0173] Optionally, in some embodiments of this application, the detection module 701 described above is specifically used for:

[0174] Obtain the zoom ratio change for each frame of the above N frames of the first image data;

[0175] Based on the aforementioned zoom ratio changes, calculate the scaling smoothness value corresponding to the aforementioned N frames of the first image data.

[0176] When the zoom ratio is the magnification ratio, if the above zoom smoothness value is greater than 0, then it is determined that the above N frames of the first image data have a second image quality problem; or,

[0177] If the zoom smoothness value is less than 0 when the zoom magnification is the reduction magnification, then it is determined that the above N frames of first image data have a second image quality problem.

[0178] The aforementioned processing module 702 is specifically used to optimize the aforementioned N frames of video image data by employing a cropping and interpolation optimization algorithm.

[0179] Optionally, in some embodiments of this application, the detection module 701 described above is specifically used for:

[0180] Obtain the optical flow diagram corresponding to each two adjacent frames of the first image data in the above N frames of first image data;

[0181] Based on the above optical flow map, the variance of the mean optical flow motion of each captured object in the above N frames of first image data is obtained;

[0182] If the variance of the mean of the optical flow motion is greater than the second preset threshold, it is determined that the above N frames of first image data have a third image quality problem.

[0183] The aforementioned processing module 702 is specifically used to optimize the aforementioned N frames of video image data using a stabilization optimization algorithm.

[0184] Optionally, in some embodiments of this application, the detection module 701 described above is specifically used for:

[0185] If there is a scene change in the first image data of the above N frames, color difference calculation is performed on the target pixel region in the second image data where there is a scene change. The target pixel region is the pixel region where the scene with the scene change is located.

[0186] If the color difference variance corresponding to the second image data is greater than the third preset threshold, it is determined that the first image data of the above N frames has a fourth image quality problem.

[0187] The aforementioned processing module 702 is specifically used to optimize the aforementioned N frames of video image data using a color gain correction optimization algorithm.

[0188] Optionally, in some embodiments of this application, the detection module 701 described above is specifically used for:

[0189] Obtain the complexity of the scene captured in the first N frames of image data mentioned above;

[0190] If the complexity exceeds the fourth preset threshold, it is determined that the first image data of the above N frames has a fifth image quality problem;

[0191] The aforementioned processing module 702 is specifically used to optimize the aforementioned N frames of video image data using a sharpening optimization algorithm.

[0192] In the image processing apparatus provided in this application embodiment, the image processing apparatus performs image quality detection on N frames of first image data. The N frames of first image data are: N frames of preview image data corresponding to the N frames of image data acquired during recording, or N frames of recorded image data, where N is a positive integer. If there are image quality problems in the N frames of first image data, an optimization algorithm corresponding to the image quality problem is used to optimize the N frames of recorded image data. In this solution, the image processing apparatus performs pre-detection on the preview image data or recorded image data to identify image quality problems in the recorded image data, and then uses the optimization algorithm corresponding to the image quality problem to optimize the recorded image data. This improves the image quality of the optimized recorded image data and reduces the power consumption of processing the recorded image data.

[0193] The image processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0194] The image processing device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0195] The image processing apparatus provided in this application embodiment can implement the various processes implemented in the image processing method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0196] Optionally, such as Figure 8 As shown, this application embodiment also provides an electronic device 800, including a processor 801 and a memory 802. The memory 802 stores a program or instructions that can run on the processor 801. When the program or instructions are executed by the processor 801, they implement the various steps of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0197] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0198] Figure 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0199] The electronic device 100 includes, but is not limited to, components such as: radio frequency unit 101, network module 102, audio output unit 103, input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, and processor 110.

[0200] Those skilled in the art will understand that the electronic device 100 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 110 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 9 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0201] The processor 110 is used to perform image quality detection on N frames of first image data. The N frames of first image data are: N frames of preview image data corresponding to the N frames of image data acquired during the recording process, or N frames of recorded image data, where N is a positive integer. The processor 110 is also used to optimize the N frames of recorded image data by using an optimization algorithm corresponding to the image quality problem when there is an image quality problem in the N frames of first image data.

[0202] Optionally, in some embodiments of this application, the processor 110 is specifically used for:

[0203] Obtain the frame interval of each of the above N frames of the first image data;

[0204] If the first frame interval is greater than the first preset threshold, it is determined that there is a first image quality problem in the N frames of first image data. The first frame interval is one frame interval in each frame interval of the N frames of first image data.

[0205] The processor 110 is specifically used to optimize the above N frames of video image data by employing a frame interpolation optimization algorithm.

[0206] Optionally, in some embodiments of this application, the processor 110 is specifically used for:

[0207] Obtain the zoom ratio change for each frame of the above N frames of the first image data;

[0208] Based on the aforementioned zoom ratio changes, calculate the scaling smoothness value corresponding to the aforementioned N frames of the first image data.

[0209] When the zoom ratio is the magnification ratio, if the above zoom smoothness value is greater than 0, then it is determined that the above N frames of the first image data have a second image quality problem; or,

[0210] If the zoom smoothness value is less than 0 when the zoom magnification is the reduction magnification, then it is determined that the above N frames of first image data have a second image quality problem.

[0211] The processor 110 is specifically used to optimize the above N frames of video image data by employing a cropping and interpolation optimization algorithm.

[0212] Optionally, in some embodiments of this application, the processor 110 is specifically used for:

[0213] Obtain the optical flow diagram corresponding to each two adjacent frames of the first image data in the above N frames of first image data;

[0214] Based on the above optical flow map, the variance of the mean optical flow motion of each captured object in the above N frames of first image data is obtained;

[0215] If the variance of the mean of the optical flow motion is greater than the second preset threshold, it is determined that the above N frames of first image data have a third image quality problem.

[0216] The processor 110 is specifically used to optimize the above N frames of video image data by employing a stabilization optimization algorithm.

[0217] Optionally, in some embodiments of this application, the processor 110 is specifically used for:

[0218] In the case of scene switching in the above N frames of first image data, color difference calculation is performed on the target pixel region in the second image data where scene switching occurs. The target pixel region is the pixel region where the scene switching occurs.

[0219] If the color difference variance corresponding to the second image data is greater than the third preset threshold, it is determined that the first image data of the above N frames has a fourth image quality problem.

[0220] The processor 110 is specifically used to optimize the above N frames of video image data by employing a color gain correction optimization algorithm.

[0221] Optionally, in some embodiments of this application, the processor 110 is specifically used for:

[0222] Obtain the complexity of the scene captured in the first N frames of image data mentioned above;

[0223] If the complexity exceeds the fourth preset threshold, it is determined that the first image data of the above N frames has a fifth image quality problem;

[0224] The processor 110 is specifically used to optimize the above N frames of video image data by employing a sharpening optimization algorithm.

[0225] In the electronic device provided in this application embodiment, the electronic device performs image quality detection on N frames of first image data. The N frames of first image data are: N frames of preview image data corresponding to the N frames of image data acquired during recording, or N frames of recorded image data, where N is a positive integer. If there are image quality problems in the N frames of first image data, an optimization algorithm corresponding to the image quality problem is used to optimize the N frames of recorded image data. In this solution, the electronic device performs pre-detection on the preview image data or recorded image data to identify image quality problems in the recorded image data, and then uses the optimization algorithm corresponding to the image quality problem to optimize the recorded image data. This improves the image quality of the optimized recorded image data and reduces the power consumption of processing the recorded image data.

[0226] It should be understood that, in this embodiment, the input unit 104 may include a graphics processing unit (GPU) 1041 and a microphone 1042. The GPU 1041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 106 may include a display panel 1061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 107 includes at least one of a touch panel 1071 and other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0227] The memory 109 can be used to store software programs and various data. The memory 109 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 109 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 109 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0228] Processor 110 may include one or more processing units; optionally, processor 110 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 110.

[0229] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0230] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0231] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0232] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0233] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0234] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0235] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0236] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: Image quality detection is performed on N frames of first image data, wherein the N frames of first image data are: N frames of preview image data corresponding to the N frames of image data obtained during the recording process, or N frames of recorded image data, where N is a positive integer; If there is an image quality problem in the first N frames of image data, an optimization algorithm corresponding to the image quality problem is used to optimize the N frames of video image data.

2. The method according to claim 1, characterized in that, The image quality detection of N frames of first image data includes: Obtain each frame interval of the N frames of the first image data; If a first frame interval is greater than a first threshold, it is determined that the N frames of first image data have a first image quality problem, where the first frame interval is one frame interval in each frame interval of the N frames of first image data; The step of using an optimization algorithm corresponding to the image quality problem to optimize the N frames of video image data includes: The N frames of video image data are optimized using a frame interpolation optimization algorithm.

3. The method according to claim 1, characterized in that, The image quality detection of N frames of first image data includes: Obtain the zoom ratio change for each frame of the N frames of the first image data; Based on the zoom ratio change, calculate the scaling smoothness value corresponding to the N frames of the first image data. When the zoom ratio is the magnification ratio, if the zoom smoothness value is greater than 0, then it is determined that the N frames of the first image data have a second image quality problem; or, If the zoom smoothness value is less than 0 when the zoom magnification is the reduction magnification, then it is determined that the N frames of the first image data have a second image quality problem. The step of using an optimization algorithm corresponding to the image quality problem to optimize the N frames of video image data includes: The N frames of video image data are optimized using a pruning and interpolation optimization algorithm.

4. The method according to claim 1, characterized in that, The image quality detection of N frames of first image data includes: Obtain the optical flow diagram corresponding to each two adjacent frames of the first image data in the N frames of first image data; Based on the optical flow map, the variance of the mean optical flow motion of each captured object in the N frames of the first image data is obtained; If the variance of the mean of optical flow motion is greater than the second threshold, it is determined that the N frames of first image data have a third image quality problem; The step of using an optimization algorithm corresponding to the image quality problem to optimize the N frames of video image data includes: The image stabilization optimization algorithm corresponding to the third image quality problem is used to optimize the N frames of video image data.

5. The method according to claim 1, characterized in that, The image quality detection of N frames of first image data includes: In the case where there is a scene change in the first image data of the N frames, color difference calculation is performed on the target pixel region in the second image data where there is a scene change, and the target pixel region is the pixel region where the scene change is located; If the color difference variance corresponding to the second image data is greater than the third threshold, it is determined that the N frames of the first image data have a fourth image quality problem; The step of using an optimization algorithm corresponding to the image quality problem to optimize the N frames of video image data includes: A color gain correction optimization algorithm is used to optimize the N frames of video image data.

6. The method according to claim 1, characterized in that, The image quality detection of N frames of first image data includes: Obtain the complexity of the scene captured in the first N frames of image data; If the complexity is greater than the fourth threshold, it is determined that the N frames of the first image data have a fifth image quality problem; The step of using an optimization algorithm corresponding to the image quality problem to optimize the N frames of video image data includes: A sharpening optimization algorithm is used to optimize the N frames of video image data.

7. An image processing apparatus, characterized in that, The device includes: a detection module and a processing module; The detection module is used to perform image quality detection on N frames of first image data. The N frames of first image data are: N frames of preview image data corresponding to the N frames of image data obtained during the recording process, or N frames of recorded image data, where N is a positive integer. The processing module is used to optimize the N frames of video image data by employing an optimization algorithm corresponding to the image quality problem when the detection module detects that there is an image quality problem in the N frames of first image data.

8. The apparatus according to claim 7, characterized in that, The detection module is specifically used for: In the case where there is a scene change in the first image data of the N frames, color difference calculation is performed on the target pixel region in the second image data where there is a scene change, and the target pixel region is the pixel region where the scene change is located; If the color difference variance corresponding to the second image data is greater than a third preset threshold, it is determined that the N frames of the first image data have a fourth image quality problem; The processing module is specifically used to optimize the N frames of video image data using a color gain correction optimization algorithm.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the image processing method as described in any one of claims 1 to 6.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the image processing method as described in any one of claims 1 to 6.

Images