Image processing method and device, equipment and storage medium

Abstract

An image processing method and device, equipment and storage medium, the method comprises: scene prediction is carried out to current frame, obtains label set;Scene switching judgment is executed, including any one or more of first, second and third judgment;The first judgment judges whether the previous frame classification label is contained in the current frame label set and whether the confidence with the corresponding initial label is greater than the preset threshold;The second judgment judges whether the color statistical information difference of the previous frame and the current frame is within the preset range;The third judgment judges whether the proportion of the pixel of the previous frame classification label feature corresponding color tone range in the current frame is greater than the preset proportion threshold;When at least one of the scene switching judgment step is yes, assign the classification label of the previous frame to the current frame;Otherwise, set the first initial label as the classification label of the current frame;Based on the classification label of the current frame, configure the corresponding quality adjustment parameter.The embodiment of the application reduces the probability of scene mis-switching and improves the quality jitter problem.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] A television's video processing unit (VPU) includes modules for color adjustment, sharpening, and noise reduction. The VPU plays a crucial role in improving the quality of the television display, enhancing image detail, and refining color. Conventional television picture quality adjustments require a set of VPU control parameters adapted to all images, making it difficult to achieve optimal picture quality in all scenarios. Some television manufacturers employ artificial intelligence algorithms to classify images, applying different picture quality parameters to different image categories in an attempt to obtain the best possible picture quality.

[0003] AIPQ (Artificial Intelligent Picture Quality) learns image classification through a convolutional neural network and applies optimal VPU parameters to different classifications to achieve the best image quality. During convolutional neural network training, an image classification training set is created based on image quality adjustment experience, resulting in a classification model. In actual image quality adjustment, the classification model predicts the scene and obtains top N (typically top 1, top 3, or top 5) classification results. The VPU parameters are then adjusted based on the classification results. The most common adjustments are to parameters such as color saturation, contrast, and DNLP.

[0004] However, the current image quality adjustment is prone to image jitter. Summary of the Invention

[0005] The problem solved by the embodiments of the present invention is to provide an image processing method, apparatus, device and storage medium to reduce the probability of scene switching errors caused by the accuracy of the classification model and improve the image quality jitter caused by scene switching errors.

[0006] To address the aforementioned problems, this invention provides an image processing method, comprising: providing a video frame sequence, including a first frame of the video frame sequence and a current frame following the first frame; performing scene prediction processing on the current frame to obtain a label set corresponding to the current frame, wherein the label set includes the top few initial labels whose prediction results best match the scene of the current frame, and the initial label with the highest accuracy is the first initial label; a previous frame adjacent to the current frame has a determined classification label; and performing a scene switching judgment step, including any one or more judgment steps of a first judgment, a second judgment, and a third judgment; wherein the first judgment includes: determining whether the classification label of the previous frame is included in the label set of the current frame and is consistent with the corresponding initial label. The first judgment result is obtained by determining whether the confidence level of the label is greater than a preset threshold; the second judgment includes determining whether the difference between the color statistics of the previous frame and the color statistics of the current frame is within a preset difference range; the third judgment includes determining whether the proportion of pixels in the hue range corresponding to the label feature of the previous frame in the current frame image is greater than a preset proportion threshold; when at least one judgment result in the scene switching judgment step is yes, the classification label of the current frame is set to be the same as the classification label of the previous frame; otherwise, the first initial label is set as the classification label of the current frame; based on the classification label of the current frame, corresponding image quality adjustment parameters are configured for the current frame.

[0007] Accordingly, embodiments of the present invention also provide an image processing apparatus, comprising: a providing unit, configured to provide a video frame sequence, including a first frame of the video frame sequence and a current frame following the first frame; a scene prediction unit, configured to obtain a classification label of the first frame, and further configured to perform scene prediction processing on the current frame to obtain a label set corresponding to the current frame, the label set including the top few initial labels whose prediction results best match the scene of the current frame, wherein the initial label with the highest accuracy is the first initial label; and a scene switching judgment unit, configured to perform scene switching judgment processing on the current frame, the scene switching judgment unit including any one or more judgment subunits of a first judgment subunit, a second judgment subunit, and a third judgment subunit; wherein the first judgment subunit is configured to determine whether the classification label of the previous frame is included in the label set of the current frame and is consistent with the classification label of the previous frame. The system determines whether the confidence level of the initial label is greater than a preset threshold and outputs a first judgment result; the second judgment subunit determines whether the difference between the color statistics of the previous frame and the color statistics of the current frame is within a preset difference range and outputs a second judgment result; the third judgment subunit determines whether the proportion of the hue range corresponding to the feature of the classification label of the previous frame in the current frame image is greater than a preset proportion threshold and outputs a third judgment result; the label setting unit is used to set the classification label of the current frame to be the same as the classification label of the previous frame when the judgment result of at least one judgment subunit in the scene switching judgment unit is yes; otherwise, the label setting unit is used to set the first initial label as the classification label of the current frame; the image quality adjustment unit is used to configure corresponding image quality adjustment parameters for the current frame based on the classification label of the current frame.

[0008] Accordingly, embodiments of the present invention also provide an apparatus including at least one memory and at least one processor, wherein the memory stores one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the aforementioned image processing method.

[0009] This invention also provides a storage medium storing one or more computer instructions for implementing the aforementioned image processing method.

[0010] Compared with the prior art, the technical solution of the embodiments of the present invention has the following advantages:

[0011] The image processing method provided in this embodiment of the invention performs scene prediction processing on the current frame and then executes a scene switching judgment step, including any one or more of a first judgment, a second judgment, and a third judgment. The first judgment determines whether the classification label of the previous frame is included in the label set of the current frame and whether the confidence level with the corresponding initial label is greater than a preset threshold. Therefore, during scene switching, not only is the first initial label of the current frame compared with the classification label of the previous frame, but also whether the classification label of the previous frame is included in the label set of the current frame and the confidence level with the corresponding initial label are checked simultaneously, which helps prevent erroneous scene switching due to the accuracy of the classification model. The second judgment determines whether the difference between the color statistics information of the previous frame and the color statistics information of the current frame is within a certain threshold. Within a preset difference range, the risk of scene switching is judged by comparing the difference in color area between two frames, thereby improving the accuracy of scene switching judgment. The third judgment is used to determine whether the proportion of pixels in the hue range corresponding to the features of the previous frame's classification label in the current frame image is greater than a preset proportion threshold, thereby enabling further judgment of the scene in the current frame, which is beneficial to improving the accuracy of scene switching judgment. In summary, the scene switching steps of this embodiment can improve the accuracy of scene switching judgment, which is beneficial to reducing the probability of scene switching errors due to the accuracy of the classification model. Accordingly, when configuring the corresponding image quality parameters based on the classification label of the current frame, it is beneficial to improve the image quality jitter caused by scene switching errors, thereby improving the quality of image quality adjustment. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating an embodiment of the image processing method of the present invention;

[0013] Figure 2 yes Figure 1 A flowchart illustrating step S9 of an embodiment;

[0014] Figure 3 yes Figure 1 A flowchart illustrating step S8 of an embodiment;

[0015] Figure 4 This is a functional block diagram of an embodiment of the image processing apparatus of the present invention;

[0016] Figure 5 yes Figure 4 A functional block diagram of an embodiment of the scene switching judgment unit 104;

[0017] Figure 6 This is a schematic diagram of the structure of an embodiment of the device of the present invention. Detailed Implementation

[0018] As can be seen from the background technology, current image quality adjustment is prone to image quality jitter.

[0019] Specifically, firstly, real-world scenes contain a large amount of semantic information and are extremely complex. For example, in the scenes corresponding to image quality adjustment classification models such as blue sky, white clouds, portraits, and buildings, the semantic information is not completely mutually exclusive. Secondly, most VPU parameters do not support parameter adjustment in arbitrary partitions. Instead, the current pixel value can be used to adjust modules such as color and DNLP (Dynamic Contrast Enhancement) through a lookup table.

[0020] Taking color adjustment as an example, suppose a scene transitions from blue sky to grass. The probability of the top-ranked blue sky in the classification model gradually decreases until the top-ranked grass changes from blue sky to grass, at which point the probability of the grass changing from blue sky to grass gradually increases. A typical image quality adjustment strategy is to increase the saturation of blue when the top-ranked color is blue, making the blue sky appear bluer; and to increase the saturation of green when the top-ranked color is grass, increasing the green color value. Even after the top-ranked color transitions from blue sky to grass, a large number of areas remain blue sky when labeled as grass. Traditional image quality adjustment algorithms directly switch the PQ (Picture Quality) parameters from blue sky to grass, which causes the saturation of blue in the remaining blue areas to suddenly decrease, while the saturation of green in the grass suddenly increases, resulting in color flickering and ultimately causing image quality jitter.

[0021] Simply using the results of a classification model for image quality adjustment ignores the stability issues of that model. Classification models are easily affected by noise, scene switching, and changes in scene content, leading to significant variations in their accuracy and probability.

[0022] If the top-N (N=3 or 5, etc.) results of the classification model are fused, or if only the top-1 results are used as input for image quality adjustment, it can easily lead to image quality jitter. Another approach is to perform temporal filtering on the output of the classification model to make the parameters acting on the VPU more stable. However, performing temporal filtering on the output of the classification model cannot fundamentally solve the problem of image quality jitter.

[0023] To address the aforementioned technical problem, this invention provides an image processing method that performs a scene switching judgment step, including any one or more of a first judgment, a second judgment, and a third judgment. The first judgment determines whether the classification label of the previous frame is included in the label set of the current frame and whether its confidence level with the corresponding initial label is greater than a preset threshold. Therefore, during scene switching, not only is the first initial label of the current frame compared with the classification label of the previous frame, but also whether the classification label of the previous frame is included in the label set of the current frame and its confidence level with the corresponding initial label are checked simultaneously. This helps prevent erroneous scene switching due to the accuracy of the classification model. The second judgment determines the difference between the color statistics information of the previous frame and the color statistics information of the current frame. The first judgment determines whether the value is within a preset difference range. By comparing the difference in color area between two frames, the risk of scene switching is judged, thereby improving the accuracy of scene switching judgment. The second judgment determines whether the proportion of pixels in the hue range corresponding to the features of the previous frame's classification label in the current frame image is greater than a preset proportion threshold. This allows for further judgment of the scene in the current frame, which is beneficial to improving the accuracy of scene switching judgment. In summary, the embodiments of the present invention can improve the accuracy of scene switching judgment, which is beneficial to reducing the probability of scene switching errors due to the accuracy of the classification model. Accordingly, when configuring the corresponding image quality parameters based on the classification label of the current frame, it is beneficial to improve the image quality jitter caused by scene switching errors, thereby improving the quality of image quality adjustment.

[0024] refer to Figure 1 The diagram illustrates a flowchart of an embodiment of the image processing method of the present invention. As an example, the image processing method includes the following basic steps:

[0025] Step S1: Provide a video frame sequence, including the first frame of the video frame sequence and the current frame after the first frame;

[0026] Step S2: Perform scene prediction processing on the current frame to obtain the label set corresponding to the current frame. The label set includes the top few initial labels whose prediction results best match the scene of the current frame, and the initial label with the highest accuracy is the first initial label.

[0027] Step S3: Obtain the classification label of the first frame;

[0028] Execute scene switching judgment step S5, which includes any one or more judgment steps among first judgment S51, second judgment S52 and third judgment S53;

[0029] The first determination S51 includes: determining whether the classification label of the previous frame adjacent to the current frame is included in the label set of the current frame and whether the confidence level with the corresponding initial label is greater than a preset threshold; the second determination S52 includes: determining whether the difference between the color statistics information of the previous frame and the color statistics information of the current frame is within a preset difference range; the third determination S53 includes: determining whether the proportion of pixels in the hue range corresponding to the label feature of the previous frame in the current frame image is greater than a preset proportion threshold.

[0030] Step S6: When at least one of the judgment results in the scene switching judgment step S5 is yes, set the classification label of the current frame to be the same as the classification label of the previous frame.

[0031] Otherwise, step S7: set the first initial label to the classification label of the current frame;

[0032] Step S8: Based on the classification label of the current frame, configure the corresponding image quality adjustment parameters for the current frame.

[0033] To make the above-mentioned objects, features and advantages of the embodiments of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0034] refer to Figure 1 Step S1: Provide a video frame sequence, including the first frame of the video frame sequence and the current frame after the first frame.

[0035] The video frame sequence is the frame sequence to be processed for image quality enhancement.

[0036] The multiple frames following the first frame of a video frame sequence are considered as the current frame.

[0037] Step S2: Perform scene prediction processing on the current frame to obtain the tag set corresponding to the current frame. The tag set includes the top N initial tags (TOP N) whose prediction results best match the scene of the current frame. The initial tag with the highest accuracy is the first initial tag TOP 1labi. Here, i refers to the position of the current frame in the video frame sequence, and i is a positive integer greater than or equal to 2.

[0038] Specifically, a classification model is used to predict the current frame, resulting in several classification results, which correspond to several initial labels. Among these, the initial label with the highest accuracy is designated as the first initial label TOP 1, meaning the initial label with the highest consistency with the current scene is used as the first initial label TOP 1.

[0039] As an example, the tag set includes the top 3 initial tags (i.e., TOP3) or the top 5 initial tags (i.e., TOP5) whose prediction results best match the current frame scene. In other embodiments, the tag set may contain other numbers of initial tags.

[0040] Classification models are susceptible to interference from noise, scene switching, and changes in scene content, which can lead to significant variations in the accuracy and probability of the classification model. Therefore, a preliminary scene prediction process is performed on the current frame to obtain the corresponding label set. This label set is then used to compare the current frame with the adjacent previous frame to determine whether a scene switch has occurred between the current frame and the previous frame.

[0041] Step S3: Obtain the classification label of the first frame.

[0042] The classification label of the first frame is obtained, so that subsequent frames after the first frame in the video frame sequence can be judged for scene switching based on the classification label of the first frame, in order to determine the classification label of the current frame.

[0043] Correspondingly, the previous frame adjacent to the current frame has a determined classification label, namely Label(i-1). The classification label of the previous frame adjacent to the current frame has been determined so that the scene of the current frame can be determined based on the classification label of the previous frame and the initial label of the current frame.

[0044] Specifically, in this embodiment, the step of obtaining the classification label of the first frame includes: performing scene prediction processing on the first frame to obtain a prediction result, which is used as the classification label of the first frame.

[0045] The image processing method further includes: Step S9: Based on the classification label of the first frame, configure corresponding image quality adjustment parameters for the first frame. Configuring corresponding image quality adjustment parameters for the first frame thereby performing corresponding image quality enhancement processing on the first frame.

[0046] Reference Figure 2 The diagram illustrates a flowchart of one embodiment of step S9. As one embodiment, the step of configuring corresponding image quality adjustment parameters for the first frame based on its classification label includes:

[0047] Step S91: Determine whether the category label of the first frame is the preset image quality adjustment scene.

[0048] Determine whether the category label of the first frame is a preset image quality adjustment scene, so as to determine whether the first frame needs to be processed for image quality adjustment corresponding to the preset image quality adjustment scene.

[0049] Accordingly, the image processing method further includes configuring multiple preset image quality adjustment scenes.

[0050] The preset image quality adjustment scenarios are pre-configured scenarios that require image quality adjustment. Specifically, based on experience in image quality adjustment, important scene categories can be pre-configured to allow for special image quality enhancement processing for important scene categories.

[0051] As an example, the preset image quality adjustment scenes may include, but are not limited to: faces, green, blue, buildings, documents, night scenes, water scenes, oceans, skies, snow scenes, etc.

[0052] Step S92: If the category label of the first frame is a preset image quality adjustment scene, configure the image quality adjustment parameters corresponding to the category label for the first frame based on the category label of the first frame.

[0053] Based on the classification label of the first frame, corresponding image quality adjustment parameters are configured for the first frame, so that different image quality adjustment parameters can be configured for images of different scenes, and corresponding image quality enhancement processing is performed on the current scene to obtain the best image quality enhancement effect.

[0054] The image quality (PQ) adjustment parameters corresponding to the category labels can include parameters such as color saturation, sharpening, contrast, and noise reduction. Specifically, the image quality adjustment parameters corresponding to the category labels are preset image quality adjustment parameters.

[0055] Step S93: Otherwise, configure the default image quality adjustment parameters for the first frame.

[0056] The default image quality adjustment parameters refer to the image quality adjustment parameters used to adapt to all images other than the preset image quality adjustment scenarios. Specifically, the default image quality adjustment parameters are pre-configured image quality adjustment parameters. The default image quality adjustment parameters can be parameters set based on experience.

[0057] In this embodiment, the image processing method further includes: obtaining color statistics information for each frame; the step of obtaining color statistics information for each frame includes: calculating the proportion of pixels with hue range values ​​corresponding to the features of the multiple preset image quality adjustment scenes in the image, as the color statistics information.

[0058] Obtain color statistics for each frame so that the difference in color area between the previous frame and the current frame can be compared.

[0059] As an example, a color statistical histogram is used to calculate the proportion of pixels in the image corresponding to the hue range values ​​of the features of the multiple preset image quality adjustment scenes.

[0060] refer to Figure 1 In an optional embodiment, the image processing method further includes: a pre-judgment step S4 for the current frame, used to determine whether the first initial label of the current frame is the same as the classification label of the previous frame.

[0061] Determine whether the first initial label of the current frame is the same as the classification label of the previous frame, and thus determine whether a scene switch has occurred in the current frame compared to the previous frame.

[0062] Specifically, determine whether the first initial label of the current frame, labi, is equal to the classification label Label(i-1) of the previous frame.

[0063] Continue to refer to Figure 1 When the result of the pre-judgment step S4 is yes, step S6 is executed: set the classification label Labeli of the current frame to be the same as the classification label Label(i-1) of the previous frame, that is, let Labeli = Label(i-1).

[0064] When the first initial label of the current frame is the same as the classification label of the previous frame, it can be preliminarily determined that no scene switching has occurred in the current frame compared to the previous frame. Therefore, the classification label of the current frame is made the same as the classification label of the previous frame.

[0065] It should be noted that in this embodiment, the image processing method is described using pre-judgment as an example. In other embodiments, the image processing method may omit the pre-judgment.

[0066] It should also be noted that in this embodiment, the pre-judgment step S4 is performed before the scene switching judgment step S5.

[0067] refer to Figure 1 The scene switching judgment step S5 is executed, including any one or more judgment steps among the first judgment S51, the second judgment S52 and the third judgment S53.

[0068] To prevent scene prediction errors caused by the accuracy and stability of the classification model, a scene switching judgment step S5 is performed to reduce erroneous scene switching caused by the model's classification accuracy and stability, thereby helping to improve the image quality jitter caused by erroneous scene switching.

[0069] In this embodiment, if the result of the pre-judgment step S4 is negative, the scene switching judgment step S5 is executed.

[0070] The first determination S51 includes: determining whether the classification label of the previous frame is included in the label set (i.e., TOP N) of the current frame and whether the confidence of the corresponding initial label is greater than a preset threshold; the second determination S52 includes: determining whether the difference between the color statistics of the previous frame and the color statistics of the current frame is within a preset difference range; the third determination includes: determining whether the proportion of pixels in the hue range corresponding to the classification label feature of the previous frame in the current frame image is greater than a preset proportion threshold.

[0071] By using the first judgment S51, during scene switching, not only is the first initial label of the current frame compared with the classification label of the previous frame, but the classification label of the previous frame is also checked to see if it is included in the label set of the current frame and its confidence level with the corresponding initial label. This helps to prevent erroneous scene switching due to the accuracy of the classification model.

[0072] By using the second judgment S52, the difference in color statistical information between two adjacent frames is judged, thereby judging the risk of scene switching by comparing the difference in color area between the two frames, and thus improving the accuracy of scene switching judgment.

[0073] The third judgment S53 determines whether the proportion of pixels in the tonal range corresponding to the features of the previous frame label in the current frame image is greater than a preset proportion threshold. This allows for the determination of the scene correlation and similarity between the previous frame and the current frame, enabling further judgment of the scene in the current frame. This further reduces classification errors caused by the accuracy and stability of the classification model, and helps improve the accuracy of scene switching judgment.

[0074] It should be noted that when the scene switching judgment step S5 includes any multiple judgment steps among the first judgment S51, the second judgment S52 and the third judgment S53, any multiple judgments among the first judgment S51 and the second judgment S52 and the third judgment S53 are executed in parallel.

[0075] The precision and degree of the first judgment S51, the second judgment S52, and the third judgment S53 are different. Compared with the scene switching judgment step S5 which only includes any one of the first judgment S51, the second judgment S52, and the third judgment S53, when the scene switching judgment step S5 includes any multiple of the first judgment S51, the second judgment S52, and the third judgment S53, the scene switching judgment can be more comprehensive, which is conducive to further improving the accuracy of the scene switching judgment and thus making the image quality more stable.

[0076] Alternatively, the judgment steps of the first judgment S51, the second judgment S52 and the third judgment S53 have a sequential order, with the judgment order of the first judgment S53 preceding the judgment order of the second judgment S52 and the third judgment S53, and the judgment order of the second judgment S52 preceding the judgment order of the third judgment S53, and the next judgment step is executed when the judgment result of the previous judgment step is negative.

[0077] The first judgment S51, the second judgment S52, and the third judgment S53 differ in their methods and precision for determining scene switching: the first judgment S51 compares the classification label of the previous frame with the label set of the current frame to determine whether a scene switch has occurred; the second judgment S52 determines scene switching by comparing the color statistics between the two frames; and the third judgment S53 determines scene switching by comparing the proportion of the features of the classification label of the previous frame in the current frame image.

[0078] When the first judgment S51, the second judgment S52 and the third judgment S53 have the above-mentioned judgment order, if the judgment result of the previous judgment step is negative, it is initially judged that a scene switch has been performed. At this time, by performing the next judgment, it is possible to further judge whether a scene switch has been performed, which is conducive to improving the accuracy of the scene switch judgment.

[0079] As an example, the scene switching judgment step S5 includes a first judgment S51, a second judgment S52, and a third judgment S53 performed sequentially or in parallel.

[0080] In this embodiment, the scene switching judgment step S5 includes a first judgment S51, a second judgment S52, and a third judgment S53 performed sequentially as an example for explanation.

[0081] The scene switching judgment step S5 includes a first judgment S51, a second judgment S52, and a third judgment S53 performed sequentially, which enables a more comprehensive judgment on scene switching, helps to further improve the accuracy of scene switching judgment, and thus makes the image quality more stable.

[0082] In other embodiments, the scene switching step may also include only one or any two of the first judgment, second judgment, and third judgment steps.

[0083] The specific steps of the scene switching judgment step S5 in this embodiment will be described in detail below with reference to the accompanying drawings.

[0084] refer to Figure 1 Execute the first judgment S51, which is used to determine whether the classification label of the previous frame is included in the label set of the current frame and whether the confidence of the corresponding initial label is greater than a preset threshold.

[0085] When the judgment result of the first judgment S51 is yes, step S6: set the classification label of the current frame to be the same as the classification label of the previous frame.

[0086] If the classification label Label(i-1) of the current frame is included in the Top N of the label set of the current frame, and the confidence of the label is greater than the preset threshold, then the probability of misclassifying the current frame due to the accuracy and stability of the classification model is high, and the risk of scene switching is too great. Therefore, the classification label of the current frame is set to be the same as the classification label of the previous frame, that is, the current frame does not switch scenes compared to the previous frame.

[0087] The confidence level between the classification label Label(i-1) of the previous frame and the corresponding initial label refers to the confidence level between the classification label Label(i-1) of the previous frame and the initial label when the classification label Label(i-1) of the previous frame is the same as an initial label in the Top N of the label set of the current frame.

[0088] When the result of the first judgment S51 is negative, the second judgment S52 is executed to determine whether the difference between the color statistics information of the previous frame and the color statistics information of the current frame is within a preset difference range.

[0089] In this embodiment, the second determination S52 includes: determining whether the difference between the color statistics information of the previous frame and the color statistics information of the current frame is within a preset difference range.

[0090] When the judgment result of the second judgment S52 is yes, step S6: set the classification label Label i of the current frame to be the same as the classification label Label (i-1) of the previous frame, that is, let Label i = Label (i-1).

[0091] By judging the difference in color statistics between two adjacent frames, and by comparing the difference in color area between the two frames, the risk of scene switching can be judged, thereby improving the accuracy of scene switching judgment.

[0092] Specifically, the smaller the difference in color area between two adjacent frames, the lower the probability of a scene switch occurring between the current frame and the previous frame, and the greater the risk of switching image quality adjustment parameters. Therefore, when the difference between the color statistics of the current frame and the color statistics of the previous frame is within a preset difference range, it can be determined that no scene switch has occurred, and there is no need to switch image quality adjustment parameters, thereby reducing the risk of image quality jitter caused by accidental scene switching. Accordingly, the classification label Label i of the current frame is set to be the same as the classification label Label (i-1) of the previous frame.

[0093] The greater the difference in color area between two adjacent frames, the higher the probability of scene switching in the current frame compared to the previous frame, and the lower the risk of switching image quality adjustment parameters. Therefore, the speed of switching image quality adjustment parameters can be accelerated.

[0094] In practice, a statistical information difference threshold Dth can be preset. By judging whether the difference between the color statistical information of two adjacent frames is less than or equal to the statistical information difference threshold Dth, it can be determined whether a scene switch has occurred.

[0095] In practical implementation, the actual setting of the statistical information difference threshold Dth requires testing the specific differences in color statistical values ​​during scene transitions in various videos, and statistically obtaining an empirical value as a reference. This embodiment limits the specific value of the statistical information difference threshold Dth.

[0096] When the result of the second judgment S52 is negative, the third judgment S53 is executed to determine whether the proportion of pixels in the hue range corresponding to the feature of the previous frame label in the current frame image is greater than a preset proportion threshold.

[0097] Specifically, it determines whether the proportion of pixels in the tonal range corresponding to the features of the previous frame label in the current frame image is greater than a preset proportion threshold.

[0098] When the judgment result of the third judgment S53 is yes, step S6: set the classification label Label i of the current frame to be the same as the classification label Label (i-1) of the previous frame, that is, let Label i = Label (i-1).

[0099] The third judgment S53 is used to determine whether the proportion of pixels in the tonal range corresponding to the features of the previous frame classification label in the current frame image is greater than a preset proportion threshold, thereby making further judgments on the scene similarity and correlation between the current frame and the previous frame, further reducing classification errors caused by the accuracy and stability of the classification model, and helping to improve the accuracy of scene switching judgment.

[0100] Accordingly, in this embodiment, the image processing method further includes configuring a hue range value corresponding to the features of the preset image quality adjustment scene. Specifically, the hue range value can be adjusted according to customer needs and actual conditions.

[0101] As an example, taking facial features as an example, the hue H range value can be set to: 0<H≤12, or 176≤H≤179; taking green features as an example, the hue H range value can be set to: 35≤H≤62; taking blue features as an example, the hue H range value can be set to: 97≤H≤113.

[0102] It should be noted that the tonal range values ​​of the above features are only examples, and the tonal range values ​​can be adjusted according to customer needs and actual conditions.

[0103] The preset percentage threshold is used to further determine the current scene classification result. This preset percentage threshold can be a value set in advance based on image quality adjustment experience. It should be noted that the preset percentage threshold is determined based on customer needs and actual conditions; this embodiment does not limit the specific value of the preset percentage threshold.

[0104] As an example, taking a face as an example, when the classification label of the previous frame is "Skin", if the proportion of the hue range value corresponding to the skin color in the current frame image is greater than a certain threshold (e.g., 5%), it can be determined that the current frame has not undergone a scene change compared to the previous frame, that is, the classification label of the current frame is still "Skin". Accordingly, the classification label Label i of the current frame is set to be the same as the classification label Label(i-1) of the previous frame.

[0105] Continue to refer to Figure 1 When at least one of the judgment results in the scene switching judgment step S5 is yes, step S6 is executed to set the classification label of the current frame to be the same as the classification label of the previous frame, i.e., Label i = Label (i-1).

[0106] When at least one of the judgment results in the scene switching judgment step S5 is yes, it can be determined that no scene switching has occurred in the current frame compared to the previous frame. Therefore, the same classification label as the previous frame is assigned to the current frame.

[0107] Specifically, in this embodiment, when the judgment result of the pre-judgment step S4 or at least one of the judgment results of the scene switching judgment step S5 is yes, the classification label of the current frame is set to be the same as the classification label of the previous frame.

[0108] Otherwise, proceed to step S7: set the first initial label to the classification label of the current frame, that is, set the classification label of the current frame to the value of the model inference Top1 (i.e., the first initial label), i.e., Labeli = labi.

[0109] When all judgment results in the scene switching judgment step S5 are negative, it indicates that the current frame has changed scene compared to the previous frame, and the judgment accuracy of the scene switching of the current frame is high. Accordingly, the first initial label of the current frame can be set as the classification label of the current frame.

[0110] Specifically, when the judgment result of the pre-judgment S4 and all judgment results in the scene switching judgment step S5 are negative, the first initial label of the current frame is assigned to the current frame.

[0111] Continue to refer to Figure 1 Step S8: Based on the classification label of the current frame, configure the corresponding image quality adjustment parameters for the current frame.

[0112] Based on the classification label of the current frame, configure the corresponding image quality adjustment parameters for the current frame so as to perform image quality enhancement processing in the corresponding mode for the current frame.

[0113] This embodiment improves the accuracy of scene switching judgment through the scene switching step S5, which helps to reduce the probability of scene switching errors caused by the accuracy of the classification model. Correspondingly, when configuring the corresponding image quality parameters based on the classification label of the current frame, it helps to improve the image quality jitter caused by scene switching errors, thereby improving the quality of image quality adjustment.

[0114] Reference Figure 3 In this embodiment, the step of configuring corresponding image quality parameters for the current frame based on the classification label of the current frame includes: Step S81: Determine whether the classification label of the current frame is a preset image quality adjustment scene.

[0115] If so, proceed to step S82: Configure image quality adjustment parameters corresponding to the classification label for the current frame.

[0116] When the classification label of the current frame belongs to the preset image quality adjustment scene, the corresponding image quality adjustment parameters are configured for the current frame based on the classification label of the current frame. Different image quality adjustment parameters can be configured for images in different scenes, and the image quality improvement processing of the current scene in the corresponding mode can be performed accordingly to obtain the best image quality improvement effect.

[0117] For example, if the current frame is categorized as a face, the image quality is adjusted accordingly to adjust the skin tone, which is equivalent to entering Skin PQ mode.

[0118] The image quality (PQ) adjustment parameters corresponding to the category labels can include parameters such as color saturation, sharpening, contrast, and noise reduction. Specifically, the image quality adjustment parameters corresponding to the category labels are preset image quality adjustment parameters.

[0119] Continue to refer to Figure 3 Otherwise, proceed to step S83: set the classification label of the current frame to the default label, and configure the default image quality parameters for the current frame.

[0120] The default image quality adjustment parameters refer to the image quality adjustment parameters used to adapt to all images except the preset image quality adjustment scenarios.

[0121] Specifically, the default image quality adjustment parameters are pre-configured image quality adjustment parameters. These default image quality adjustment parameters can be set based on experience.

[0122] Accordingly, the present invention also provides an image processing apparatus. Figure 4 This is a functional block diagram of an embodiment of the image processing apparatus of the present invention.

[0123] The image processing apparatus 10 of this embodiment will be described in detail below with reference to the accompanying drawings.

[0124] refer to Figure 4 The image processing apparatus 10 includes: a providing unit 101, used to provide a video frame sequence, including the first frame of the video frame sequence and the current frame after the first frame;

[0125] The scene prediction unit 102 is used to obtain the classification label of the first frame and to perform scene prediction processing on the current frame to obtain the label set corresponding to the current frame. The label set includes the top few initial labels whose prediction results best match the scene of the current frame, and the initial label with the highest accuracy is the first initial label.

[0126] Scene switching judgment unit 104 is used to perform scene switching judgment processing on the current frame. The scene switching judgment unit includes any one or more judgment subunits among the first judgment subunit 1041, the second judgment subunit 1042 and the third judgment subunit 1043.

[0127] The first judgment subunit 1041 is used to determine whether the classification label of the previous frame is included in the label set of the current frame and whether the confidence of the corresponding initial label is greater than a preset threshold, and outputs a first judgment result; the second judgment subunit 1042 is used to determine whether the difference between the color statistics information of the previous frame and the color statistics information of the current frame is within a preset difference range, and outputs a second judgment result; the third judgment subunit 1043 is used to determine whether the proportion of the hue range corresponding to the feature of the label of the previous frame in the current frame image is greater than a preset proportion threshold, and outputs a third judgment result.

[0128] The label setting unit 106 is used to set the classification label of the current frame to be the same as the classification label of the previous frame when the judgment result of at least one judgment subunit in the scene switching judgment unit 104 is yes; otherwise, the label setting unit 106 is used to set the first initial label to the classification label of the current frame.

[0129] The image quality adjustment unit 105 is used to configure the corresponding image quality adjustment parameters for the current frame based on the classification label of the current frame.

[0130] In this embodiment, the image processing device 10 further includes a scene switching judgment unit 104, which is used to perform scene switching judgment processing on the current frame. The scene switching judgment unit 104 includes any one or more judgment subunits among the first judgment subunit 1041, the second judgment subunit 1042, and the third judgment subunit 1043.

[0131] The first judgment subunit 1041 is used to determine whether the classification label of the previous frame is included in the label set of the current frame and whether the confidence level with the corresponding initial label is greater than a preset threshold. Therefore, when switching scenes, not only is the first initial label of the current frame compared with the classification label of the previous frame, but also the classification label of the previous frame is checked to see whether it is included in the label set of the current frame and the confidence level with the corresponding initial label. This helps to prevent scene switching errors caused by the accuracy of the classification model.

[0132] The second judgment subunit 1042 is used to determine whether the difference between the color statistics information of the previous frame and the color statistics information of the current frame is within a preset difference range. By comparing the difference in color area between the two frames, the risk of scene switching is judged, thereby improving the accuracy of scene switching judgment.

[0133] The third judgment subunit 1043 is used to determine whether the proportion of pixels in the tonal range corresponding to the features of the previous frame label in the current frame image is greater than a preset proportion threshold, thereby enabling further judgment of the scene in the current frame, which helps to improve the accuracy of scene switching judgment.

[0134] In summary, this embodiment can improve the accuracy of scene switching judgment, which helps to reduce the probability of scene switching errors caused by the accuracy of the classification model. Accordingly, when the image quality adjustment unit 105 configures the corresponding image quality parameters based on the classification label of the current frame, it helps to improve the image quality jitter caused by scene switching errors, thereby improving the quality of image quality adjustment.

[0135] The providing unit 101 is used to provide a video frame sequence, including the first frame of the video frame sequence and the current frame after the first frame.

[0136] The video frame sequence is the sequence of frames to be processed for image quality enhancement. The multiple frames following the first frame of the video frame sequence are considered the current frame.

[0137] The scene prediction unit 102 is used to obtain the classification label of the first frame.

[0138] The scene prediction unit 102 obtains the classification label of the first frame, so that the scene switching judgment unit 104 can perform scene switching judgment on the current frame after the first frame in the video frame sequence based on the classification label of the first frame, so as to determine the classification label of the current frame.

[0139] Specifically, in this embodiment, the scene prediction unit 102 is used to perform scene prediction processing on the first frame to obtain a prediction result, which serves as the classification label for the first frame.

[0140] The scene prediction unit 102 is also used to perform scene prediction processing on the current frame to obtain the label set corresponding to the current frame. The label set includes the top N initial labels (TOP N) whose prediction results best match the scene of the current frame. The initial label with the highest accuracy is the first initial label TOP 1labi. The previous frame adjacent to the current frame has a determined classification label.

[0141] Specifically, the scene prediction unit 102 can use a classification model to predict the scene of the current frame and obtain several classification results, which correspond to several initial labels. Among them, the initial label with the highest accuracy is the first initial label TOP 1, that is, the initial label with the highest conformity to the current scene is used as the first initial label TOP 1labi.

[0142] As an example, the tag set includes the top 3 initial tags (i.e., TOP3) or the top 5 initial tags (i.e., TOP5) whose prediction results best match the current frame scene. In other embodiments, the tag set may contain other numbers of initial tags.

[0143] Classification models are susceptible to interference from noise, scene switching, and changes in scene content, which can lead to significant variations in the accuracy and probability of the classification model. Therefore, the scene prediction unit 102 performs a preliminary scene prediction process on the current frame to obtain the label set corresponding to the current frame. This allows the scene switching judgment unit 104 to compare the current frame with the adjacent previous frame based on the label set, thereby improving the accuracy of scene switching judgment.

[0144] The previous frame adjacent to the current frame has a determined classification label, namely Label(i-1). The classification label of the previous frame adjacent to the current frame has been determined so that the pre-judgment unit 103 and the scene switching judgment unit 104 can determine whether the scene is switching based on the classification label of the previous frame and the initial label of the current frame.

[0145] In an optional embodiment, the image processing device 10 may further include: a pre-judgment unit 103, configured to determine whether the first initial label of the current frame is the same as the classification label of the previous frame, and output a pre-judgment result; and configured to set the classification label of the current frame to be the same as the classification label of the previous frame when the pre-judgment result is yes.

[0146] The pre-judgment unit 103 is used to determine whether the first initial label of the current frame is the same as the classification label of the previous frame, thereby determining whether a scene switch has occurred in the current frame compared to the previous frame.

[0147] Specifically, the pre-judgment unit 103 determines whether the first initial label of the current frame, labi, is equal to the classification label Label(i-1) of the previous frame.

[0148] When the pre-judgment result is yes, the pre-judgment unit 103 sets the classification label of the current frame to be the same as the classification label of the previous frame, that is, let Labeli = Label(i-1).

[0149] When the first initial label of the current frame is the same as the classification label of the previous frame, it can be preliminarily determined that no scene switching has occurred in the current frame compared to the previous frame. Therefore, the classification label of the current frame is made the same as the classification label of the previous frame.

[0150] It should be noted that in this embodiment, the image processing device 10 further includes a pre-judgment unit 103 as an example for description. In other embodiments, the image processing device 10 may not include the pre-judgment unit.

[0151] The scene switching judgment unit 104 is used to perform scene switching judgment processing on the current frame. The scene switching judgment unit 104 includes any one or more judgment subunits among the first judgment subunit 1041, the second judgment subunit 1042 and the third judgment subunit 1043.

[0152] To prevent scene prediction errors caused by the accuracy and stability of the classification model, the scene switching judgment unit 104 performs scene switching judgment processing on the current frame to reduce erroneous scene switching caused by the accuracy and stability of the model classification, thereby helping to improve the problem of image quality jitter caused by erroneous scene switching.

[0153] In this embodiment, the scene switching judgment unit 104 is connected to the pre-judgment unit 103. The scene switching judgment unit 104 is used to perform scene switching judgment processing on the current frame when the pre-judgment result output by the pre-judgment unit 103 is negative.

[0154] The first judgment subunit 1041 is used to determine whether the classification label of the previous frame is included in the label set (i.e., TOP N) of the current frame and whether the confidence of the corresponding initial label is greater than a preset threshold, and outputs a first judgment result; the second judgment subunit 1042 is used to determine whether the difference between the color statistics information of the previous frame and the color statistics information of the current frame is within a preset difference range, and outputs a second judgment result; the third judgment subunit 1043 is used to determine whether the proportion of pixels in the hue range corresponding to the classification label feature of the previous frame in the current frame image is greater than a preset proportion threshold, and outputs a third judgment result.

[0155] By setting the first judgment subunit 1041, when switching scenes, not only is the first initial label of the current frame compared with the classification label of the previous frame, but the classification label of the previous frame is also checked whether it is included in the label set of the current frame and the confidence level with the corresponding initial label. This helps to prevent scene switching errors caused by the accuracy of the classification model.

[0156] By setting a second judgment subunit 1042, the difference in color statistical information between two adjacent frames is judged. By comparing the difference in color area between the two frames, the risk of scene switching is judged, thereby improving the accuracy of scene switching judgment.

[0157] By setting a third judgment subunit 1043, it is used to determine whether the proportion of pixels in the tonal range corresponding to the features of the previous frame label in the current frame image is greater than a preset proportion threshold. This enables the determination of the scene correlation and similarity between the previous frame and the current frame, further judging the scene of the current frame, further reducing classification errors caused by the accuracy and stability issues of the classification model, and improving the accuracy of scene switching judgment.

[0158] When the scene switching judgment unit 104 includes any and more judgment subunits among the first judgment subunit 1041, the second judgment subunit 1042, and the third judgment subunit 1043, the multiple judgment subunits are connected in parallel, or the multiple judgment subunits are connected sequentially. When the multiple judgment subunits are connected sequentially, the judgment order of the first judgment subunit 1041 is before the judgment order of the second judgment subunit 1042 and the third judgment subunit 1043, the judgment order of the second judgment subunit 1042 is before the judgment order of the third judgment subunit 1043, and the next judgment subunit is connected when the judgment result output by the previous judgment subunit is negative.

[0159] As an example, the scene switching judgment unit 104 includes a first judgment subunit 1041, a second judgment subunit 1042, and a third judgment subunit 1043 connected sequentially or in parallel.

[0160] In this embodiment, the scene switching judgment step S5 includes a first judgment subunit 1041, a second judgment subunit 1042, and a third judgment subunit 1043 connected in sequence as an example for illustration.

[0161] The scene switching judgment unit 104 includes a first judgment subunit 1041, a second judgment subunit 1042, and a third judgment subunit 1043 connected in sequence. Thus, the scene switching judgment unit 104 can make a more comprehensive judgment on scene switching, which is conducive to further improving the accuracy of scene switching judgment and making the image quality more stable.

[0162] The following combination Figure 5 The scene switching judgment unit 104 described in this embodiment will be described in detail.

[0163] refer to Figure 5 The first judgment subunit 1041 is used to determine whether the classification label of the previous frame is included in the label set of the current frame, and whether the confidence of the corresponding initial label is greater than a preset threshold, and outputs the first judgment result.

[0164] When the classification label of the previous frame is included in the label set of the current frame and the confidence of the corresponding initial label is greater than a preset threshold, the probability of misclassifying the current frame due to the accuracy and stability of the classification model is high, and the risk of scene switching is too great.

[0165] When the first judgment result is negative, the second judgment subunit 1042 is connected to determine whether the difference between the color statistics information of the previous frame and the color statistics information of the current frame is within a preset difference range, and the second judgment result is output.

[0166] The second judgment subunit 1042 judges the difference in color statistical information between two adjacent frames, thereby judging the risk of scene switching by comparing the difference in color area between the two frames, and thus improving the accuracy of scene switching judgment.

[0167] Specifically, the smaller the difference in color area between two adjacent frames, the lower the probability of a scene switch occurring between the current frame and the previous frame. Correspondingly, the risk of switching image quality adjustment parameters is greater. Therefore, if the difference between the color statistics of the current frame and the previous frame is within a preset range, it can be determined that no scene switch has occurred, and there is no need to switch image quality adjustment parameters, thereby reducing the risk of image quality jitter caused by accidental scene switching. Accordingly, the classification label of the current frame is set to be the same as the classification label of the previous frame.

[0168] The greater the difference in color area between two adjacent frames, the higher the probability of scene switching and the lower the risk of switching image quality adjustment parameters. Therefore, the speed of switching image quality adjustment parameters can be accelerated.

[0169] In practice, a statistical information difference threshold Dth can be preset. By judging whether the difference between the color statistical information of two adjacent frames is less than or equal to the statistical information difference threshold Dth, it can be determined whether a scene switch has occurred.

[0170] In practice, the actual setting of the statistical information difference threshold Dth requires testing the specific difference in color statistical values ​​during scene switching in each video, and statistically obtaining an empirical value as a reference.

[0171] When the second judgment result is negative, the third judgment subunit 1043 is connected to determine whether the proportion of pixels in the tone range corresponding to the previous frame label feature in the current frame image is greater than a preset proportion threshold, and the third judgment result is output.

[0172] When the third judgment result is yes, the third judgment subunit 1043 is executed to set the first initial label as the classification label of the current frame.

[0173] The third judgment subunit 1043 is used to determine whether the proportion of pixels in the tonal range corresponding to the features of the previous frame label in the current frame image is less than a preset proportion threshold, so as to further judge the scene of the current frame, further reduce the classification error caused by the accuracy and stability of the classification model, and help improve the accuracy of scene switching judgment.

[0174] As an example, taking a human face as an example, when the first initial label of the current frame is skin, if the proportion of the hue range value corresponding to the skin color in the image is greater than a certain threshold (e.g., 5%), it can be determined that the scene classification of the current frame is correct, and the classification label of the current frame is set to the value of the model inference Top1 (i.e., the first initial label), that is, Labeli = labi.

[0175] The label setting unit 106 is used to set the classification label of the current frame to be the same as the classification label of the previous frame when the judgment result of at least one judgment subunit in the scene switching judgment unit 104 is yes; otherwise, the label setting unit 106 is used to set the first initial label to the classification label of the current frame.

[0176] When the judgment result of at least one judgment subunit in the scene switching judgment unit 104 is yes, it can be determined that no scene switching has occurred in the current frame compared to the previous frame. Therefore, the same classification label as the previous frame is assigned to the current frame.

[0177] Specifically, in this embodiment, when the result of the pre-judgment unit 103 or the judgment result of at least one judgment subunit in the scene switching judgment unit 104 is yes, the label setting unit 106 sets the classification label of the current frame to be the same as the classification label of the previous frame.

[0178] Otherwise, the label setting unit 106 sets the first initial label to the classification label of the current frame, that is, the classification label of the current frame is set to the value of the model inference Top1 (i.e. the first initial label), i.e., Labeli = labi.

[0179] When the judgment results of all judgment subunits in the scene switching judgment step S5 are negative, it indicates that the current frame has changed scene compared to the previous frame, and the judgment accuracy of the current frame changing scene is high. Accordingly, the first initial label of the current frame can be set as the classification label of the current frame.

[0180] Specifically, when the judgment result of the pre-judgment unit 103 and the judgment results of all judgment subunits in the scene switching judgment unit 104 are both negative, the first initial label of the current frame is assigned to the current frame.

[0181] The image quality adjustment unit 105 is used to configure corresponding image quality adjustment parameters for the current frame based on the classification label of the current frame.

[0182] Based on the classification label of the current frame, configure the corresponding image quality adjustment parameters for the current frame so as to perform image quality enhancement processing in the corresponding mode for the current frame.

[0183] This embodiment improves the accuracy of scene switching judgment through the scene switching judgment unit 104, which helps to reduce the probability of scene switching errors caused by the accuracy of the classification model. Accordingly, when configuring the corresponding image quality parameters based on the classification label of the current frame, it helps to improve the image quality jitter caused by scene switching errors, thereby improving the quality of image quality adjustment.

[0184] Specifically, in this embodiment, the image quality adjustment unit 105 is used to determine whether the classification label of the current frame is a preset image quality adjustment scene; if so, the image quality adjustment unit 105 configures the image quality adjustment parameters corresponding to the classification label for the current frame.

[0185] When the classification label of the current frame belongs to the preset image quality adjustment scene, the corresponding image quality adjustment parameters are configured for the current frame based on the classification label of the current frame. Different image quality adjustment parameters can be configured for images in different scenes, and the image quality improvement processing of the current scene in the corresponding mode can be performed accordingly to obtain the best image quality improvement effect.

[0186] For example, if the classification label of the current frame is a face, the image quality adjustment unit 105 performs corresponding skin tone quality adjustment processing on the image, that is, enters the skin PQ mode.

[0187] The image quality (PQ) adjustment parameters corresponding to the category labels can include parameters such as color saturation, sharpening, contrast, and noise reduction. Specifically, the image quality adjustment parameters corresponding to the category labels are preset image quality adjustment parameters.

[0188] Furthermore, by setting the scene switching judgment unit 104, this embodiment improves the accuracy of scene switching judgment, which helps to reduce the probability of scene switching errors caused by the accuracy of the classification model. Accordingly, when configuring the corresponding image quality parameters based on the classification label of the current frame, it helps to improve the image quality jitter caused by scene switching errors, thereby improving the quality of image quality adjustment.

[0189] When the classification label of the current frame does not belong to the preset image quality adjustment scenario, the image quality adjustment unit 105 sets the classification label of the current frame as the default label and configures the default image quality parameters for the current frame.

[0190] The default image quality adjustment parameters refer to the image quality adjustment parameters used to adapt to all images except the preset image quality adjustment scenarios.

[0191] Specifically, the default image quality adjustment parameters are pre-configured image quality adjustment parameters. These default image quality adjustment parameters can be set based on experience.

[0192] The image quality adjustment unit 105 is also used to configure corresponding image quality adjustment parameters for the first frame based on the first initial tag of the first frame.

[0193] Specifically, in this embodiment, the image quality adjustment unit 105 is used to determine whether the first initial label of the first frame is a preset image quality adjustment scene, so as to determine whether the first frame needs to be processed for image quality adjustment corresponding to the preset image quality adjustment scene.

[0194] If the category label of the first frame belongs to a preset image quality adjustment scene, the image quality adjustment unit 105 is used to configure image quality adjustment parameters corresponding to the category label for the first frame based on the category label of the first frame.

[0195] Based on the classification label of the first frame, corresponding image quality adjustment parameters are configured for the first frame, so that different image quality adjustment parameters can be configured for images of different scenes, and corresponding image quality enhancement processing is performed on the current scene to obtain the best image quality enhancement effect.

[0196] The image quality (PQ) adjustment parameters corresponding to the category labels can include parameters such as color saturation, sharpening, contrast, and noise reduction. Specifically, the image quality adjustment parameters corresponding to the category labels are preset image quality adjustment parameters.

[0197] Otherwise, the image quality adjustment unit 105 is used to configure default image quality adjustment parameters for the first frame.

[0198] The default image quality adjustment parameters refer to the image quality adjustment parameters used to adapt to all images other than the preset image quality adjustment scenarios. Specifically, the default image quality adjustment parameters are pre-configured image quality adjustment parameters. The default image quality adjustment parameters can be parameters set based on experience.

[0199] Accordingly, embodiments of the present invention also provide a device that can implement the image processing method provided by the present invention by loading a program in the form of the above-described image processing method.

[0200] An optional hardware structure of the device provided in this embodiment of the invention can be as follows: Figure 6 As shown, it includes: at least one processor 01, at least one communication interface 02, at least one memory 03, and at least one communication bus 04.

[0201] In this embodiment of the invention, the number of processor 01, communication interface 02, memory 03 and communication bus 04 is at least one, and processor 01, communication interface 02 and memory 03 communicate with each other through communication bus 04.

[0202] Optionally, the communication interface 02 can be an interface for a communication module used for network communication, such as the interface of a GSM module.

[0203] Optionally, processor 01 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0204] Optionally, memory 03 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0205] The memory 03 stores one or more computer instructions, which are executed by the processor 01 to implement the image processing method provided in this embodiment of the invention.

[0206] It should be noted that the aforementioned terminal device may also include other devices (not shown) that may not be essential to understanding the content disclosed in the embodiments of the present invention; given that these other devices may not be essential for understanding the content disclosed in the embodiments of the present invention, the embodiments of the present invention will not describe them one by one.

[0207] Accordingly, embodiments of the present invention also provide a storage medium storing one or more computer instructions, which are used to implement the image processing method described in the embodiments of the present invention.

[0208] The storage medium is a computer-readable storage medium, which can be a read-only memory (ROM), random access memory (RAM), USB flash drive, portable hard drive, magnetic disk or optical disk, or any other medium that can store program code.

[0209] The embodiments of the present invention described above are combinations of elements and features of the present invention. Unless otherwise stated, the elements or features described are optional. Individual elements or features may be practiced without combination with other elements or features. Furthermore, embodiments of the present invention may be constructed by combining some elements and / or features. The order of operations described in the embodiments of the present invention may be rearranged. Some constructions of any embodiment may be included in another embodiment and may be replaced by corresponding constructions of another embodiment. It will be apparent to those skilled in the art that claims in the appended claims that are not expressly referenced to each other may be combined to form embodiments of the present invention, or may be included as new claims in amendments made after the filing of this application.

[0210] Embodiments of the present invention can be implemented by various means, such as hardware, firmware, software, or combinations thereof. In a hardware configuration, the method according to an exemplary embodiment of the present invention can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0211] In firmware or software configuration, embodiments of the present invention can be implemented in the form of modules, processes, functions, etc. Software code can be stored in a memory unit and executed by a processor. The memory unit is located inside or outside the processor and can send data to and receive data from the processor via various known means.

[0212] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. An image processing method, characterized in that, include: Provide a video frame sequence, including the first frame of the video frame sequence and the current frame after the first frame; Obtain the category label of the first frame; Perform scene prediction processing on the current frame to obtain the label set corresponding to the current frame. The label set includes the top few initial labels whose prediction results best match the scene of the current frame. The initial label with the highest accuracy is the first initial label. The execution scenario switching judgment steps include any one or more of the first judgment, second judgment, and third judgment steps; The first determination includes: determining whether the classification label of the previous frame adjacent to the current frame is included in the label set of the current frame and whether the confidence level of the classification label of the previous frame with the corresponding initial label is greater than a preset threshold; the confidence level of the classification label of the previous frame with the corresponding initial label refers to the confidence level of the classification label of the previous frame with the initial label when the classification label of the previous frame is the same as an initial label in the label set of the current frame; the second determination includes: determining whether the difference between the color statistics information of the previous frame and the color statistics information of the current frame is within a preset difference range; the third determination includes: determining whether the proportion of pixels in the hue range corresponding to the classification label feature of the previous frame in the current frame image is greater than a preset proportion threshold; When at least one of the judgment results in the scene switching judgment step is yes, the classification label of the current frame is set to be the same as the classification label of the previous frame. Otherwise, set the first initial label to the classification label of the current frame; Based on the classification label of the current frame, configure the corresponding image quality parameters for the current frame.

2. The image processing method as described in claim 1, characterized in that, The steps to obtain the classification label of the first frame include: performing scene prediction processing on the first frame to obtain a prediction result, which is used as the classification label of the first frame; The image processing method further includes: configuring corresponding image quality adjustment parameters for the first frame based on the classification label of the first frame.

3. The image processing method as described in claim 2, characterized in that, The step of configuring corresponding image quality adjustment parameters for the first frame based on the classification label of the first frame includes: determining whether the classification label of the first frame is a preset image quality adjustment scene; if so, configuring the image quality adjustment parameters corresponding to the classification label for the first frame. Otherwise, configure the default image quality adjustment parameters for the first frame.

4. The image processing method as described in claim 1, characterized in that, When the scene switching judgment step includes any multiple judgment steps among the first judgment, the second judgment, and the third judgment, any multiple judgments among the first judgment, the second judgment, and the third judgment are executed in parallel; Alternatively, the judgment steps of the first judgment, the second judgment, and the third judgment have a sequential order, with the first judgment preceding the second and third judgments, and the second judgment preceding the third judgment. If the judgment result of the previous judgment step is negative, the next judgment step is executed.

5. The image processing method as described in claim 4, characterized in that, The scene switching judgment step includes a first judgment, a second judgment, and a third judgment performed sequentially or in parallel.

6. The image processing method as described in claim 1, characterized in that, The step of configuring corresponding image quality parameters for the current frame based on the classification label of the current frame includes: determining whether the classification label of the current frame is a preset image quality adjustment scene; if so, configuring the image quality adjustment parameters corresponding to the classification label for the current frame. Otherwise, set the category label of the current frame to the default label, and configure the default image quality adjustment parameters for the current frame.

7. The image processing method as described in claim 1, characterized in that, The image processing method further includes: pre-judging the current frame to determine whether the first initial label of the current frame is the same as the classification label of the previous frame; When the result of the pre-judgment or at least one of the judgment results in the scene switching judgment step is yes, the classification label of the current frame is set to be the same as the classification label of the previous frame. When the result of the pre-judgment and all the judgment results in the scene switching judgment step are negative, the first initial label of the current frame is assigned to the current frame.

8. The image processing method as described in claim 7, characterized in that, Before performing the scene switching judgment step, the pre-judgment is performed; and if the result of the pre-judgment is negative, the scene switching judgment step is executed.

9. The image processing method as described in claim 1, characterized in that, The image processing method further includes: configuring multiple preset image quality adjustment scenes; and obtaining color statistics information for each frame. The steps for obtaining color statistics information for each frame include: calculating the proportion of pixels in the image corresponding to the hue range values ​​of the features of the multiple preset image quality adjustment scenes, and using this as the color statistics information.

10. The image processing method as described in claim 9, characterized in that, The image processing method further includes: configuring a hue range value corresponding to the features of the preset image quality adjustment scene before acquiring the color statistics information of each frame.

11. The image processing method as described in claim 9, characterized in that, Using color statistical histograms, the proportion of pixels in the image corresponding to the hue range values ​​of the features of the multiple preset image quality adjustment scenes is statistically analyzed.

12. The image processing method as described in claim 1, characterized in that, The tag set includes the top 3 or top 5 initial tags whose prediction results best match the current frame scene.

13. An image processing apparatus, characterized in that, include: A providing unit is used to provide a video frame sequence, including the first frame of the video frame sequence and the current frame after the first frame; The scene prediction unit is used to obtain the classification label of the first frame and to perform scene prediction processing on the current frame to obtain the label set corresponding to the current frame. The label set includes the top few initial labels whose prediction results best match the scene of the current frame, and the initial label with the highest accuracy is the first initial label. A scene switching judgment unit is used to perform scene switching judgment processing on the current frame. The scene switching judgment unit includes any one or more judgment subunits among a first judgment subunit, a second judgment subunit, and a third judgment subunit. The first judgment subunit is used to determine whether the classification label of the previous frame is included in the label set of the current frame and whether the confidence score with the corresponding initial label is greater than a preset threshold, and outputs a first judgment result; the confidence score between the classification label of the previous frame and the corresponding initial label refers to the confidence score between the classification label of the previous frame and the initial label when the classification label of the previous frame is the same as an initial label in the label set of the current frame; the second judgment subunit is used to determine whether the difference between the color statistics information of the previous frame and the color statistics information of the current frame is within a preset difference range, and outputs a second judgment result; the third judgment subunit is used to determine whether the proportion of the hue range corresponding to the feature of the classification label of the previous frame in the current frame image is greater than a preset proportion threshold, and outputs a third judgment result; The label setting unit is used to set the classification label of the current frame to be the same as the classification label of the previous frame when the judgment result of at least one judgment subunit in the scene switching judgment unit is yes; otherwise, the label setting unit is used to set the first initial label to the classification label of the current frame. The image quality adjustment unit is used to configure corresponding image quality adjustment parameters for the current frame based on the classification label of the current frame.

14. The image processing apparatus as claimed in claim 13, characterized in that, The image quality adjustment unit is further configured to configure corresponding image quality adjustment parameters for the first frame based on the classification label of the first frame.

15. A device, characterized in that, It includes at least one memory and at least one processor, the memory storing one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the image processing method as described in any one of claims 1-12.

16. A storage medium, characterized in that, The storage medium stores one or more computer instructions, which are used to implement the image processing method as described in any one of claims 1-12.

Images