Saturation adjustment method, electronic device, storage medium and program product

Abstract

The invention provides a saturation adjustment method, electronic equipment, a storage medium and a program product. The method includes: acquiring a high dynamic range (HDR) video; performing emotion analysis on each image frame in the HDR video to obtain an emotion analysis result of each image frame, the emotion analysis result being at least used for representing an emotion category expressed by the image frame; determining a saturation coefficient of each image frame based on the emotion analysis result of each image frame; and adjusting the saturation of each image frame based on the saturation coefficient of each image frame. Through the scheme provided by the invention, the display effect of the HDR video can be improved.

Description

Technical Field

[0001] This application relates to the field of video processing technology, and in particular to a saturation adjustment method, electronic device, storage medium, and program product. Background Technology

[0002] High Dynamic Range (HDR) technology is used for imaging and display. HDR technology expands the brightness range (e.g., 0-10000 nits) and color depth (e.g., 0-1023) of an image, resulting in richer details and more vibrant colors. HDR technology is widely used in film, gaming, and virtual reality, offering more possibilities for enhancing visual experiences.

[0003] Videos generated using HDR technology are called HDR videos. Color saturation is a key parameter for HDR video display. Color saturation determines the vibrancy of colors in HDR videos. The higher the color saturation, the more vibrant the colors in the HDR video; the lower the color saturation, the closer the colors in the HDR video are to gray. HDR technology provides the foundation for displaying HDR videos with high color saturation, making the colors in HDR videos richer and more vibrant.

[0004] Typically, in the display of HDR videos, all HDR videos use a uniform color saturation strategy, which makes it difficult to fully showcase the advantages of HDR videos and results in unsatisfactory display effects. Summary of the Invention

[0005] To address the related technical problems, embodiments of this application provide a saturation adjustment method, an electronic device, a storage medium, and a program product.

[0006] The technical solution of this application embodiment is implemented as follows: This application provides a saturation adjustment method, the method comprising: Get HDR video; Sentiment analysis is performed on each image frame in the HDR video to obtain a sentiment analysis result for each image frame, wherein the sentiment analysis result is used to at least represent the sentiment category expressed by the image frame; Based on the sentiment analysis results of each image frame, the saturation coefficient of each image frame is determined; The saturation of each image frame is adjusted based on its saturation coefficient.

[0007] In the above scheme, the sentiment analysis results include the probabilities corresponding to various sentiment categories; determining the saturation coefficient of each image frame based on the sentiment analysis results of each image frame includes: Based on the probabilities corresponding to the various emotion categories, an emotion coefficient for each image frame is determined, wherein the emotion coefficient characterizes the strength of the emotion expressed by the image frame. The saturation coefficient of the corresponding image frame is determined based on the emotion coefficient of each image frame.

[0008] In the above scheme, determining the emotion coefficient of each image frame based on the probabilities corresponding to the various emotion categories includes: Based on the weight scores corresponding to the various emotion categories, the probabilities corresponding to the various emotion categories are weighted to obtain the emotion coefficient of each image frame.

[0009] In the above scheme, adjusting the saturation of each image frame based on its saturation coefficient includes: The target saturation of each image frame is determined based on the first saturation, the second saturation, and the saturation coefficient of each image frame, wherein the first saturation is the initial saturation of the image frame, and the second saturation is the preset minimum saturation. Adjust the saturation of each image frame to the target saturation level for each image frame.

[0010] In the above scheme, determining the target saturation of each image frame based on the first saturation, the second saturation, and the saturation coefficient of each image frame includes: Determine the difference between the first saturation and the second saturation for each image frame; The target saturation of each image frame is determined based on the first saturation, the difference between the first saturation and the second saturation, and the saturation coefficient.

[0011] In the above scheme, the step of performing sentiment analysis on each image frame in the HDR video to obtain the sentiment analysis result for each image frame includes: Each image frame in the HDR video is divided into multiple image blocks; Extract semantic features from the multiple image patches; Based on the semantic features of the multiple image blocks, sentiment analysis is performed on each image frame to obtain the sentiment analysis results for each image frame.

[0012] In the above scheme, extracting the semantic features of the multiple image patches includes: Multi-head attention processing is performed on the multiple image patches to obtain their semantic features.

[0013] This application also provides an electronic device, including: a processor and a memory for storing a computer program that can run on the processor; wherein, when the processor runs the computer program, it executes the method provided in this application.

[0014] This application also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the method provided in this application.

[0015] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in this application.

[0016] The saturation adjustment method, electronic device, storage medium, and program product provided in this application embodiment can determine the emotional category expressed by each image frame in an HDR video by performing emotion analysis on each image frame. Then, based on the emotion analysis results of each image frame, the saturation coefficient of each image frame is determined, thereby establishing a correlation between image content and saturation. By adjusting the saturation of each image frame through the saturation coefficient of each image frame, the saturation of each image frame in the HDR video is adapted to the image frame content. This can improve the coordination between the color and content of the HDR video, realize the dynamic adjustment of the saturation of image frames in the HDR video, and thus fully demonstrate the advantages of HDR video, making the HDR video picture have a better display effect. Attached Figure Description

[0017] Figure 1 This is a schematic flowchart of a saturation adjustment method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of a sentiment analysis model provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a saturation adjustment program module provided in an embodiment of this application; Figure 4 This is a flowchart illustrating an application example of a saturation adjustment method provided in this application embodiment; Figure 5 This is a schematic diagram of the structure of a saturation adjustment device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] The present application will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] HDR technology has been widely adopted in film, television, and gaming. For example, numerous video streaming platforms support HDR video streaming, allowing users to watch HDR video content on electronic devices that support HDR display technology. Simultaneously, many high-end TVs and monitors also support HDR display technology to provide a more realistic and vibrant visual experience. Many user terminals also support HDR display functionality and can display HDR video. Furthermore, many games have begun to adopt HDR technology to provide more realistic visual effects and a more immersive gaming experience.

[0020] For example, consider a shooting scenario. An electronic device supporting HDR technology can activate HDR shooting at the user's request, capturing and displaying HDR video. Even if the scene being shot is backlit, the HDR video will still retain detail in the darker areas, thus enhancing the visual realism of the HDR video.

[0021] While HDR video offers more realistic visuals, it is prone to color oversaturation. Color saturation is a crucial parameter for HDR video display. It can be simply referred to as color saturation or color intensity. Color oversaturation occurs when the saturation is too high, causing the colors in the HDR video to exceed the range of colors an object would normally display under normal lighting, resulting in distortion. Furthermore, even without oversaturation, excessively vibrant colors can also distort the HDR video. In such cases, adjusting the saturation can make the HDR video appear more natural.

[0022] Electronic devices offer a manual saturation adjustment option. If the HDR video display is unsatisfactory, the device can adjust the saturation at the user's discretion to meet their color preferences. However, this manual saturation adjustment is complex and can disrupt the viewing experience and immersion.

[0023] Furthermore, current saturation adjustment methods all employ a uniform color saturation strategy, applying the same saturation parameter indiscriminately to all HDR videos. This approach ignores the differences in color expression across various scenes. For example, when an HDR video displays vibrant flowers, a certain degree of color saturation, or even slight oversaturation, can actually enhance the flowers' lively visual qualities and better showcase their brilliant colors. Similarly, when an HDR video presents food, bright colors and high saturation are more likely to stimulate the viewer's appetite. However, when an HDR video presents a serene, rainy scene, high saturation can disrupt the narrative. Clearly, this saturation adjustment method is disconnected from the HDR video content, making it difficult to achieve harmony between the image's color and content, thus hindering the full expression of HDR video's advantages and resulting in poor display quality.

[0024] Based on this, this application provides a saturation adjustment scheme. By performing emotion analysis on each image frame in an HDR video, an electronic device can determine the emotional information expressed by each image frame. Based on the emotion analysis results, the saturation coefficient of each image frame is determined, establishing a correlation between image content and saturation. By adjusting the saturation of the corresponding image frame using the saturation coefficient of each image frame, the saturation of the HDR video is dynamically adjusted. This ensures that the color of each image frame in the HDR video displayed by the electronic device matches the emotion expressed by the image, improving the harmony between the color and content of the HDR video. This fully showcases the advantages of HDR video and gives it a better display effect.

[0025] Furthermore, the saturation adjustment scheme provided in this application embodiment can achieve adaptive adjustment of HDR video saturation without manual intervention, saving labor costs, improving the continuity and immersion of the user's viewing experience, meeting the user's needs for HDR video color, and enhancing the user experience.

[0026] First, this application provides a saturation adjustment method applied to an electronic device. The electronic device has a display screen, through which it displays HDR video and / or saturation-adjusted HDR video.

[0027] The electronic devices provided in this application can be mobile phones, tablets, desktop computers, laptops, handheld computers, notebook computers, super mobile personal computers, netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, display devices, etc. This application does not impose any special limitations on the specific form of the electronic devices.

[0028] For example, consider a mobile phone as an electronic device. The phone has a video playback application installed. The phone plays an HDR video through this application. During HDR video playback, the phone adjusts the saturation of each image frame in the HDR video using the video playback application, matching the colors of the displayed image frames to the emotions they convey, thus making the color representation of the image frames more vivid.

[0029] The saturation adjustment method provided in the embodiments of this application will be described below with reference to the accompanying drawings. Figure 1 As shown, the method includes the following steps: Step 101: Electronic device acquires HDR video; Step 102: For each image frame in the HDR video, the electronic device performs sentiment analysis on each image frame to obtain the sentiment analysis results for each image frame; Step 103: The electronic device determines the saturation coefficient of each image frame based on the sentiment analysis results of each image frame; Step 104: The electronic device adjusts the saturation of each image frame based on the saturation coefficient of each image frame.

[0030] In practical applications, in step 101, the electronic device can receive HDR video from other devices. For example, the electronic device can receive HDR video from the server of a video playback application. Alternatively, the electronic device can read stored HDR video from its internal memory. Or, the electronic device can capture and generate HDR video in real time via a camera.

[0031] HDR video consists of multiple image frames. These frames are all HDR images, which have a wider range of brightness and color depth compared to standard dynamic range (SDR) images. The multiple image frames in an HDR video are arranged in chronological order.

[0032] In step 102, the electronic device performs frame-by-frame analysis on multiple image frames in the HDR video according to the arrangement order of multiple image frames, determines the emotion expressed by each image frame, and obtains the emotion analysis result of each image frame.

[0033] In this embodiment, for each image frame in an HDR video, the electronic device can input each image frame into a pre-trained emotion classification model, and use the emotion classification model to perform emotion analysis on each image frame, obtaining the emotion analysis result of each image frame output by the emotion classification model. The pre-trained emotion classification model is a machine learning model. The electronic device uses the pre-trained emotion classification model to extract the semantic features of each image frame, and then obtains the emotion analysis result output by the pre-trained emotion classification model based on the semantic features of each image frame.

[0034] In this embodiment, the sentiment analysis result is used at least to represent the sentiment category (or sentiment band) expressed by the image frame. The sentiment category expressed by the image frame can be understood as the category or level of emotional fluctuation caused by the content presentation of the image frame. For example, when an image frame includes bright flowers, the image frame tends to cause strong emotional fluctuations. Therefore, after performing sentiment analysis on the image frame, the electronic device determines that the sentiment category expressed by the image frame is "strong fluctuation." The sentiment analysis result can indicate the sentiment category of "strong fluctuation." As another example, when an image frame includes a rainy scene, the image frame tends to cause slight emotional fluctuations. Therefore, after performing sentiment analysis on the image frame, the electronic device determines that the sentiment category expressed by the image frame is "slight fluctuation." The sentiment analysis result can indicate the sentiment category of "slight fluctuation."

[0035] Based on the intensity of the emotion expressed by the image frame, emotions can be divided into multiple emotion categories (i.e., multiple emotion bands) from weak to strong. Each emotion category corresponds to a category or level of emotional fluctuation. For example, let's divide emotions into 5 emotion categories based on the intensity of the emotion expressed by the image frame. These 5 emotion categories are represented as Emotion Category 1, Emotion Category 2, Emotion Category 3, Emotion Category 4, and Emotion Category 5. Here, Emotion Category 1 is calm; Emotion Category 2 is slight fluctuation; Emotion Category 3 is small fluctuation; Emotion Category 4 is large fluctuation; and Emotion Category 5 is strong fluctuation.

[0036] Mood category 1 (i.e., calm) indicates that the atmosphere of the image frame tends to be stable and restrained. Image frames of this mood category are usually characterized by soft tones and low color saturation. For example, image frames of static landscapes or overcast and rainy scenes can be classified as mood category 1.

[0037] Emotion Category 2 (i.e., slight fluctuation) indicates that the image frame contains initial emotional fluctuations, but the overall image remains relatively stable. Image frames in this emotion category typically exhibit some color accents and lower color saturation. For example, image frames depicting a scene with a gentle breeze or showing subtle facial expressions can be classified as Emotion Category 2.

[0038] Emotion category 3 (i.e., less fluctuation) indicates that the image frame expresses a relatively small emotional fluctuation, and the image frame has a certain vitality. Image frames of this emotion category are usually characterized by a certain degree of color contrast and color saturation. For example, image frames of city street scenes and daily life scenes can be classified as emotion category 3.

[0039] Emotion category 4 (i.e., greater fluctuation) indicates that the image frame expresses a greater emotional fluctuation. Image frames in this emotion category are typically characterized by strong color contrast and high color saturation. For example, image frames of celebrations or children playing can be classified as emotion category 4.

[0040] Emotion category 5 (i.e., strong fluctuations) indicates that the image frame expresses strong emotional fluctuations. Image frames in this emotion category are usually characterized by strong color contrast and color saturation. For example, brightly colored flowers and food can be classified as emotion category 5.

[0041] In some optional implementations, the sentiment analysis results include probabilities corresponding to multiple sentiment categories. This allows the electronic device to determine the sentiment category expressed by a given image frame based on the probabilities of each sentiment category within the sentiment analysis results for that frame. For example, consider a sentiment analysis result that includes probabilities corresponding to the aforementioned five sentiment categories. The sentiment analysis result could be a vector Q=[q1,q2,q3,q4,q5]. T Where q1 represents the probability of the emotion category being calm; q2 represents the probability of the emotion category being slightly fluctuating; q3 represents the probability of the emotion category being relatively fluctuating; q4 represents the probability of the emotion category being relatively fluctuating; and q5 represents the probability of the emotion category being strongly fluctuating.

[0042] To better extract semantic features from each image frame, in some optional embodiments, the emotion classification model employs a Transformer-based classification model. The emotion classification model effectively captures the contextual dependencies within each image frame through the attention mechanism provided by the Transformer architecture, extracting more accurate semantic features for each image frame.

[0043] For each image frame in an HDR video, the electronic device divides each image frame into multiple image patches, such as dividing each image frame into multiple image patches of equal size. The electronic device inputs these multiple image patches from each image frame into a sentiment classification model. The attention mechanism of the sentiment classification model extracts features from these multiple image patches, obtaining the semantic features corresponding to each patch. The electronic device then uses the sentiment classification model to obtain the sentiment analysis result of the image frame based on the semantic features corresponding to each of the multiple image patches. For example, the electronic device uses the sentiment classification model to perform fully connected processing and normalization on the semantic features corresponding to each of the multiple image patches to obtain the sentiment analysis result of the image frame.

[0044] In this embodiment, by splitting each image frame, the electronic device can establish associations between multiple image blocks. Thus, through the attention mechanism provided by the emotion classification model, the contextual dependencies between the various image blocks of the image frame can be effectively captured. The semantic features corresponding to each image block are fused with information from other image blocks, achieving a deep understanding of the global semantics of the image frame and improving the accuracy of the sentiment analysis results.

[0045] For example, the structure of an emotion classification model is as follows: Figure 2 As shown, the model includes a multi-head attention module, a fully connected neural network, and a softmax classifier. The multi-head attention module encodes the input image patch sequence (i.e., multiple image patches), capturing the contextual information of the image patch sequence and outputting its semantic features. The fully connected neural network performs nonlinear transformations and feature integration on the semantic features of the input image patch sequence to obtain fully connected features. The softmax classifier maps the fully connected features to normalized sentiment analysis results.

[0046] For any given image frame, the electronic device divides the image frame into n image blocks. These n image blocks can be represented as P1 to P2. n The electronic device will connect P1 to P... n The input is fed into the emotion classification model. The multi-head attention module of the emotion classification model employs a multi-layered Transformer encoder. The multi-head attention module processes P1 to P through multiple stacked Transformer encoders. n Perform multi-head attention processing to achieve attention from P1 to P2. n Encode to obtain P1 to P n The corresponding semantic feature vectors (i.e., semantic features) F1 to F1 are respectively. n The emotion classification model uses a fully connected neural network (or fully connected layer) and a softmax classifier to classify semantic feature vectors F1 to F2. nPerform sentiment analysis on the image frames and output the sentiment analysis result Q=[q1,q2,q3,q4,q5]. T .

[0047] In the example, the multi-head attention module extracts semantic features from multiple image patches through a multi-head attention mechanism (i.e., multi-head attention processing) using multiple cascaded Transformer encoders. This mechanism uses multiple parallel Transformer encoders as attention heads, each independently processing multiple image patches and obtaining attention results. The attention results from multiple attention heads are then fused to obtain the semantic features of the multiple image patches. In this way, multiple attention heads simultaneously capture the dependencies between multiple image patches from different perspectives, resulting in richer semantic features and improved accuracy of sentiment analysis results.

[0048] The emotion classification model in this embodiment is trained based on image samples. For example, an electronic device acquires a large number of HDR images as image samples, and uses these image samples and their corresponding emotion labels as a training dataset. The electronic device inputs the image samples from the training dataset into an untrained emotion classification model to obtain the emotion prediction results of the model. Then, the electronic device uses a loss function (such as cross-entropy loss function) to calculate the difference between the emotion prediction results and the emotion labels of the image samples, obtains the model loss, and updates the model parameters of the emotion classification model using a backpropagation algorithm based on the calculated model loss. This iterative optimization continues until the emotion classification model converges, resulting in a fully trained emotion classification model.

[0049] It is understandable that the electronic device used to train the emotion classification model and the electronic device used to perform emotion analysis on image frames using the pre-trained emotion classification model can be the same or different electronic devices. To achieve reasonable load distribution and response speed, in one example, the electronic device using the pre-trained emotion classification model acts as the user terminal. The video playback application runs the pre-trained emotion classification model to perform emotion analysis on each image frame in the HDR video. The task of training the emotion classification model can be implemented by the server-side of the video playback application. After the server-side of the video playback application completes the training of the emotion classification model, it can send the pre-trained emotion classification model, carried within the video playback application, to the user terminal. When the user terminal plays the HDR video through the video playback application, it runs the pre-trained emotion classification model to perform emotion analysis on the image frames in the HDR video.

[0050] To improve the accuracy and efficiency of sentiment analysis, in some optional embodiments, before step 102, i.e., before performing sentiment analysis on each image frame and obtaining the sentiment analysis result for each image frame, the electronic device can also preprocess the image frames in the HDR video. For example, the electronic device can perform preprocessing operations such as filtering and resolution adjustment on the image frames in the HDR video to make the preprocessed image frames suitable for the sentiment analysis model to extract semantic features.

[0051] For example, after an electronic device acquires an image frame from an HDR video, it filters the image frame to enhance it and remove noise. This improves the signal-to-noise ratio of the image frame and enhances the accuracy of semantic feature extraction.

[0052] Correspondingly, electronic devices can also perform resolution adjustment operations on image frames, scaling them to a resolution size that matches the emotion classification model. In this way, by adjusting the resolution of image frames, the resolution of the image frames can meet the requirements of the emotion classification model. Furthermore, it facilitates the even division of image frames into multiple image blocks.

[0053] In step 103 above, after obtaining the sentiment analysis result of any image frame in the HDR video, the electronic device determines the saturation coefficient of the image frame based on the sentiment analysis result of the image frame.

[0054] For example, electronic devices have preset weight scores for various emotion categories. Different emotion categories have different weight scores. The weight score for an emotion category is related to the degree of emotional fluctuation associated with that category. The higher the degree of emotional fluctuation, the larger the weight score for the emotion category. The lower the degree of emotional fluctuation, the smaller the weight score for the emotion category. Taking the emotion categories "strong fluctuation" and "slight fluctuation" as examples, the degree of emotional fluctuation corresponding to "strong fluctuation" is higher than that corresponding to "slight fluctuation," therefore, the weight score for "strong fluctuation" is greater than the weight score for "slight fluctuation."

[0055] In some alternative implementations, when the sentiment analysis result indicates an emotion category, the electronic device can determine the saturation coefficient of the current image frame based on the weight score corresponding to the emotion category indicated by the sentiment analysis result. For example, the electronic device determines the saturation coefficient of the current image frame as the weight score corresponding to the emotion category indicated by the sentiment analysis result.

[0056] In this embodiment, the saturation coefficient is used to adjust the saturation of an image frame. For example, the saturation coefficient of an image frame is directly proportional to its saturation. The larger the saturation coefficient of an image frame, the greater the saturation of the adjusted image frame. Conversely, the smaller the saturation coefficient of an image frame, the less saturated the adjusted image frame.

[0057] To more accurately determine the saturation coefficient of an image frame, in some optional embodiments, when the emotion analysis result includes the probabilities of multiple emotion categories, the electronic device can determine the emotion coefficient of each image frame based on the probabilities of the multiple emotion categories included in the emotion analysis result. This emotion coefficient is used to characterize the strength of the emotion expressed by the image frame (i.e., the degree of emotional fluctuation). Then, the electronic device determines the saturation coefficient of the corresponding image frame based on the emotion coefficient of each image frame.

[0058] In this embodiment, the intensity of emotion can be quantified through the emotion coefficient, thereby transforming the probability distribution corresponding to the emotion analysis result into a simple quantitative coefficient (i.e., the emotion coefficient). This reduces the complexity and computational cost of saturation adjustment and improves the efficiency of saturation adjustment.

[0059] For example, taking any image frame as an example, the electronic device can determine the emotion category with the highest probability among multiple emotion categories included in the emotion analysis results of that image frame. Then, the electronic device determines the saturation coefficient of the image frame based on the weight score corresponding to the emotion category with the highest probability. For example, the electronic device determines the emotion coefficient of the image frame as the weight score of the emotion category with the highest probability in the emotion analysis results.

[0060] In order to more accurately represent the intensity of emotions through the emotion coefficient, in some optional implementations, the electronic device can weight the probabilities of multiple emotion categories based on the weight scores corresponding to multiple emotion categories in the emotion analysis results of the image frame, and obtain the emotion coefficient of the image frame.

[0061] For example, consider the probability that the sentiment analysis results of an image frame include five sentiment categories. The electronic device calculates the sentiment coefficient of the image frame using formula (1): Formula (1) Where x represents the emotion coefficient of the image frame; q i v represents the probability corresponding to the i-th emotion category in the sentiment analysis results; i This represents the weight score corresponding to the i-th emotion category.

[0062] In this embodiment, the probabilities of various emotion categories are weighted by weight scores corresponding to multiple emotion categories, thereby fully considering the contribution of various emotion categories to the emotion coefficient, making the assessment of the intensity of emotion more accurate. The obtained emotion coefficient can more comprehensively and accurately reflect the intensity of the emotion expressed by the image frame, thus making the saturation adjustment more accurate.

[0063] It is understood that the weight score corresponding to each emotion category can be set according to the actual application scenario or needs. This application embodiment does not limit the specific value of the weight score. The weight scores corresponding to multiple emotion categories increase from small to large according to the degree of emotional fluctuation represented by the emotion category. For example, taking the above 5 emotion categories as an example, the weight scores of emotion category 1 to emotion category 5 are 10, 20, 30, 40, and 50 respectively.

[0064] After determining the sentiment coefficient of an image frame, the electronic device can use the determined sentiment coefficient as the saturation coefficient of that image frame. Alternatively, the electronic device can map the sentiment coefficient to a range of 0 to 1 to obtain the saturation coefficient of the image frame. For example, the electronic device can calculate the saturation coefficient of the image frame using formula (2): Formula (2) Where y represents the saturation coefficient; x represents the emotion coefficient; and a and b represent preset parameters.

[0065] It is understandable that parameters a and b can be set based on experience. Alternatively, parameters a and b can be determined through training. For example, the emotion coefficients of multiple image samples can be used as input data, and the saturation coefficients of multiple image samples under ideal conditions can be used as supervision data. Substituting the emotion coefficients of the image samples into formula (2), the saturation coefficients of multiple image samples are obtained. Through optimization algorithms (such as gradient descent), parameters a and b are fitted, the difference between the saturation coefficients of the image samples and the supervision data is calculated, and the values ​​of parameters a and b are adjusted based on the difference. This process is iteratively executed until the error between the obtained saturation coefficients and the supervision data is less than a predetermined threshold. The parameters a and b determined at this time are used as the parameters a and b used to calculate the saturation coefficients of the image frames. In this way, the accuracy of parameters a and b can be improved, thereby improving the accuracy of the saturation coefficients of the image frames.

[0066] In step 104, the electronic device uses the saturation coefficient of each image frame to adjust the saturation of the corresponding image frame. Thus, when displaying each image frame in the HDR video, the electronic device can use a saturation that matches the content of each image frame to improve the display effect of the HDR video.

[0067] For example, for any image frame in an HDR video, the electronic device determines a target saturation based on the initial saturation (i.e., first saturation) of the image frame and a saturation coefficient. For instance, the electronic device calculates the product of the initial saturation and the saturation coefficient of the image frame to obtain the target saturation. Then, the electronic device sets the saturation of the image frame to the target saturation.

[0068] To finely adjust the saturation of image frames, in some optional implementations, for each image frame in an HDR video, the electronic device can determine a target saturation based on the initial saturation of the image frame, a preset minimum saturation, and a saturation coefficient. The electronic device then adjusts the saturation of the image frame to the target saturation.

[0069] In this embodiment, the electronic device has a preset minimum saturation level (i.e., a second saturation level) for HDR video display. This minimum saturation level serves as the lower limit of the saturation adjustment range, ensuring that even if the image frame depicts a calm scene, it will not completely lose color after saturation adjustment. Correspondingly, the initial saturation level of the image frame is the saturation level before saturation adjustment using a saturation coefficient. This initial saturation level also serves as the upper limit of saturation adjustment, ensuring that even if the image frame depicts a scene with strong emotions, its saturation will not exceed the initial saturation level after adjustment. This enhances the visual expressiveness of the image frame through saturation adjustment while reducing the possibility of distortion due to over-adjustment of saturation.

[0070] For example, for each image frame in an HDR video, the electronic device can determine the target saturation based on the initial saturation of the image frame, the difference between the initial saturation and the minimum saturation, and the saturation coefficient. That is, the electronic device can calculate the target saturation of the image frame using formula (3): Formula (3) Where s represents the target saturation; s0 represents the preset minimum saturation; s1 represents the initial saturation of the image frame; and y represents the saturation coefficient, which takes the value between 0 and 1, i.e., y is greater than or equal to 0 and less than or equal to 1.

[0071] By determining the initial saturation and the preset minimum saturation of the image frame, the range of saturation adjustment is determined. That is, after saturation adjustment, the saturation of the image frame is within the range formed by the initial saturation and the minimum saturation, thereby reducing the possibility of distortion in the HDR video while preserving the visual effect of the HDR video.

[0072] Furthermore, the saturation adjustment of image frames is adapted to the content of those frames. When the emotion expressed in an image frame is stronger, the saturation coefficient y approaches 1, and the target saturation s approaches the initial saturation s1 of the image frame. Therefore, after setting the image frame to the target saturation, the image frame can fully display its original vibrant colors. Conversely, when the emotion expressed in an image frame is calmer, the saturation coefficient y approaches 0, and the target saturation s approaches the preset minimum s0. Therefore, after setting the image frame to the target saturation, the image frame becomes softer. This enhances the display effect of HDR video.

[0073] The following application example illustrates the saturation adjustment method provided in this application. In this example, the electronic device is a user terminal with video playback application software installed. The video playback application software has an embedded saturation adjustment program module. When the user terminal starts the video playback application software and plays an HDR video through it, the user terminal calls the saturation adjustment program module to adjust the saturation of each image frame in the displayed HDR video. Figure 3 As shown, the saturation adjustment program module embedded in the video playback application software includes an HDR video image interface module, a preprocessing module, an image segmentation module, a multi-head attention module, a sentiment analysis module, a sentiment coefficient calculation module, a saturation coefficient calculation module, and a saturation adjustment module.

[0074] The system comprises the following modules: an HDR video image interface module for receiving HDR video and extracting image frames; a preprocessing module for preprocessing the image frames; an image segmentation module for segmenting the image frames into multiple image blocks; a multi-head attention mechanism module for receiving multiple image blocks and extracting their semantic feature vectors; a sentiment analysis module, including a fully connected neural network and a softmax classifier, for performing sentiment analysis on the image frames based on the semantic feature vectors of the multiple image blocks; a sentiment coefficient calculation module for calculating the sentiment coefficient of the image frames based on the sentiment analysis results; a saturation coefficient calculation module for calculating the saturation coefficient based on the sentiment coefficient; and a color saturation adjustment module for adjusting the saturation of the image frames based on the saturation coefficient.

[0075] In this application example, an electronic device receives an HDR video stream (i.e., HDR video) transmitted from a server (such as a video media platform) of a video playback application and processes each image frame in the HDR video stream in real time. The processing flow for each image frame is the same. The following example uses the processing flow of a single image frame to illustrate the saturation adjustment method provided in this application example. Figure 4 As shown, the saturation adjustment method provided in this application example includes the following steps: Step 401: The user terminal extracts image frames from the HDR video stream; In response to a user's click on the video playback application icon, the user terminal launches the video playback application and displays its interface. Upon receiving the user's instruction to play a video, the user terminal initiates a video playback request to the video playback application's server. In response to this request, the video playback application's server transmits an HDR video stream to the user terminal.

[0076] The user terminal receives the HDR video stream transmitted by the server through the HDR video image interface module and extracts the current image frame from the HDR video stream.

[0077] Step 402: The user terminal preprocesses the image frames; The user terminal uses a preprocessing module to preprocess the image frame, such as filtering and adjusting the resolution, so that the preprocessed image frame is suitable for subsequent image frame segmentation and feature extraction.

[0078] Step 403: The user terminal divides the image frame into multiple image blocks; The user terminal uses an image segmentation module to segment the image frame into n image blocks P1 to P2. n .

[0079] Step 404: The user terminal determines the semantic features of multiple image patches; The user terminal extracts n image blocks P1 to P2 using a multi-head attention module. n The corresponding semantic feature vectors F1 to F n .

[0080] Multi-head attention modules can employ multi-layered Transformer encoders. The user terminal processes n image blocks P1 to P2 through multiple stacked Transformer encoders. n Encode the semantic feature vectors F1 to F2. n .

[0081] Step 405: The user terminal determines the sentiment analysis result of the image frame based on the semantic features of multiple image patches; The user terminal uses the sentiment analysis module to analyze semantic feature vectors F1 to F... n Sentiment analysis is performed on image frames to obtain the sentiment analysis results.

[0082] The sentiment analysis module may include a fully connected neural network (i.e., a fully connected layer) and a softmax classifier. The user terminal uses the fully connected neural network and the softmax classifier to analyze the semantic feature vectors F1 to F2. nSentiment analysis is performed on image frames to obtain sentiment analysis results.

[0083] The sentiment analysis results include the probabilities corresponding to five sentiment categories. The sentiment analysis result Q = [q1, q2, q3, q4, q5]. T q1 represents the probability that the emotion category is calm; q2 represents the probability that the emotion category is slightly fluctuating; q3 represents the probability that the emotion category is relatively small fluctuating; q4 represents the probability that the emotion category is relatively large fluctuating; and q5 represents the probability that the emotion category is strongly fluctuating.

[0084] Step 406: The user terminal determines the sentiment coefficient of the image frame based on the sentiment analysis results of the image frame; The user terminal uses the emotion coefficient calculation module to calculate the emotion coefficient of the image frame based on the emotion analysis results. For example, the user terminal uses the emotion coefficient calculation module to substitute the emotion analysis results into the above formula (1) to calculate the emotion coefficient of the image frame.

[0085] Step 407: The user terminal determines the saturation coefficient of the image frame based on the emotion coefficient of the image frame; The user terminal uses the saturation coefficient calculation module to calculate the saturation coefficient of the image frame based on the sentiment coefficient of the image frame. For example, the user terminal uses the saturation coefficient calculation module to substitute the sentiment coefficient of the image frame into the above formula (2) to calculate the saturation coefficient of the image frame.

[0086] Step 408: The user terminal adjusts the saturation of the image frame according to the saturation coefficient of the image frame.

[0087] The user terminal uses a saturation adjustment module to calculate the target saturation based on the saturation coefficient of the image frame, and then adjusts the saturation of the image frame to the target saturation.

[0088] The user terminal uses the saturation adjustment module to substitute the saturation coefficient of the image frame into the above formula (3) to calculate the target saturation. This target saturation can then be used as the saturation of the current image frame. After adjusting the saturation of the image frame, the user terminal displays the image frame with the adjusted saturation, that is, the user equipment displays the image frame with the target saturation.

[0089] This application example uses an image frame from an HDR video to illustrate the process of adjusting the saturation of an image frame. By extracting the semantic features of the image frame, analyzing the emotion it expresses, and adjusting the saturation of the image frame accordingly, the displayed color of the image frame matches the emotion it conveys. Each image frame in an HDR video can be adjusted using the saturation adjustment process provided in this application example, thus adapting the saturation of each image frame in the HDR video to the image frame's visual quality. This achieves dynamic adjustment of HDR video saturation, improving the display effect of HDR video and enhancing the user's viewing experience.

[0090] To implement the saturation adjustment method provided in this application embodiment, this application embodiment also provides a saturation adjustment device, which is installed on an electronic device, such as... Figure 5 As shown, the device includes: Module 51 is used to acquire HDR video; Analysis module 52 performs sentiment analysis on each image frame in the HDR video to obtain a sentiment analysis result for each image frame, wherein the sentiment analysis result is used to at least represent the sentiment category expressed by the image frame; Module 53 determines the saturation coefficient of each image frame based on the sentiment analysis results of each image frame; The adjustment module 54 adjusts the saturation of each image frame based on the saturation coefficient of each image frame.

[0091] In some optional embodiments, the emotion analysis results include probabilities corresponding to various emotion categories; the determining module 53 is specifically used for: Based on the probabilities corresponding to the various emotion categories, an emotion coefficient for each image frame is determined, wherein the emotion coefficient characterizes the strength of the emotion expressed by the image frame. The saturation coefficient of the corresponding image frame is determined based on the emotion coefficient of each image frame.

[0092] In some optional embodiments, the determining module 53 is specifically used for: Based on the weight scores corresponding to the various emotion categories, the probabilities corresponding to the various emotion categories are weighted to obtain the emotion coefficient of each image frame.

[0093] In some optional embodiments, the adjustment module 54 is specifically used for: The target saturation of each image frame is determined based on the first saturation, the second saturation, and the saturation coefficient of each image frame, wherein the first saturation is the initial saturation of the image frame, and the second saturation is the preset minimum saturation. Adjust the saturation of each image frame to the target saturation level for each image frame.

[0094] In some optional embodiments, the adjustment module 54 is specifically used for: Determine the difference between the first saturation and the second saturation for each image frame; The target saturation of each image frame is determined based on the first saturation, the difference between the first saturation and the second saturation, and the saturation coefficient.

[0095] In some optional embodiments, the analysis module 52 is specifically used for: Each image frame in the HDR video is divided into multiple image blocks; Extract semantic features from the multiple image patches; Based on the semantic features of the multiple image blocks, sentiment analysis is performed on each image frame to obtain the sentiment analysis results for each image frame.

[0096] In some optional embodiments, the analysis module 52 is specifically used for: Multi-head attention processing is performed on the multiple image patches to obtain their semantic features.

[0097] In practical applications, the acquisition module 51 can be implemented by the processor in the saturation adjustment device in conjunction with the communication interface, and the analysis module 52, the determination module 53 and the adjustment module 54 can be implemented by the processor in the saturation adjustment device.

[0098] It should be noted that the saturation adjustment device provided in the above embodiments is only illustrated by the division of the above-described program units. In practical applications, the above processing can be assigned to different program units as needed, that is, the internal structure of the device can be divided into different program units to complete all or part of the processing described above. In addition, the saturation adjustment device and the saturation adjustment method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0099] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device, such as... Figure 6 As shown, the electronic device includes: The communication interface 601 enables information exchange with other electronic devices (such as servers); The processor 602 is connected to the communication interface 601 to enable information interaction with other electronic devices (such as servers) and to execute the methods provided by one or more of the above technical solutions when running computer programs; The computer program is stored in memory 603.

[0100] Specifically, the processor 602 is used for: HDR video is acquired through the communication interface 601; Sentiment analysis is performed on each image frame in the HDR video to obtain a sentiment analysis result for each image frame, wherein the sentiment analysis result is used to at least represent the sentiment category expressed by the image frame; Based on the sentiment analysis results of each image frame, the saturation coefficient of each image frame is determined; The saturation of each image frame is adjusted based on its saturation coefficient.

[0101] In some alternative embodiments, the processor 602 is specifically used for: Based on the probabilities corresponding to the various emotion categories, an emotion coefficient for each image frame is determined, wherein the emotion coefficient characterizes the strength of the emotion expressed by the image frame. The saturation coefficient of the corresponding image frame is determined based on the emotion coefficient of each image frame.

[0102] In some alternative embodiments, the processor 602 is specifically used for: Based on the weight scores corresponding to the various emotion categories, the probabilities corresponding to the various emotion categories are weighted to obtain the emotion coefficient of each image frame.

[0103] In some alternative embodiments, the processor 602 is specifically used for: The target saturation of each image frame is determined based on the first saturation, the second saturation, and the saturation coefficient of each image frame, wherein the first saturation is the initial saturation of the image frame, and the second saturation is the preset minimum saturation. Adjust the saturation of each image frame to the target saturation level for each image frame.

[0104] In some alternative embodiments, the processor 602 is specifically used for: Determine the difference between the first saturation and the second saturation for each image frame; The target saturation of each image frame is determined based on the first saturation, the difference between the first saturation and the second saturation, and the saturation coefficient.

[0105] In some alternative embodiments, the processor 602 is specifically used for: Each image frame in the HDR video is divided into multiple image blocks; Extract semantic features from the multiple image patches; Based on the semantic features of the multiple image blocks, sentiment analysis is performed on each image frame to obtain the sentiment analysis results for each image frame.

[0106] In some alternative embodiments, the processor 602 is specifically used for: Multi-head attention processing is performed on the multiple image patches to obtain their semantic features.

[0107] It should be noted that the specific processing procedures of the processor 602 and the communication interface 601 can be understood with reference to the above-described saturation adjustment method.

[0108] Of course, in practical applications, the various components in an electronic device are coupled together through a bus system 604. It can be understood that the bus system 604 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 604 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 6 The general designated all buses as Bus System 604.

[0109] The memory 603 in this embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0110] The methods disclosed in the embodiments of this application can be applied to the processor 602, or implemented by the processor 602. The processor 602 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 602 or by instructions in the form of software. The processor 602 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 602 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the memory 603. The processor 602 reads the information in the memory 603 and combines its hardware to complete the steps of the aforementioned method.

[0111] In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0112] It is understood that the memory 603 in this embodiment can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0113] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 603 storing a computer program, which can be executed by a processor 602 of an electronic device to complete the steps described in the aforementioned saturation adjustment method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0114] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor 602 of an electronic device to perform the steps described in the aforementioned saturation adjustment method.

[0115] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0116] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0117] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A saturation adjustment method characterized by, The method comprises: acquiring a high dynamic range (HDR) video; performing emotion analysis on each image frame in the HDR video to obtain an emotion analysis result of each image frame, wherein the emotion analysis result is used to at least represent an emotion category expressed by the image frame; determining a saturation coefficient of each image frame based on the emotion analysis result of each image frame; adjusting the saturation of each image frame based on the saturation coefficient of each image frame.

2. The method of claim 1, wherein, The emotion analysis result comprises probabilities corresponding to a plurality of emotion categories respectively; The determination of the saturation coefficient of each image frame based on the emotion analysis result of each image frame comprises: determining an emotion coefficient of each image frame according to the probabilities corresponding to the plurality of emotion categories respectively, wherein the emotion coefficient represents the intensity of the emotion expressed by the image frame; and determining the saturation coefficient of the corresponding image frame according to the emotion coefficient of each image frame.

3. The method of claim 2, wherein, The determination of the emotion coefficient of each image frame according to the probabilities corresponding to the plurality of emotion categories respectively comprises: weighting the probabilities corresponding to the plurality of emotion categories respectively based on weight scores corresponding to the plurality of emotion categories respectively to obtain the emotion coefficient of each image frame.

4. The method of claim 1, wherein, The adjustment of the saturation of each image frame based on the saturation coefficient of each image frame comprises: determining a target saturation of each image frame according to a first saturation, a second saturation and the saturation coefficient of each image frame, wherein the first saturation is an initial saturation of the image frame, and the second saturation is a preset minimum saturation; and adjusting the saturation of each image frame to the target saturation of each image frame.

5. The method of claim 4, wherein, The determination of the target saturation of each image frame according to the first saturation, the second saturation and the saturation coefficient of each image frame comprises: determining a difference between the first saturation and the second saturation of each image frame; and determining the target saturation of each image frame according to the first saturation, the difference between the first saturation and the second saturation and the saturation coefficient.

6. The method according to any one of claims 1 to 5, characterized in that, The emotion analysis on each image frame in the HDR video to obtain the emotion analysis result of each image frame comprises: segmenting each image frame in the HDR video into a plurality of image blocks; extracting semantic features of the plurality of image blocks; performing emotion analysis on each image frame based on the semantic features of the plurality of image blocks to obtain the emotion analysis result of each image frame.

7. The method of claim 6, wherein, The extraction of the semantic features of the plurality of image blocks comprises: performing multi-head attention processing on the plurality of image blocks to obtain the semantic features of the plurality of image blocks.

8. An electronic device, comprising: comprise: a processor and a memory for storing a computer program capable of running on the processor; wherein the processor is used to run the computer program, and perform the steps of the method according to any one of claims 1 to 7.

9. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 7. The computer program is executed by the processor to implement the steps of the method according to any one of claims 1 to 7.

Images