Video-based lighting dynamics optimization methods, storage media, and electronic devices

Abstract

This application provides a video-based dynamic lighting optimization method, storage medium, and electronic device. The method includes: acquiring a video frame; dividing the video frame into multiple Regions of Interest (ROIs); acquiring the representative color of each ROI; identifying subtitle regions and skin tone regions in the video frame; adjusting the color weights of the subtitle regions and skin tone regions respectively; acquiring a first video frame; converting the representative colors of each ROI region in the first video frame to the HSV color space; performing smoothing and limiting processing on the HSV color space; acquiring a second video frame; acquiring the current video scene mode; adjusting the second video frame according to pre-configured dimming parameters for the current video scene mode; and generating and outputting a third video frame. This application can balance robustness, comfort, and scene adaptability, effectively solving problems such as color cast caused by bright subtitles / skin tone interference, light flickering caused by rapid scene switching, and scene adaptation delay.

Description

Technical Field

[0001] This application belongs to the field of intelligent lighting control technology, and in particular relates to the field of video-driven lighting technology. Background Technology

[0002] With the increasing popularity of home theaters, gaming, and immersive displays, users have higher demands for the synchronized experience of video footage and lighting. Current technologies for controlling the synchronization of video footage and lighting are mostly based on simple pixel averaging / edge color statistics, which have the following drawbacks:

[0003] 1. Insufficient anti-interference: When there are highlighted subtitles or skin-colored areas in the video, the lighting color is easily interfered with by non-subject content, resulting in a large color cast rate in the subtitles and a large color temperature shift caused by the skin-colored areas.

[0004] 2. Flickering and discomfort: When the video switches quickly or there are local bright changes, the lights flicker noticeably due to the lack of smooth transition, which affects the viewing experience and causes visual fatigue for users.

[0005] 3. Lack of modes: Existing solutions mostly use a single dimming strategy, which cannot automatically switch to the optimal parameters according to the user scenario (movie, game, music). Currently, most systems do not distinguish between movie / game / music, and users need to manually switch scenarios, and the switching delay is relatively large. Summary of the Invention

[0006] This application provides a video-based lighting dynamic optimization method, storage medium, and electronic device for improving the user experience of synchronizing video images with lighting.

[0007] In a first aspect, embodiments of this application provide a video-based dynamic lighting optimization method, comprising: acquiring a video frame; dividing the video frame into multiple Regions of Interest (ROIs) and acquiring the representative color of each ROI; identifying subtitle regions and skin tone regions in the video frame respectively, and adjusting the color weights of the subtitle regions and skin tone regions respectively to acquire a first video frame; converting the representative colors of each ROI region in the first video frame to HSV space, and performing smoothing and limiting processing on the HSV space to acquire a second video frame; acquiring a current video scene mode, and adjusting the second video frame according to dimming parameters pre-configured for the current video scene mode, and generating and outputting a third video frame.

[0008] In one implementation of the first aspect, obtaining the regional representative color of each ROI region includes: extracting the pixel components of the R, G, and B channels of all pixels within the ROI region; sorting the pixel components of each channel separately, and truncating the beginning and end of the pixel components of each channel in the sorting according to a preset ratio; obtaining the mean or median of the pixel components of each channel after truncation as the representative color of each channel; and synthesizing the representative colors of each channel to obtain the regional representative color of the ROI region.

[0009] In one implementation of the first aspect, identifying the subtitle region in the video frame includes: converting the video frame from an RGB image to an HSV / LAB image, and extracting the luminance channel of the HSV / LAB image; binarizing the luminance channel using a dynamic thresholding method, and separating the bright region from the video frame based on the luminance threshold of the luminance channel; filtering the bottom region of the video frame from the bright region based on a preset spatial position constraint, and using the filtered bottom region as the subtitle region.

[0010] In one implementation of the first aspect, identifying the skin color region of a person in the video frame includes: obtaining potential skin color pixels in the video frame through a skin color detection model; inputting the potential skin color pixels into a CNN classifier to obtain the confidence score of the skin color; when the confidence score of the skin color is greater than a confidence score threshold, confirming that the region is a skin color region of a person; otherwise, confirming that the region is a non-skin color region of a person.

[0011] In one implementation of the first aspect, obtaining potential skin-colored pixels in the video frame through a skin-colored detection model includes: converting the video frame from the RGB color space to the YCbCr color space; extracting the Cb and Cr chromaticity components of each pixel in the video frame from the YCbCr color space; traversing each pixel in the video frame and detecting whether the Cb and Cr chromaticity components of each pixel are within the skin-colored range; and, in response to the Cb and Cr chromaticity components of a pixel being within the skin-colored range, marking the pixel as a potential skin-colored pixel.

[0012] In one implementation of the first aspect, adjusting the color weights of the subtitle region and the skin tone region includes: reducing the color weight of the subtitle region to a first suppression range; and reducing the saturation contribution of the skin tone region to a second suppression range.

[0013] In one implementation of the first aspect, the smoothing and limiting processing of the HSV space includes: applying exponential smoothing and shortest path correction to the hue H in the HSV space; and limiting the brightness in the HSV space.

[0014] In one implementation of the first aspect, the shortest path correction of the angle adopts the shortest circular arc path correction method, and the exponential smoothing adopts the quaternion interpolation method.

[0015] In a second aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the video-based dynamic lighting optimization method described in any one of the first aspects of this application.

[0016] Thirdly, embodiments of this application provide an electronic device, the electronic device including a processor and a memory; the memory stores program instructions; the processor is configured to run the program instructions to execute the video-based dynamic lighting optimization method as described in any one of the first aspects of this application.

[0017] The video-based dynamic lighting optimization method provided in this application has the following beneficial effects:

[0018] This application can balance robustness, comfort, and scene adaptability, and effectively solve technical problems such as color distortion caused by bright subtitles / skin color interference, light flickering caused by rapid screen switching, and scene adaptation delay. Attached Figure Description

[0019] Figure 1 The diagram shown illustrates the implementation principle of a video-based dynamic lighting optimization method according to an embodiment of this application.

[0020] Figure 2 The diagram shown is an overall flowchart of a video-based dynamic lighting optimization method according to an embodiment of this application.

[0021] Figure 3 The diagram shown illustrates the principle of ROI color selection and robust statistics in a video-based dynamic lighting optimization method according to an embodiment of this application.

[0022] Figure 4 The flowchart shown is a video-based dynamic lighting optimization method for obtaining the region representative color of a ROI region according to an embodiment of this application.

[0023] Figure 5 The diagram shown illustrates the principle of subtitle / skin color detection and suppression in a video-based dynamic lighting optimization method according to an embodiment of this application.

[0024] Figure 6The flowchart shown is a process for obtaining the subtitle region in a video-based dynamic lighting optimization method according to an embodiment of this application.

[0025] Figure 7 The image shown is a pixel distribution histogram for subtitle detection in a video-based dynamic lighting optimization method according to an embodiment of this application.

[0026] Figure 8 The flowchart shown is a process for obtaining potential skin color pixels in a video-based lighting dynamic optimization method according to an embodiment of this application.

[0027] Figure 9 The flowchart shown is a process for obtaining potential skin color pixels in a video frame using a skin color detection model in a video-based dynamic lighting optimization method according to an embodiment of this application.

[0028] Figure 10 The image shown is a pixel distribution map of skin color detection in a video-based dynamic lighting optimization method according to an embodiment of this application.

[0029] Figure 11 The diagram shown is a flowchart illustrating the principle of HSV smoothing and Δ limiting processing in a video-based dynamic lighting optimization method according to an embodiment of this application.

[0030] Figure 12 The diagram shows the overall implementation process of a video-based dynamic lighting optimization method according to an embodiment of this application.

[0031] Figure 13 The figure shown is an example of comparing the dynamic curves of light under different modes during the implementation of a video-based lighting dynamic optimization method according to an embodiment of this application.

[0032] Figure 14 The diagram shown is a structural schematic of an electronic device according to an embodiment of this application. Detailed Implementation

[0033] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0034] The video-based dynamic lighting optimization method provided in this application is applicable to scenarios such as home theaters, e-sports entertainment, and stage performances, and is also suitable for in-vehicle immersive lighting. This method integrates robust region representative colors for anti-interference ROI areas, dual suppression of subtitles / skin tones, HSV smoothing limiting, and multi-mode adaptive control technologies to achieve low-latency, high-comfort immersive lighting synchronization on edge hardware (GPU / FPGA / ASIC).

[0035] The following will refer to the appendices in the embodiments of this application. Figure 1 To be continued Figure 11 The technical solutions in the embodiments of this application will be described in detail. This will enable those skilled in the art to understand and implement the video-based dynamic lighting optimization method of this embodiment without creative effort.

[0036] Figure 1 The diagram shown illustrates the implementation principle of a video-based dynamic lighting optimization method according to an embodiment of this application. Figure 1 As shown, in this application, robust regional representative colors are obtained by truncating the mean or median color of each ROI region pixel, effectively avoiding the influence of bright noise and abnormal pixels. Then, dual suppression of subtitles / skin tones is performed to reduce their weight on the overall color, ensuring that the light color focuses on the main tone of the video scene. After that, HSV smoothing and Δ limiting are applied to the video image. Exponential smoothing and shortest path correction are used for hue H, and Δ limiting is introduced for brightness V to limit the rapid changes in a single frame and avoid light flicker. Finally, the parameter chain (preset parameters for movies / music / games, etc.) is loaded according to the user-selected mode, and the light control commands are output.

[0037] Figure 2 The flowchart shown is a video-based dynamic lighting optimization method according to an embodiment of this application. Figure 1 As shown, the video-based dynamic lighting optimization method provided in this application includes the following steps S100 to S400.

[0038] Step S100: Obtain video frames, divide the video frame into multiple ROI regions, and obtain the region representative color of each ROI region.

[0039] Step S200: Identify the subtitle area and the skin tone area of ​​the person in the video frame respectively, and adjust the color weight of the subtitle area and the skin tone area of ​​the person respectively to obtain the first video frame;

[0040] Step S300: Convert the regional representative color of each ROI region in the first video frame to the HSV space, and perform smoothing and limiting processing on the HSV space to obtain the second video frame;

[0041] Step S400: Obtain the current video scene mode, adjust the second video frame according to the dimming parameters pre-configured for the current video scene mode, and generate and output the third video frame.

[0042] The video-based dynamic lighting optimization method provided in this application uses a fusion architecture of robust ROI region representative color + subtitle / skin tone dual interference suppression + multi-mode parameter chain to achieve light color deviation ≤△E3, a 70% reduction in flickering events, and mode switching latency <1 second, resulting in a more natural and comfortable immersive lighting experience. It can solve the technical problems of color cast caused by bright subtitle / skin tone interference (△E>15), light flickering caused by rapid scene switching (△V>0.5 / frame), and scene adaptation latency.

[0043] The following is in conjunction with the appendix Figure 2 To be continued Figure 10 The above steps S100 to S400 of the video-based dynamic lighting optimization method in this embodiment will be described in detail.

[0044] Step S100: Obtain video frames, divide the video frames into multiple ROI regions, and obtain the region representative color of each ROI region.

[0045] ROI (Region of Interest) refers to a key region in an image or video frame that has specific value. By selectively focusing on local features, processing efficiency and accuracy can be improved.

[0046] First, read the video frames. For example, you can use the OpenCV library in Python to read video frames.

[0047] In this embodiment, the video frame is divided into multiple ROI regions according to requirements. This can be achieved by specifying the coordinates of the top-left and bottom-right corners of each region. The specific process includes:

[0048] Get the frame size: For example, get the height and width of the video frame using frame.shape, and ensure that the ROI coordinates are within the valid range.

[0049] Define the ROI region:

[0050] Manual specification: Directly define the coordinates of the top left corner (x1, y1) and bottom right corner (x2, y2) of each ROI.

[0051] Automatic division: The screen is evenly divided into grids (e.g., 2×2, 3×3), and the coordinates of each ROI are automatically calculated.

[0052] Extract and label ROIs: Extract ROIs using NumPy slices and label them on the original frame with rectangles (cv2.rectangle) for easy visualization.

[0053] Figure 3 This diagram illustrates the principle of ROI color selection and robust statistics in a video-based dynamic lighting optimization method according to an embodiment of this application. Figure 3 As shown, the video frame is divided into multiple ROI regions. The mean or median of the pixels in each region is truncated to remove outliers and highlight interference, thus obtaining a robust representative color for each region.

[0054] Specifically, the video frame is divided into an N×M ROI grid (N, M≥3), and for each region, a truncation mean is calculated to discard the highest / lowest K% of pixels, for example, K∈[5,15]).

[0055] Figure 4 This is a flowchart illustrating the process of obtaining the region representative color of a ROI region in a video-based dynamic lighting optimization method according to an embodiment of this application. For example... Figure 4 As shown, in one implementation of this embodiment, obtaining the region representative color of each ROI region includes:

[0056] Step S110: Extract the pixel components of the R channel, G channel, and B channel of all pixels within the ROI region;

[0057] Step S120: Sort the pixel components of each channel separately, and truncate the beginning and end of the pixel components of each channel in the sorting according to a preset ratio; that is, sort each channel separately and truncate the beginning and end according to the ratio.

[0058] Step S130: Obtain the mean or median of the pixel components of each channel after truncation, and use it as the representative color of each channel.

[0059] The truncated mean is calculated by removing a certain percentage of extreme values. Specifically, the ROI region is flattened into a one-dimensional array, and the truncated mean is calculated for each channel. The lower and upper bounds of the truncation are also calculated. Then, the arrays are sorted and truncated, and the truncation ratio can be adjusted according to actual needs. For example, if there is a lot of highlight interference in the video frame, the truncation ratio can be appropriately increased. Finally, the truncated mean is calculated.

[0060] Step S140: Combine the representative colors of each channel to obtain the regional representative color of the ROI region. The bounding box of the ROI region can be drawn on the original image to display the representative color of each ROI region. When combining the representative colors of each channel, weighting coefficients can be configured for the selected channels.

[0061] Specifically, in this embodiment, a binary mask matrix (Mask) is created to mark the pixel positions of the corresponding ROI. In OpenCV, this can be generated using `cv2.inRange()` or connected component analysis. The RGB channels are processed separately, with the eigenvalues ​​of each channel calculated independently. A Quickselect algorithm can be used to accelerate the calculation. Then, the truncated mean / median is calculated, and finally, the three-dimensional vectors are synthesized to form the final representative color.

[0062] Specifically, in this embodiment, the video frame is divided into a 3×3 or 5×5 grid, adaptively based on the lighting fixture arrangement. In obtaining robust regional representative colors through truncation of the mean, the highest / lowest 10% of pixels (adjustable range 5%-15%) are discarded before calculating the mean. In obtaining robust regional representative colors through a median color algorithm, the median value of the HSV color space is used instead of the RGB HSV median color, and the CIE76 color difference formula is adopted to ensure brightness independence.

[0063] The above processing can be accelerated in parallel on GPU / FPGA through histogram statistics. The latency of FPGA parallel pipeline processing is preferably ≤2ms (resolution 3840x2160).

[0064] Step S200: Identify the subtitle area and the skin tone area of ​​the person in the video frame respectively, and adjust the color weight of the subtitle area and the skin tone area of ​​the person respectively to obtain the first video frame.

[0065] Figure 5 This diagram illustrates the principle of subtitle / skin color detection and suppression in a video-based dynamic lighting optimization method according to an embodiment of this application. Figure 5 As shown, the subtitle area (highlighted, near white, located at the bottom) is identified by brightness and color features, reducing the weight of the subtitle area on the overall color of the video. A skin tone detection model (YCbCr elliptical model) is used to identify the skin tone area of ​​the person, reducing the offset of the ambient light color. Through dual suppression of subtitles and skin tone, their weight on the overall color is reduced, ensuring that the lighting color focuses on the main tone of the video scene.

[0066] In this embodiment, when detecting and acquiring the subtitle area, it is done by combining Otsu threshold segmentation with position determination (e.g., the bottom 15% area ± 5%). Figure 6 The flowchart shown is a process for obtaining the subtitle region in a video-based dynamic lighting optimization method according to an embodiment of this application. For example... Figure 6 As shown, in one implementation of this embodiment, the identification of the subtitle region in the video frame includes the following steps S211 to S213.

[0067] Step S211: Convert the video image from an RGB image to an HSV / LAB image, and extract the luminance channel of the HSV / LAB image; specifically, use the OpenCV cv2.cvtColor function to convert the RGB image to an HSV or LAB image, then extract the V channel from the HSV image and the L channel from the LAB image.

[0068] Step S212: The luminance channel is binarized using a dynamic thresholding method, and the bright areas are separated from the video image based on the luminance threshold of the luminance channel.

[0069] Step S213: Based on preset spatial location constraints, filter the bottom region of the video frame from the highlighted region, and use the filtered bottom region as the subtitle region. For example, limit the search range to the lower half of the image (such as the bottom 1 / 4 region) to filter out highlighted interference that is not subtitles.

[0070] One method is to filter the highlighted areas using aspect ratio and area: unreasonable areas are excluded based on empirical values, such as areas with an aspect ratio > 5:1 or areas that are too small. Figure 7 As shown, the pixel distribution histogram of subtitle detection in a video-based dynamic lighting optimization method according to an embodiment of this application is displayed.

[0071] In this embodiment, a skin color detection model (YCbCr elliptical model) is combined with a CNN classifier to identify the skin color region of a person.

[0072] Figure 8 The flowchart shown is a process for obtaining potential skin color pixels in a video-based dynamic lighting optimization method according to an embodiment of this application. Figure 8 As shown, in one implementation of this embodiment, the identification of the skin color region of a person in the video image includes the following steps S221 to S225.

[0073] Step S221: Obtain potential skin color pixels in the video frame using a skin color detection model;

[0074] Step S222: Input the potential skin color pixels into a CNN classifier to obtain the confidence score of the skin color; wherein, the confidence score is, for example, >0.9.

[0075] Step S223: Determine whether the confidence level of the skin color is greater than the confidence threshold. If yes, continue to step S224: When the confidence level of the skin color is greater than the confidence threshold, confirm that the area is a human skin color area. If no, continue to step S225: Confirm that the area is a non-human skin color area.

[0076] In this embodiment, the input layer of the CNN classifier fuses 6 channels of features (RGB + YCbCr). The CNN classifier uses depthwise separable convolution + SE attention mechanism for feature extraction and then outputs a binary classification result: the probability of being a human skin color region (i.e., confidence) or the probability of being a non-human skin color region.

[0077] Figure 9 The flowchart shown is a process for obtaining potential skin color pixels in a video frame using a skin color detection model in a video-based dynamic lighting optimization method according to an embodiment of this application. Figure 9 As shown, obtaining potential skin color pixels in the video frame through a skin color detection model includes the following steps S2211 to S2214.

[0078] Step S2211: Convert the video frame from the RGB color space to the YCbCr color space;

[0079] In this embodiment, shadow compensation can be performed on the Y channel in the YCbCr color space. Specifically, the Y channel is brightened using the Retinex algorithm to alleviate color shift caused by shadows.

[0080] In the YCbCr color space, the Y channel directly reflects the brightness information of an image, while the Cb and Cr channels represent chromaticity (color) information. Therefore, processing the Y channel using the Retinex algorithm can alleviate color cast by brightening shadow areas while avoiding affecting the realism of the colors themselves, making it an effective solution for shadow compensation.

[0081] Specifically, shadow compensation for the Y channel in the YCbCr color space includes:

[0082] 1) Color space conversion: Convert the video frame from BGR (OpenCV default format) to YCbCr, separating the three channels: Y (luminance), Cb (blue chrominance), and Cr (red chrominance).

[0083] 2) Retinex processing of the Y channel: The core of the Retinex algorithm is to separate the illumination component and the reflection component of the image. By enhancing the reflection component, the shadows (dark areas) are brightened, while the highlight details are preserved (avoiding overexposure).

[0084] 3) Channel merging and color conversion: Merge the processed Y channel with the original Cb and Cr channels, and then convert it back to BGR format to obtain the shadow-compensated frame.

[0085] In Retinex, the image brightness I(x,y) is the product of the illumination component L(x,y) and the reflection component R(x,y). The illumination component L(x,y) can be approximated by Gaussian blur (different scale Gaussian kernels correspond to different ranges of illumination), and multi-scale Retinex balances local and global brightness by averaging Gaussian blurs at multiple scales.

[0086] Step S2212: Extract the Cb chromaticity component and Cr chromaticity component of each pixel in the video frame from the YCbCr color space; the Cb chromaticity component is used to control the blue tone component in skin color, and the Cr chromaticity component controls the red tone component in skin color.

[0087] Step S2213: Traverse each pixel in the video frame and detect whether the Cb chromaticity component and the Cr chromaticity component of each pixel are within the skin color range;

[0088] In the YCbCr elliptical model, the skin color range is as follows:

[0089] 77 ≤ Cb ≤ 127, 133 ≤ Cr ≤ 173.

[0090] In this embodiment, the color space dynamic threshold adjustment of YCbCr is also included. Specifically, the Cb / Cr boundary is adaptively expanded according to the scene illumination intensity, for example, the current skin color range in the YCbCr elliptical model is ±5%.

[0091] In the YCbCr color space, the chromaticity information (Cb and Cr channels) of skin tone has relatively stable distribution characteristics, and its range can usually be described by an elliptical model. However, changes in scene lighting (such as strong light and shadow) can cause skin tone chromaticity shifts. Therefore, it is necessary to dynamically adjust the boundary range of Cb / Cr according to the light intensity (such as expanding or shrinking the parameters of the elliptical model). In this embodiment, the light intensity is evaluated through the Y channel, and the Cb / Cr boundary of the skin tone elliptical model is adaptively expanded, as follows:

[0092] 1) Color space conversion: Convert video frames from BGR to YCbCr, separating the Y (luminance), Cb (blue chrominance), and Cr (red chrominance) channels.

[0093] 2) Light intensity assessment: Determine the scene lighting (too dark, too bright, or normal) by using the statistical characteristics of the Y channel (such as average brightness).

[0094] 3) Basic skin color ellipse model: Preset skin color Cb / Cr range under normal lighting (ellipse parameters: center coordinates, major axis / minor axis length, rotation angle).

[0095] 4) Dynamically adjust the boundary: Calculate the expansion coefficient based on the light intensity (such as the expansion range under extreme light conditions), adjust the major / minor axis (or boundary threshold) of the ellipse, and achieve adaptive expansion of the Cb / Cr range (such as ±5%~±15%).

[0096] Step S2214: In response to the pixel's Cb and Cr chromaticity components being within the skin tone range, the pixel is marked as a potential skin tone pixel. If both the Cb and Cr chromaticity components are within both ranges, it is determined to be a candidate point, i.e., a potential skin tone pixel. Figure 10 The image shown is a pixel distribution map of skin color detection in a video-based dynamic lighting optimization method according to an embodiment of this application.

[0097] Specifically, in one implementation of this embodiment, adjusting the color weights of the subtitle area and the skin tone area includes: reducing the color weight of the subtitle area to a first suppression range; and reducing the saturation contribution of the skin tone area to a second suppression range. For example, reducing the color weight of the subtitle area to ≤10% and reducing the saturation contribution of the skin tone area by 30%.

[0098] Step S300: Convert the region representative color of each ROI region in the first video frame to the HSV space, and perform smoothing and limiting processing on the HSV space to obtain the second video frame.

[0099] In this embodiment, the ROI representative color is converted to the HSV space, and then the hue H in the HSV space is smoothed exponentially and corrected by the shortest path angle. The brightness V in the HSV space is introduced with Δ limiting to restrict the rapid changes in a single frame and avoid light flicker.

[0100] Figure 11 The diagram shown illustrates the principle flowchart of HSV smoothing and Δ limiting processing in a video-based dynamic lighting optimization method according to an embodiment of this application. Figure 11 As shown, in one implementation of this embodiment, the smoothing and limiting processing of the HSV space includes: applying exponential smoothing and shortest path correction to the hue H in the HSV space; and limiting the brightness in the HSV space.

[0101] In this embodiment, the exponential smoothing method is: H'=α·H+(1−α)·H_prev, where the range of α is preferably [0.1,0.3].

[0102] This formula is used to reduce hue jumps, avoid light flicker, and improve visual comfort. Here, H represents the luminance value of the current frame (usually corresponding to the Value component in the HSV color space), which represents the original luminance of the current video frame; H_prev represents the luminance value of the previous frame, used to compare with the current frame to achieve inter-frame smoothing; H' represents the new luminance value after exponential smoothing, which is a weighted average of the current and previous frames, used to output the smoothed luminance. α is the smoothing factor (weighting parameter), controlling the degree of influence of the current and previous frames on the smoothing process.

[0103] A larger α value (closer to 1) results in a greater weight for the current frame and a weaker smoothing effect; a smaller α value (closer to 0) results in a greater weight for the previous frame and a stronger smoothing effect. The preferred range is [0.1, 0.3], indicating that the system tends towards a stronger smoothing effect to avoid abrupt changes in brightness. Exponential smoothing reduces inter-frame brightness fluctuations, suppresses flicker, and improves visual comfort. The "quaternion interpolation method" in the formula is used to handle angle values ​​(such as hue) to avoid jumps between 0° and 360° and ensure a smooth transition.

[0104] In one implementation of this embodiment, the shortest angle path correction adopts the shortest circular arc path correction method, and the exponential smoothing adopts the quaternion interpolation method to avoid 0°↔360° jump.

[0105] In this embodiment, the preferred amplitude limiting Δ is |V−V_prev| ≤ 0.2 (i.e., a maximum fluctuation of 20% per frame) to achieve visual smoothing. Where ΔV ≤ ΔV_max, it is adaptive according to the input frame rate fp.

[0106] Where V represents the luminance value of the current frame (similar to H, possibly referring to the Value component in HSV); V_prev represents the luminance value of the previous frame; ΔV represents the luminance change, i.e., the absolute value of |V - V_prev|, representing the luminance difference between adjacent frames; ΔV_max represents the maximum allowed luminance change, used for the limiting threshold. Its value is adaptively adjusted according to the input frame rate fp.

[0107]

[0108] Where fp represents the frame rate (unit: fps) of the input video, which means the number of frames per second.

[0109] △V_max increases with frame rate. At low frame rates (fp≤30), the limit is more stringent (△V_max=0.1) because changes per frame are more noticeable; at high frame rates (fp>60), greater changes are allowed (△V_max=0.3) because the intervals between frames are shorter.

[0110] By limiting the brightness changes between frames, flickering caused by sudden changes is prevented, thus achieving "visual smoothing".

[0111] For example, |V - V_prev| ≤ 0.2 ensures that the fluctuation does not exceed 20%, and combined with adaptive rules, it adapts to different video smoothness requirements.

[0112] In this embodiment, by performing HSV smoothing and Δ limiting on the first video frame, light flicker can be reduced, significantly improving user comfort.

[0113] Step S400: Obtain the current video scene mode, adjust the second video frame according to the dimming parameters pre-configured for the current video scene mode, and generate and output the third video frame.

[0114] In this embodiment, the video scene mode includes, but is not limited to, movie mode, music mode, and game mode; the dimming parameters pre-configured for the current video scene mode are, for example, as follows:

[0115] 1) Movie Mode: α=0.1, ΔV=0.1, Weights: Scene ROI=70%, Subtitle ROI≤5%;

[0116] 2) Music mode: α=0.25, ΔV=0.3, increase saturation by S×1.2;

[0117] 3) Game mode: α=0.35, ΔV=0.5, ROI update rate doubled, enhancing real-time feedback.

[0118] For example, α (smoothing factor) is set as follows:

[0119] Movie mode: α=0.1 (strong smoothing, suitable for static scenes).

[0120] Music mode: α=0.25 (medium smooth, balanced dynamics).

[0121] Game mode: α=0.35 (weak smoothing, priority to response speed).

[0122] Adjust the smoothing intensity according to different content types. Movie mode requires high smoothing to avoid flickering; game mode requires reduced smoothing to maintain real-time feedback.

[0123] The △V (amplitude limit) setting is as follows:

[0124] Movie mode: △V=0.1 (strict amplitude limiting to avoid brightness fluctuations).

[0125] Music mode: △V=0.3 (medium amplitude limit, allowing some variation).

[0126] Game mode: △V=0.5 (relaxed amplitude limit, adapting to fast-paced changes).

[0127] To control the range of brightness changes, the movie mode prioritizes stability, while the game mode tolerates greater variations to enhance realism.

[0128] Other parameter settings are as follows:

[0129] Weights (e.g., scene ROI=70%, subtitle ROI≤5%): ROI (Region of Interest) weights indicate that the main scene of the video (70%) is given priority during processing, while the subtitle area has a smaller impact (≤5%), in order to avoid subtitle flickering interfering with the main content.

[0130] Saturation S: Increases saturation by S×1.2 in music mode, which means increasing saturation by 20% to enhance color vibrancy, suitable for the visual presentation of music videos.

[0131] ROI update rate: Doubled in game mode, meaning the frequency of ROI detection and adjustment is increased, resulting in more frequent brightness adjustments to adapt to rapidly changing game scenes and enhance real-time performance.

[0132] Figure 12 This diagram illustrates the overall implementation process of a video-based dynamic lighting optimization method according to an embodiment of this application. Figure 12 As shown, after the video frame is input, the ROI region is first divided and color is extracted. Robust color statistics are then performed, obtaining a robust representative color for each ROI region by truncating the mean or median color of each pixel, effectively avoiding the influence of bright noise and abnormal pixels. Next, subtitle / skin tone detection and suppression are performed to reduce their weight on the overall color, ensuring the lighting color focuses on the main tone of the video scene. Then, HSV smoothing and Δ limiting are applied to the video image. Hue (H) is smoothed exponentially and corrected using the shortest path angle, while brightness (V) is Δ limiting to restrict rapid changes in a single frame and prevent light flicker. Then, multi-mode preset adjustments are performed, loading parameter chains (preset parameters for movies / music / games, etc.) according to the user-selected mode. Finally, lighting control output is performed, outputting lighting control commands.

[0133] Figure 13 The image shown is an example comparing the dynamic curves of light under different modes during the implementation of a video-based lighting dynamic optimization method according to an embodiment of this application. Figure 13 As shown, Figure 13 Four dynamic curves visually compare the output effects of the same video brightness signal under different processing modes, clearly demonstrating the optimization focus of different algorithm strategies.

[0134] like Figure 13As shown, the gray curve (original video brightness) serves as the baseline, representing the unprocessed raw signal, which fluctuates rapidly and dramatically. The blue curve (movie mode) changes the most smoothly, with peaks and troughs significantly smoothed, effectively eliminating rapid fluctuations and providing the most stable visual experience, suitable for immersive viewing environments. The green curve (game mode) has the fastest response speed, closely following the rapid changes in the raw signal with extremely low latency. Although slightly smoothed, it basically retains the instantaneous characteristics of the signal, suitable for game scenarios requiring fast feedback. The red curve (music mode) has characteristics between movie and game modes, offering a certain smoothness to ensure comfort while responding even faster than movie mode. Its significant feature is a higher overall output level (especially at the peaks), indicating that this mode may have superimposed saturation or brightness enhancement strategies to create a more impactful atmosphere.

[0135] pass Figure 13 The effectiveness of the multi-mode dimming scheme was verified. By adjusting parameters such as the smoothing coefficient (α) and the variation threshold (ΔV), optimizations for different application scenarios were successfully achieved.

[0136] 1) Movie Mode: Prioritize stability and suppress flicker.

[0137] 2) Game Mode: Prioritize response speed and reduce latency.

[0138] 3) Music Mode: Balances responsiveness and expressiveness, and enhances color effects.

[0139] This differentiated processing strategy ensures that users receive the best lighting synchronization experience in different usage scenarios.

[0140] For different scene modes, the video-based dynamic lighting optimization method of this application can achieve the following implementation effects:

[0141] 1) Home Theater: Subtitle weight reduced to 10%, color cast rate reduced from 42% to 5% (ΔE<3), processing latency ≤15ms;

[0142] 2) Esports games: Bandwidth limiting remains smooth, flickering events are reduced by 72% (ΔV≤0.2 / frame), and response latency is ≤45ms;

[0143] 3) Concert: Skin color interference reduced by 88%, saturation mapping error <5%.

[0144] The scope of protection of the video-based dynamic lighting optimization method described in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0145] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the video-based dynamic lighting optimization method provided in any embodiment of this application.

[0146] In the embodiments of this application, any combination of one or more storage media may be used. The storage medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0147] This application also provides an electronic device. Figure 14 The diagram shown is a structural schematic of the electronic device 100 provided in an embodiment of this application. In some embodiments, the electronic device 100 can be a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), or other terminal device. Furthermore, the video-based dynamic lighting optimization method provided in this application can also be applied to databases, servers, and service response systems based on terminal artificial intelligence. This application does not impose any limitations on the specific application scenarios of the video-based dynamic lighting optimization method.

[0148] like Figure 14 As shown, the electronic device 100 provided in this application embodiment includes a memory 101 and a processor 102.

[0149] The memory 101 is used to store computer programs; preferably, the memory 101 includes various media that can store program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card or optical disk.

[0150] Specifically, memory 101 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. Electronic device 100 may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 101 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0151] The processor 102 is connected to the memory 101 and is used to execute the computer program stored in the memory 101 so that the electronic device 100 executes the video-based lighting dynamic optimization method provided in any embodiment of this application.

[0152] Optionally, the processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0153] Optionally, in this embodiment, the electronic device 100 may further include a display 103. The display 103 is communicatively connected to the memory 101 and the processor 102, and is used to display the relevant GUI interactive interface of the video-based lighting dynamic optimization method.

[0154] In summary, this application balances robustness, comfort, and scene adaptability, effectively solving technical problems such as color distortion caused by bright subtitles / skin tone interference, light flickering caused by rapid scene switching, and scene adaptation delay. Therefore, this application effectively overcomes the various shortcomings of the prior art and has high industrial application value.

[0155] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A video-based dynamic lighting optimization method, characterized in that, include: Acquire video frames, divide the video frame into multiple ROI regions, and acquire the region representative color of each ROI region; The subtitle area and the skin tone area of ​​the person in the video frame are identified respectively, and the color weight of the subtitle area and the skin tone area of ​​the person are adjusted respectively to obtain the first video frame; The region representative color of each ROI region in the first video frame is converted to the HSV space, and the HSV space is smoothed and clipped to obtain the second video frame. Obtain the current video scene mode, adjust the second video frame according to the dimming parameters pre-configured for the current video scene mode, and generate and output the third video frame; The identified subtitle regions in the video frame include: The video image is converted from an RGB image to an HSV / LAB image, and the luminance channel of the HSV / LAB image is extracted. The luminance channel is binarized using a dynamic thresholding method, and the bright areas are separated from the video image based on the luminance threshold of the luminance channel. Based on preset spatial position constraints, the bottom area of ​​the video frame is selected from the highlighted area, and the selected bottom area is used as the subtitle area.

2. The video-based dynamic lighting optimization method according to claim 1, characterized in that, The method of obtaining the region representative color for each ROI region includes: Extract the pixel components of the R, G, and B channels of all pixels within the ROI region; The pixel components of each channel are sorted separately, and the beginning and end of the pixel components of each channel in the sorting are truncated according to a preset ratio. Obtain the mean or median of the pixel components of each channel after truncation, and use it as the representative color of each channel; The representative colors of each channel are combined to obtain the regional representative color of the ROI region.

3. The video-based dynamic lighting optimization method according to claim 1, characterized in that, Identifying skin tone regions in video footage includes: The potential skin color pixels in the video frame are obtained using a skin color detection model; The potential skin color pixels are input into a CNN classifier to obtain the confidence score of the skin color; If the confidence level of the skin color is greater than the confidence threshold, the area is confirmed as a human skin color area; otherwise, the area is confirmed as a non-human skin color area.

4. The video-based dynamic lighting optimization method according to claim 3, characterized in that, The step of obtaining potential skin color pixels in the video frame through a skin color detection model includes: The video frame is converted from the RGB color space to the YCbCr color space; Extract the Cb and Cr chromaticity components of each pixel in the video frame from the YCbCr color space; Traverse each pixel in the video frame and detect whether the Cb chromaticity component and the Cr chromaticity component of each pixel are within the skin color range; If the Cb chromaticity component and the Cr chromaticity component of the pixel are within the skin tone range, then the pixel is marked as a potential skin tone pixel.

5. The video-based dynamic lighting optimization method according to claim 1 or 3, characterized in that, The adjustment of the color weights of the subtitle area and the character's skin tone area includes: Reduce the color weight of the subtitle region to a first suppression range; Reduce the saturation contribution of the skin color region of the person to the second suppression range.

6. The video-based dynamic lighting optimization method according to claim 1, characterized in that, The smoothing and clipping processing of the HSV space includes: Exponential smoothing and shortest path correction are applied to the hue H in the HSV space; The brightness in the HSV space is limited.

7. The video-based dynamic lighting optimization method according to claim 6, characterized in that, The angle shortest path correction adopts the shortest circular arc path correction method, and the exponential smoothing adopts the quaternion interpolation method.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the video-based dynamic lighting optimization method as described in any one of claims 1 to 7.

9. An electronic device, characterized in that, The electronic device includes: Processor and memory; The memory stores program instructions; The processor is configured to run the program instructions to perform the video-based dynamic lighting optimization method as described in any one of claims 1 to 7.

Images