Video processing method, device and display device

Abstract

The application discloses a video processing method, device and display equipment. The video processing method can divide an initial video into multiple video segments and calculate the color temperatures of the multiple video segments. For each target video segment in the multiple video segments, the method can determine a color temperature adjustment coefficient of the target video segment based on the color temperature of the target video segment and a color balance correction coefficient of the target video segment, and adjust the color temperature of at least one video frame in the target video segment by using the color temperature adjustment coefficient. Thus, the color temperature adjustment effect of each video segment can be ensured to be good, and the display effect of the processed video can be ensured to be good.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a video processing method, apparatus and display device. Background Technology

[0002] In the field of image processing technology, color temperature is one of the standards for measuring image display quality. Due to the limitations of the performance of image acquisition devices (such as cameras), the images captured by these devices may deviate to some extent from the true color of the target object; that is, the images captured by the image acquisition devices have color temperature deviations.

[0003] In related technologies, in order to improve the display effect of an image, a color temperature adjustment coefficient can be used to adjust the color temperature of the image, thereby improving the display effect of the image.

[0004] However, in video scenarios, the colors of video images captured by image acquisition devices vary greatly in different scenarios. Therefore, if the same color temperature adjustment coefficient is used to adjust the color temperature of multiple frames of video images, the display effect of the video images cannot be guaranteed. Summary of the Invention

[0005] This application provides a video processing method, apparatus, and display device, which can solve the problem of poor video image display quality in related technologies. The technical solution is as follows:

[0006] On the one hand, a video processing method is provided, the method comprising:

[0007] The initial video is divided into multiple video segments, each of which includes one or more video frames, wherein the multiple video frames are consecutive.

[0008] For each of the plurality of video segments, the color temperature of the video segment is determined based on the color temperature of at least one video frame in the video segment;

[0009] For each target video segment among the plurality of video segments, a color temperature adjustment coefficient for the target video segment is determined based on the color temperature of the target video segment and the color balance correction coefficient of the target video segment. The color temperature adjustment coefficient is positively correlated with both the color temperature of the target video segment and the color balance correction coefficient of the target video segment. The number of target video segments included in the initial video is less than or equal to the number of the plurality of video segments.

[0010] For each target video segment, the color temperature of each video frame in the target video segment is adjusted using the color temperature adjustment coefficient of the target video segment;

[0011] The target video is obtained by splicing together the at least one target video segment with adjusted color temperature and other video segments in the playback order.

[0012] Optionally, determining the color temperature of the video segment based on the color temperature of at least one video frame in the video segment includes:

[0013] Perform spatial transformation on each video frame in the video segment;

[0014] The color temperature of the video frame is determined based on the average value of each channel of the multiple pixels included in each video frame after spatial transformation.

[0015] The color temperature of the video segment is determined based on the average color temperature of at least one video frame in the video segment.

[0016] Optionally, the spatial transformation of each video frame in the video segment includes:

[0017] Each video frame in the video segment is converted from the red-green-blue (RGB) color space to the hue-saturation-luminance (HSV) color space and the luminance-chrominance (YUV) color space, respectively.

[0018] Determining the color temperature of a video frame based on the average value of each channel of multiple pixels included in each video frame after spatial transformation includes:

[0019] For each video frame in the video segment, the color temperature of the video frame is determined based on the average values ​​of the multiple pixels included in the video frame in the Y-channel, U-channel, and V-channel of the YUV color space, and the average value of the multiple pixels included in the HSV color space in the S-channel. The color temperature of the video frame is positively correlated with the average values ​​of the multiple pixels included in the video frame in the Y-channel and U-channel, and negatively correlated with the average values ​​of the multiple pixels included in the video frame in the V-channel and S-channel.

[0020] Optionally, the color temperature CT of the video frame satisfies:

[0021]

[0022] Where α is a preset gain coefficient, Y mean U is the mean value on the Y channel. mean V is the mean value on the U channel. mean C is the mean value on the V channel. minus For the Umean and the V mean The absolute value of the difference, S mean The mean value on the S channel.

[0023] Optionally, multiple video frames in the initial video are high-dynamic-range (HDR) images, and the value of α is 5000.

[0024] Optionally, before determining the color temperature adjustment coefficient of the target video segment, the method further includes:

[0025] Based on the color temperature of the plurality of video segments, at least one target video segment is determined from the plurality of video segments;

[0026] The color temperature of each target video segment is outside the preset color temperature range.

[0027] Optionally, the color temperature range is 6000 Kelvin (K) to 7000 K.

[0028] Optionally, the color temperature adjustment coefficient of the target video segment includes: R channel adjustment coefficient K. R G-channel adjustment coefficient K G and the B-channel adjustment coefficient K B ;

[0029] Wherein, the R-channel adjustment coefficient K R Satisfy: K R =CT`×β1×Avg_gain′ R ;

[0030] The G-channel adjustment coefficient K G Satisfy: K G =CT`×β2×Avg_gain′ G ;

[0031] The B-channel adjustment coefficient K B Satisfy: K B =CT`×β3×Avg_gain′ B ;

[0032] Where β1, β2, and β3 are preset color temperature reference coefficients, CT` is the color temperature of the target video segment, and Avg_gain′ R Avg_gain′ is the color balance correction coefficient for the target video segment in the R channel. G Avg_gain′ is the color balance correction coefficient for the target video segment in the G channel. B The color balance correction coefficient for the target video segment in the B channel.

[0033] Optionally, adjusting the color temperature of each video frame in the target video segment using the color temperature adjustment coefficient of the target video segment includes:

[0034] For each video frame in the target video segment, the R-channel adjustment coefficient K is used. R The R pixel value of each pixel in the video frame is adjusted.

[0035] Using the G-channel adjustment coefficient K G The G-pixel value of each pixel in the video frame is adjusted.

[0036] Using the B-channel adjustment coefficient K B The B-pixel value of each pixel in the video frame is adjusted.

[0037] Optionally, the Avg_gain′ R The mean value of the color balance correction coefficients on the R channel for at least one video frame in the target video segment;

[0038] The Avg_gain′ G The mean value of the color balance correction coefficients on the G channel for at least one video frame in the target video segment;

[0039] The Avg_gain′ B It is the mean of the color balance correction coefficients on the B channel for at least one video frame in the target video segment.

[0040] Optionally, the color balance correction coefficient gain on the R channel for each video frame R Satisfy: gain R =K / R avg +a;

[0041] The color balance correction factor gain for each video frame on the G channel R Satisfy: gain B =K / G avg +b;

[0042] The color balance correction factor gain for each video frame on the B channel B Satisfy: gain B =K / B avg +c;

[0043] Among them, R avg G is the average R pixel value of multiple pixels in the video frame. avg B is the average of the G pixel values ​​of multiple pixels in the video frame.avg Let B be the average value of multiple pixels in the video frame, a, b, and c be preset reference deviation values, and K be the R... avg The G avg and the aforementioned B avg The mean.

[0044] Optionally, multiple video frames in the initial video are HDR images, where the value of 'a' is 20, the value of 'b' is 10, and the value of 'c' is 0.

[0045] Optionally, dividing the initial video into multiple video segments from multiple video frames includes:

[0046] According to the playback order of the multiple video frames included in the initial video, the similarity between each video frame and the previous video frame is calculated sequentially.

[0047] Based on the calculated similarity between each pair of adjacent video frames, the initial video is divided into multiple video segments.

[0048] Optionally, each video frame in the initial video includes multiple image blocks; the step of calculating the similarity between each video frame and the previous video frame in the order of playback of the multiple video frames included in the initial video includes:

[0049] In each video frame of the initial video, at least one target image patch is identified, and the number of the at least one target image patch is less than the number of the plurality of image patches;

[0050] According to the playback order of the multiple video frames included in the initial video, the similarity between at least one target image block in each video frame and at least one target image block in the previous video frame is calculated sequentially.

[0051] In each video frame, at least one target image block is positioned in the same position as at least one target image block in the previous video frame.

[0052] Optionally, the method further includes calculating the similarity between each video frame and the previous video frame in the order of playback of the multiple video frames included in the initial video.

[0053] Each initial video frame in the initial video is subjected to dimensionality reduction processing to obtain the plurality of video frames.

[0054] Optionally, the step of sequentially calculating the similarity between each video frame and the previous video frame includes:

[0055] Based on the mean of the image data of each video frame and the mean of the image data of the previous video frame, the standard deviation of the image data of the video frame and the standard deviation of the image data of the previous video frame, and the covariance of the image data of the video frame and the image data of the previous video frame, the structural similarity between the video frame and the previous video frame is determined.

[0056] The similarity between the video frame and the previous video frame is determined based on the structural similarity between the video frame and the previous video frame.

[0057] Optionally, the method further includes: using a structural similarity (SSIM) algorithm to sequentially calculate the similarity between each video frame and the previous video frame.

[0058] On the other hand, a video processing apparatus is provided, the apparatus comprising:

[0059] A segmentation module is used to divide the initial video, which includes multiple video frames, into multiple video segments, each of which includes one or more video frames, wherein the multiple video frames are consecutive.

[0060] The first determining module is used to determine the color temperature of each of the plurality of video segments based on the color temperature of at least one video frame in the video segment;

[0061] The second determining module is used to determine the color temperature adjustment coefficient of each target video segment among the plurality of video segments, based on the color temperature of the target video segment and the color balance correction coefficient of the target video segment, wherein the color temperature adjustment coefficient is positively correlated with both the color temperature and the color balance correction coefficient, wherein the number of target video segments included in the initial video is less than or equal to the number of the plurality of video segments;

[0062] The processing module is used to adjust the color temperature of each video frame in the target video segment by using the color temperature adjustment coefficient of the target video segment;

[0063] The splicing module is used to splice the at least one target video segment after color temperature adjustment, as well as other video segments besides the at least one target video segment, in the order of playback to obtain the target video.

[0064] Optionally, the first determining module is configured to:

[0065] Perform spatial transformation on each video frame in the video segment;

[0066] The color temperature of the video frame is determined based on the average value of each channel of the multiple pixels included in each video frame after spatial transformation.

[0067] The color temperature of the video segment is determined based on the average color temperature of at least one video frame in the video segment.

[0068] Optionally, the first determining module is configured to:

[0069] Each video frame in the video segment is converted from the RGB color space to the HSV color space and the YUV color space, respectively.

[0070] For each video frame in the video segment, the color temperature of the video frame is determined based on the average values ​​of the multiple pixels included in the video frame in the Y-channel, U-channel, and V-channel of the YUV color space, and the average value of the multiple pixels included in the HSV color space in the S-channel. The color temperature of the video frame is positively correlated with the average values ​​of the multiple pixels included in the video frame in the Y-channel and U-channel, and negatively correlated with the average values ​​of the multiple pixels included in the video frame in the V-channel and S-channel.

[0071] Optionally, the color temperature CT of the video frame satisfies:

[0072]

[0073] Where α is a preset gain coefficient, Y mean U is the mean value on the Y channel. mean V is the mean value on the U channel. mean C is the mean value on the V channel. minus For the U mean and the V mean The absolute value of the difference, S mean The mean value on the S channel.

[0074] Optionally, multiple video frames in the initial video are HDR images, and the value of α is 5000.

[0075] Optionally, the video processing apparatus further includes: a third determining module, the third determining module being used to determine at least one target video segment from the plurality of video segments based on the color temperature of the plurality of video segments;

[0076] The color temperature of each target video segment is outside the preset color temperature range.

[0077] Optionally, the color temperature range is 6000K to 7000K.

[0078] Optionally, the color temperature adjustment coefficient of the target video segment includes: R channel adjustment coefficient K. R G-channel adjustment coefficient K G and the B-channel adjustment coefficient K B ;

[0079] Wherein, the R-channel adjustment coefficient K R Satisfy: K R =CT`×β1×Avg_gain′ R ;

[0080] The G-channel adjustment coefficient K G Satisfy: K G =CT`×β2×Avg_gain′ G ;

[0081] The B-channel adjustment coefficient K B Satisfy: K B =CT`×β3×Avg_gain′ B ;

[0082] Where β1, β2, and β3 are preset color temperature reference coefficients, CT` is the color temperature of the target video segment, and Avg_gain′ R Avg_gain′ is the color balance correction coefficient for the target video segment in the R channel. G Avg_gain′ is the color balance correction coefficient for the target video segment in the G channel. B The color balance correction coefficient for the target video segment in the B channel.

[0083] Optionally, the processing module is configured to:

[0084] For each video frame in the target video segment, the R-channel adjustment coefficient K is used. R The R pixel value of each pixel in the video frame is adjusted.

[0085] Using the G-channel adjustment coefficient K G The G-pixel value of each pixel in the video frame is adjusted.

[0086] Using the B-channel adjustment coefficient K B The B-pixel value of each pixel in the video frame is adjusted.

[0087] Optionally, the Avg_gain′ R The mean value of the color balance correction coefficients on the R channel for at least one video frame in the target video segment;

[0088] The Avg_gain′ GThe mean value of the color balance correction coefficients on the G channel for at least one video frame in the target video segment;

[0089] The Avg_gain′ B It is the mean of the color balance correction coefficients on the B channel for at least one video frame in the target video segment.

[0090] Optionally, the color balance correction coefficient gain on the R channel for each video frame R Satisfy: gain R =K / R avg +a;

[0091] The color balance correction factor gain for each video frame on the G channel B Satisfy: gain B =K / G avg +b;

[0092] The color balance correction factor gain for each video frame on the B channel B Satisfy: gain B =K / B avg +c;

[0093] Among them, R avg G is the average R pixel value of multiple pixels in the video frame. avg B is the average of the G pixel values ​​of multiple pixels in the video frame. avg Let B be the average value of multiple pixels in the video frame, a, b, and c be preset reference deviation values, and K be the R... avg The G avg and the aforementioned B avg The mean.

[0094] Optionally, multiple video frames in the initial video are HDR images, where the value of 'a' is 20, the value of 'b' is 10, and the value of 'c' is 0.

[0095] Optionally, the partitioning module is used for:

[0096] According to the playback order of the multiple video frames included in the initial video, the similarity between each video frame and the previous video frame is calculated sequentially.

[0097] Based on the calculated similarity between each pair of adjacent video frames, the initial video is divided into multiple video segments.

[0098] Optionally, each video frame in the initial video includes multiple image blocks; the segmentation module is used to:

[0099] In each video frame of the initial video, at least one target image patch is identified, and the number of the at least one target image patch is less than the number of the plurality of image patches;

[0100] According to the playback order of the multiple video frames included in the initial video, the similarity between at least one target image block in each video frame and at least one target image block in the previous video frame is calculated sequentially.

[0101] In each video frame, at least one target image block is positioned in the same position as at least one target image block in the previous video frame.

[0102] Optionally, the video processing device further includes a dimensionality reduction module, which is used to perform dimensionality reduction processing on each initial video frame in the initial video before the segmentation module calculates the similarity between each video frame and the previous video frame in the playback order of the multiple video frames included in the initial video, so as to obtain the multiple video frames.

[0103] Optionally, the partitioning module is used for:

[0104] Based on the mean of the image data of each video frame and the mean of the image data of the previous video frame, the standard deviation of the image data of the video frame and the standard deviation of the image data of the previous video frame, and the covariance of the image data of the video frame and the image data of the previous video frame, the structural similarity between the video frame and the previous video frame is determined.

[0105] The similarity between the video frame and the previous video frame is determined based on the structural similarity between the video frame and the previous video frame.

[0106] In another aspect, a display device is provided, the display device comprising: a display screen, a processor, and a memory, the memory storing instructions which are loaded and executed by the processor to implement the video processing method provided above.

[0107] In another aspect, a computer-readable storage medium is provided, wherein instructions are stored therein, which are loaded and executed by a processor to implement the video processing method provided in the above aspects.

[0108] In another aspect, a computer program product is provided, the computer program product including computer instructions, which are loaded and executed by a processor to implement the video processing method provided in the above aspects.

[0109] The beneficial effects of the technical solution provided in this application include at least the following:

[0110] This application provides a video processing method, apparatus, and display device. The video processing method divides an initial video into multiple video segments and calculates the color temperature of each video segment. For each target video segment, the method determines a color temperature adjustment coefficient based on the target video segment's color temperature and its color balance correction coefficient, and then uses this adjustment coefficient to adjust the color temperature of at least one video frame within that target video segment. This ensures a good color temperature adjustment effect for each video segment, thereby ensuring a good display effect for the processed video. Attached Figure Description

[0111] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0112] Figure 1 This is a schematic diagram of the structure of a video processing system provided in an embodiment of this application;

[0113] Figure 2 This is a flowchart illustrating a video processing method provided in an embodiment of this application;

[0114] Figure 3 This is a flowchart illustrating another video processing method provided in an embodiment of this application;

[0115] Figure 4 This is a schematic diagram illustrating how a terminal processes an initial video according to an embodiment of this application;

[0116] Figure 5 This is a color temperature distribution map of a video clip provided in an embodiment of this application;

[0117] Figure 6 This is a schematic diagram of the structure of a video processing device provided in an embodiment of this application;

[0118] Figure 7 This is a schematic diagram of another video processing device provided in an embodiment of this application;

[0119] Figure 8 This is a schematic diagram of the structure of a display device provided in an embodiment of this application. Detailed Implementation

[0120] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0121] Figure 1 This is a schematic diagram of the structure of a video processing system provided in an embodiment of this application. See also... Figure 1 The system includes a server 110 and a terminal 120. A wired or wireless communication connection is established between the server 110 and the terminal 120. Optionally, the server 110 can be a standalone physical server, a server cluster consisting of multiple physical servers, or a distributed system. The terminal 120 can be a personal computer (PC), in-vehicle terminal, tablet computer, smartphone, wearable device, or intelligent robot, or any other terminal with a display screen and data computing, processing, and storage capabilities.

[0122] In this embodiment, the terminal 120 in the system can be used to acquire an initial video and send it to the server 110. The server 110 can then process and analyze the initial video and send the processed target video to the terminal 120 for display. That is, the terminal 120 can be a display device.

[0123] Optionally, the terminal 120 may have a browser or video application installed, and the server 110 may be a backend server for a video website or video application.

[0124] Understandably, the terminal 120 can also process and analyze the initial video, and adjust its color temperature. Correspondingly, the video processing system may not include the server 110.

[0125] Figure 2 This is a flowchart illustrating a video processing method provided in an embodiment of this application. This method can be applied to... Figure 1 The scenario shown is represented by server 110 or terminal 120. The following explanation uses the application of this video processing method to a terminal as an example. Figure 2 As shown, the method includes the following steps.

[0126] Step 101: Divide the multiple video frames included in the initial video into multiple video segments.

[0127] In this embodiment, after acquiring an initial video, the terminal can divide the multiple video frames into multiple video segments based on the similarity of these frames. Each video segment includes one or more video frames. If each video segment includes multiple video frames, these frames are consecutive. Each video segment can also be referred to as a scene, and correspondingly, the process of dividing the video segments can be called scene segmentation.

[0128] It is understandable that the multiple video frames in this initial video were captured by an image acquisition device (such as a camera), and this initial video may contain multiple different scenes. The color temperature of video frames from different scenes will be different, while the color temperature of video frames from the same scene can be basically the same. Therefore, before processing an initial video, the terminal can first segment the initial video into scenes. This allows the terminal to further process multiple video frames based on the scenes they belong to.

[0129] Optionally, the terminal may pre-store an image similarity algorithm. Based on this algorithm, the terminal can calculate the similarity between each video frame and the previous video frame, thereby dividing the initial video into multiple video segments. For example, the image similarity algorithm can be the SSIM algorithm, cosine similarity algorithm, or Euclidean distance (also known as Euclidean distance) algorithm.

[0130] Step 102: For each of the multiple video segments, determine the color temperature of the video segment based on the color temperature of at least one video frame in the video segment.

[0131] In this embodiment, for each video segment, the terminal can first calculate the color temperature of each video frame in the video segment. Then, the terminal can determine the color temperature of the video segment based on the color temperature of at least one video frame included in the video segment.

[0132] For example, if the video clip consists of only one video frame, the terminal can directly determine the color temperature of that video frame as the color temperature of the video clip. If the video clip consists of at least two video frames, the terminal can determine the average color temperature of the at least two video frames as the color temperature of the video clip. Alternatively, the terminal can determine the median color temperature of the at least two video frames as the color temperature of the video clip. Still another option is to determine the mode color temperature of the at least two video frames as the color temperature of the video clip.

[0133] Step 103: For each target video segment among multiple video segments, determine the color temperature adjustment coefficient of the target video segment based on the color temperature of the target video segment and the color balance correction coefficient of the target video segment.

[0134] In this embodiment, the terminal pre-stores an image processing algorithm for calculating the color balance correction coefficients of video frames. For each target video segment, the terminal first uses the image processing algorithm to calculate the color balance correction coefficients of at least one video frame in the target video segment, and then determines the color balance correction coefficient of the target video segment based on the color balance correction coefficients of the at least one video frame. The color temperature adjustment coefficient of the target video segment is positively correlated with both the color temperature of the target video segment and the color balance correction coefficient of the target video segment.

[0135] It is understood that the number of target video segments included in the initial video can be less than or equal to the number of multiple video segments. That is, the terminal can process some or all of the multiple video segments. Optionally, the terminal can determine at least one target video segment from the multiple video segments based on the image features of the multiple video segments. The image features may include at least one of the following features: color temperature, saturation, sharpness, and color.

[0136] Optionally, the image processing algorithm pre-stored in the terminal can be one of the following algorithms: gray world algorithm, perfect reflection algorithm, dynamic thresholding algorithm, and automatic white balance algorithm based on color temperature estimation.

[0137] Step 104: For each target video segment, use the color temperature adjustment coefficient of the target video segment to adjust the color temperature of each video frame in the target video segment.

[0138] In this embodiment, in the RGB color space, each pixel in each video frame of the target video segment includes channel values ​​for three color channels: red (R), green (G), and blue (B). These three channel values ​​can also be referred to as R pixel values, G pixel values, and B pixel values. The terminal can adjust the color temperature of the video frame by adjusting the R pixel values, G pixel values, and B pixel values ​​of each pixel in the video frame.

[0139] It is understandable that the color cast (i.e., color temperature deviation) of video frames within the same target video segment (i.e., the same scene) is identical. Therefore, the terminal can use the same color temperature adjustment coefficient to adjust the color temperature of at least one video frame within the target video segment, thereby achieving color balance correction (i.e., white balance correction) for that at least one video frame. This ensures that the processing effect of video frames within the scene corresponding to the target video segment is good, thus ensuring a good display effect for the processed video.

[0140] Step 105: According to the playback order, splice at least one target video segment after color temperature adjustment, and the video segments other than at least one target video segment, to obtain the target video.

[0141] In this embodiment, after adjusting the color temperature of at least one target video segment, the terminal can splice the at least one video segment and the video segments that have not undergone color temperature adjustment from the multiple video segments obtained by segmenting the initial video, according to the playback order, to obtain the target video. That is, in the target video, some video segments have undergone color temperature adjustment, while others have not. This ensures that the display effect of the target video is good.

[0142] In summary, this application provides a video processing method in which a terminal can divide an initial video into multiple video segments and calculate the color temperature of each video segment. For each target video segment, the terminal can determine a color temperature adjustment coefficient based on the color temperature and color balance correction coefficient of the target video segment, and then use this adjustment coefficient to adjust the color temperature of at least one video frame in the target video segment. This ensures a good color temperature adjustment effect for each video segment, thereby ensuring a good display effect for the processed video.

[0143] Figure 3 This is a flowchart illustrating another video processing method provided in an embodiment of this application. This method can be applied to... Figure 1 The scenario shown is represented by server 110 or terminal 120. The following explanation uses the application of this video processing method to a terminal as an example. Figure 3 As shown, the method includes the following steps.

[0144] Step 201: Calculate the similarity between each video frame and the previous video frame in the order of playback of the multiple video frames included in the initial video.

[0145] In this embodiment, the terminal pre-stores an image similarity algorithm. After acquiring the initial video, the terminal can use this image similarity algorithm to calculate the similarity between each video frame and the previous video frame.

[0146] Understandably, when calculating the similarity between each video frame and the previous video frame using an image similarity algorithm, the terminal can determine the similarity between the two video frames based on their structural similarity. Optionally, the terminal can determine the structural similarity between the two video frames based on the mean of the image data of each video frame and the mean of the image data of the previous video frame, the standard deviation of the image data of the current video frame and the standard deviation of the image data of the previous video frame, and the covariance of the image data of the current video frame and the image data of the previous video frame.

[0147] Optionally, the terminal can employ the SSIM algorithm to sequentially calculate the structural similarity between each video frame and the previous video frame, thereby determining the similarity between each video frame and the previous video frame. The structural similarity SSIM(X, Y) between video frame X and the previous video frame Y satisfies:

[0148]

[0149] Where, μ x μ represents the mean of the image data in video frame X. y σ represents the mean of the image data in video frame Y. x σ represents the standard deviation of the image data in video frame X. y σ represents the standard deviation of the image data in video frame Y. xy SSIM(X,Y) represents the covariance of the image data of video frame X and video frame Y. c1 and c2 are preset constants. The structural similarity between video frame X and the previous video frame Y can satisfy: 0≤SSIM(X,Y)≤1. The closer the structural similarity values ​​of two video frames are to 1, the more similar the two video frames are.

[0150] It is understandable that video frames in the same scene have a high degree of structural similarity, while video frames in different scenes have a low degree of structural similarity. Therefore, the SSIM algorithm can be used to calculate the structural similarity between two adjacent video frames in the initial video, so that the terminal can perform scene segmentation on the initial video based on the structural similarity and ensure that the scene segmentation effect is good.

[0151] As a first possible implementation, each video frame in the initial video may include multiple image blocks. The terminal can determine at least one target image block in each video frame of the initial video, and calculate the similarity between at least one target image block in each video frame and at least one target image block in the previous video frame in the order of playback of the multiple video frames included in the initial video.

[0152] In each video frame, the number of at least one target image block is less than the number of multiple image blocks, and the position of the at least one target image block selected in each video frame is the same as and corresponds one-to-one with the position of the at least one target image block selected in the previous video frame.

[0153] In this implementation, the terminal only needs to calculate the similarity of a portion of the regions (i.e., corresponding target image patches in the two video frames) between two video frames, and determine the similarity of this portion of the regions as the similarity between the two video frames. This effectively reduces the computational complexity of similarity calculation and improves the efficiency of similarity calculation. The similarity of the target image patches in the two adjacent video frames can be calculated using the SSIM algorithm.

[0154] As a second possible implementation, the terminal can perform dimensionality reduction processing on each initial video frame in the initial video to obtain multiple video frames. Then, the terminal can use an image similarity algorithm (such as the SSIM algorithm) to calculate the similarity between each of these multiple video frames and the previous video frame.

[0155] In this embodiment, the terminal may have a pre-stored data dimensionality reduction algorithm. The terminal can use this algorithm to reduce the dimensionality of each initial video frame in the initial video, thereby transforming the image data of the initial video frame from high dimension to low dimension, thus reducing the amount of data in each initial video frame. This data dimensionality reduction algorithm can be principal component analysis (PCA), t-distributed stochastic neighbor embedding (t-SNE), or a neural network algorithm, etc.

[0156] Understandably, using video frames that have undergone dimensionality reduction for similarity calculation at the terminal can effectively reduce the complexity of similarity calculation and improve computational efficiency.

[0157] Step 202: Based on the calculated similarity between each pair of adjacent video frames, divide the initial video into multiple video segments.

[0158] After calculating the similarity between any two adjacent video frames in the initial video, the terminal can sequentially traverse each video frame according to the playback order of the multiple video frames included in the initial video. For each traversed video frame, if the similarity between the video frame and the previous video frame is greater than a similarity threshold, the terminal can classify the video frame and the previous video frame into the same video segment (i.e., the same scene). If the similarity between the video frame and the previous video frame is not greater than the similarity threshold, the terminal can classify the video frame and the previous video frame into different video segments.

[0159] For example, for each video frame traversed by the terminal, if the similarity between the current video frame and the previous video frame is greater than a similarity threshold, the terminal can determine that the two video frames belong to the same video segment and continue traversing the video frames. If the similarity between the current video frame and the previous video frame is less than or equal to the similarity threshold, the terminal can record the frame number of the currently traversed video frame. After traversing all the video frames, the terminal can identify the video frame indicated by each recorded frame number as the first video frame of a video segment.

[0160] Based on the above segmentation method, the terminal can divide the initial video into multiple video segments. Each video segment includes at least one consecutive video frame. Each video segment can also be referred to as a scene, and the process of dividing the video into segments can also be called scene segmentation.

[0161] It is understandable that the multiple video frames in this initial video were captured by an image acquisition device (such as a camera), and this initial video may contain multiple different scenes. The color temperature of video frames from different scenes will be different, while the color temperature of video frames from the same scene can be basically the same. Therefore, before processing an initial video, the terminal can first segment the initial video into scenes. This allows the terminal to further process multiple video frames based on the scenes they belong to.

[0162] Example, reference Figure 4 After the initial video is segmented into scenes by the terminal, n video segments, or n scenes, can be obtained. Here, n is an integer greater than or equal to 1.

[0163] Understandably, the similarity threshold used by the terminal when segmenting video clips can be set according to the needs of the application scenario. For example, if the terminal uses the SSIM algorithm to calculate the similarity between every two adjacent video frames, the similarity threshold can be 0.5.

[0164] Step 203: For each video segment in the multiple video segments, perform spatial transformation on each video frame in that video segment.

[0165] In this embodiment, to facilitate the determination of the color temperature of each video frame in a video segment, the terminal can first perform color space conversion on each video frame in each video segment. Since the color space of each video frame in a video segment is generally RGB, while HSV and YUV color spaces are more convenient for the terminal to determine the color temperature of each video frame, for each video segment in multiple video segments, the terminal can convert (i.e., map) each video frame in that video segment from the RGB color space to the HSV and YUV color spaces respectively. In the HSV color space, H represents chroma, S represents saturation, and V represents brightness. In the YUV color space, Y represents brightness (i.e., grayscale), and U and V represent chroma.

[0166] Step 204: Determine the color temperature of the video frame based on the average value of each channel of the multiple pixels included in each video frame after spatial transformation.

[0167] In this embodiment, for each video frame in a video segment, after spatial transformation of the video frame, the terminal can calculate the average value of each channel of the multiple pixels included in the video frame after spatial transformation. Then, based on the calculated average values, the terminal can determine the color temperature of the video frame.

[0168] For example, for each video frame in a video segment, after converting the video frame from RGB color space to YUV and HSV color spaces, the terminal can first calculate the average values ​​of the multiple pixels in the video frame in the YUV color space (Y channel, U channel, and V channel) and the average value of the multiple pixels in the HSV color space (S channel). Then, based on the calculated average values, the terminal can determine the color temperature of the video frame. The color temperature CT of the video frame can satisfy the following:

[0169]

[0170] Where α is a preset gain coefficient, Y mean U is the mean value on the Y channel. mean V is the mean value on channel U. mean C is the mean value on the V channel. minus For U mean and V mean The absolute value of the difference, S mean This is the average value over the S channel. The value of this preset gain coefficient α can be determined based on the image type of the video frames. For example, when multiple video frames in the initial video are HDR images, the value of α can be 5000.

[0171] Referring to the above formula (1), it can be seen that the color temperature CT of a video frame is related to the average value Y of the multiple pixels included in that video frame in the Y channel. mean and the mean U on the U channel mean All are positively correlated with the mean V of the multiple pixels included in the video frame on the V channel. mean and the mean S on the S channel mean Both are negatively correlated.

[0172] It is understandable that when the color temperature of a video frame is high, the colors of that video frame will appear cooler. When the color temperature of a video frame is low, the colors of that video frame will appear warmer.

[0173] It is also understood that, referring to the above formula (1), the terminal uses the average value of the three channels in the YUV color space and the average value of the S channel in the HSV color space to calculate the color temperature. Optionally, the terminal may also use the average value of the three channels in the YUV color space and the average value of the H channel or V channel in the HSV color space to calculate the color temperature. This application does not limit this.

[0174] Step 205: For each of the multiple video segments, determine the color temperature of the video segment based on the average color temperature of at least one video frame in the video segment.

[0175] In this embodiment of the application, for each video segment in the initial video, after the terminal calculates the color temperature of each video frame in the video segment, it can calculate the average color temperature of at least one video frame in the video segment, thereby obtaining the color temperature of the video segment.

[0176] It is understandable that factors such as ambient light intensity and the color of the light source will affect the color temperature of video frames. Therefore, for any two video clips, if the multiple video frames in the two video clips were captured by the image acquisition device in the same environment, the color temperatures of the two video clips may be the same or very close. If the multiple video frames in the two video clips were captured by the image acquisition device in different environments, the color temperatures of the two video clips may be different.

[0177] It is understandable that steps 203, 204, and 205 above can be referred to as the color temperature estimation process for the video clip. (Reference) Figure 4 The terminal can estimate the color temperature of each of the n video segments after scene segmentation to determine the color temperature of each of the n video segments. For example, if n is 18, the color temperatures of the 18 video segments can be shown in Table 1. The color temperature distribution of the 18 video segments can be as follows: Figure 5 As shown. Among them, in Figure 5In the distribution diagram shown, the horizontal axis represents the sequence number of each video segment in the initial video, and the vertical axis represents the color temperature.

[0178] Table 1

[0179]

[0180] Refer to Table 1 and Figure 5 It can be seen that the color temperature of any two video clips among the 18 video clips is not exactly the same, but there are video clips with relatively close color temperatures, such as video clip 1, video clip 2, and video clip 3. Therefore, it can be determined that these three video clips were likely captured by the image acquisition device in the same environment.

[0181] Step 206: Based on the color temperature of multiple video clips, determine at least one target video clip from the multiple video clips.

[0182] In this embodiment, the terminal can select at least one target video segment whose color temperature needs adjustment from multiple video segments based on the expected display effect of the processed video segments. The initial video includes a number of target video segments less than or equal to the number of multiple video segments, and the color temperature of each target video segment can be outside a preset color temperature range. That is, video segments whose color temperature is within the color temperature range meet the expected display effect, and the terminal does not need to adjust the color temperature of these video segments. Target video segments outside the color temperature range do not meet the expected display effect, and the terminal needs to further adjust the color temperature of these video segments.

[0183] Understandably, the expected display effect after video clip processing can be determined based on the needs of the application scenario. For example, the expected display effect can be normal tone, cool tone, or warm tone. Normal tone video clips typically have a color temperature of 6000K to 7000K; cool tone video clips typically have a color temperature greater than 7000K (i.e., a higher color temperature); and warm tone video clips typically have a color temperature less than 6000K (i.e., a lower color temperature).

[0184] For example, if the expected display effect of the processed video clip is a normal tone, then the color temperature range can be 6000K to 7000K. The terminal can identify video clips with color temperatures outside this range (i.e., video clips with color temperatures that are too high or too low) as target video clips. (Reference) Figure 4 The terminal can select m target video segments from n video segments by scene selection, where m is an integer not greater than n.

[0185] Step 207: For each target video segment among multiple video segments, determine the color temperature adjustment coefficient of the target video segment based on the color temperature of the target video segment and the color balance correction coefficient of the target video segment.

[0186] In this embodiment, the terminal pre-stores an image processing algorithm for calculating color balance correction coefficients of video frames. For each target video segment, the terminal first uses the image processing algorithm to calculate the color balance correction coefficients of at least one video frame in the target video segment. Then, based on the color balance correction coefficients of the at least one video frame, the color balance correction coefficient of the target video segment is determined. The color temperature adjustment coefficient for each target video segment may include: an R channel adjustment coefficient K. R G-channel adjustment coefficient K G and the B-channel adjustment coefficient K B .

[0187] Optionally, the R-channel adjustment coefficient K R G-channel adjustment coefficient K G and the B-channel adjustment coefficient K B They respectively satisfy:

[0188] K R =CT`×β1×Avg_gain′ R Formula (2)

[0189] K G =CT`×β2×Avg_gain′ G Formula (3)

[0190] K B =CT`×β3×Avg_gain′ B Formula (4)

[0191] β1, β2, and β3 are preset color temperature reference coefficients. The values ​​of these three coefficients can be adjusted based on the expected display effect of the target video segment and the image format of the video frames within that segment. The values ​​of these three reference coefficients can range from 0 to 1 / 2000. CT` represents the color temperature of the target video segment, and its value can range from 0 to 10000K. Avg_gain′ R Avg_gain′ is the color balance correction factor for the target video clip in the R channel. G Avg_gain′ is the color balance correction factor for the target video clip in the G channel. B This represents the color balance correction factor for the target video clip in the B channel. The values ​​of these three color balance correction factors can range from 0 to 2.

[0192] Referring to formulas (2), (3), and (4) above, it can be seen that the color temperature adjustment coefficient (i.e., K) of the target video segment is... R K G and K B The color temperature CT' and the color balance correction coefficient (Avg_gain') of the target video segment are compared with those of the target video segment. R 、Avg_gain′ G and Avg_gain′ R All are positively correlated.

[0193] Optionally, the image processing algorithm pre-stored in the terminal for calculating the color balance correction coefficients of video frames can be a grayscale world algorithm. For each target video segment in multiple video segments, the terminal can use the grayscale world algorithm to calculate the color balance correction coefficients of each video frame in the target video segment in the R channel, G channel, and B channel. Then, the terminal can determine the average value of the color balance correction coefficients of at least one video frame in the target video segment in the R channel as the color balance correction coefficient of the target video segment in the R channel (i.e., Avg_gain′ in the above formula (2)). R The mean value of the color balance correction coefficients of at least one video frame in the target video segment on the G channel is determined as the color balance correction coefficient of the target video segment on the G channel (i.e., Avg_gain′ in the above formula (3)). G The mean value of the color balance correction coefficients of at least one video frame in the target video segment on the B channel is determined as the color balance correction coefficient of the target video segment on the B channel (i.e., Avg_gain′ in the above formula (3)). B ).

[0194] Example, reference Figure 4 The process by which the terminal calculates the color balance correction coefficients for each target video segment can also be called the gain calculation process. Optionally, such as... Figure 4 As shown, the terminal can also perform gain calculations on all n video segments in the initial video.

[0195] Specifically, for each target video segment among multiple video segments, the color balance correction coefficient gain on the R channel for each video frame in that target video segment. R color balance correction factor gain on the G channel B and the color balance correction factor gain on the B channel B They respectively satisfy:

[0196] gain R =K / R avg +a formula (5)

[0197] gainB =K / G avg +b Formula (6)

[0198] gain B =K / B avg +c Formula (7)

[0199] Among them, R avg G is the average of the R pixel values ​​of multiple pixels in a video frame. avg G is the average of the G pixel values ​​of multiple pixels in a video frame, and B is the average of the G pixel values ​​of multiple pixels in a video frame. avg Let B be the mean value of multiple pixels in the video frame, a, b, and c be preset reference deviation values, and K be R. avg G avg and B avg The mean.

[0200] It is understandable that the R pixel value of each pixel in a video frame is the channel value of the R channel in the RGB color space, the G pixel value is the channel value of the G channel in the RGB color space, and the B pixel value is the channel value of the B channel in the RGB color space.

[0201] Referring to formulas (5), (6), and (7) above, in order to enable the grayscale world algorithm to adapt to video frames with different image content, this embodiment adds reference deviation values ​​a, b, and c to the grayscale world algorithm. The value of the reference deviation value can be adjusted according to the image format, image content, etc. of each video frame in the initial video, so that the grayscale world algorithm can be more adapted to the video frames. For example, when multiple video frames in the initial video are HDR images, the value of a can be 20, the value of b can be 10, and the value of c can be 0.

[0202] Step 208: For each video frame in the target video segment, use the color temperature adjustment coefficient of the target video segment to adjust the pixel value of each pixel in the video frame.

[0203] In this embodiment of the application, for each video frame in the target video segment, the terminal can use the R channel adjustment coefficient determined in step 207 above to adjust the R pixel value of each pixel in the video frame, use the G channel adjustment coefficient to adjust the G pixel value of each pixel in the video frame, and use the B channel adjustment coefficient to adjust the B pixel value of each pixel in the video frame.

[0204] It's understandable that adjusting the pixel value of each pixel in a video frame can adjust the color temperature of that frame. This color temperature adjustment can also be called white balance processing.

[0205] Wherein, after the terminal adjusts the pixel value of each pixel in the video frame, the R pixel value R1, G pixel value G1, and B pixel value B1 of that pixel respectively satisfy:

[0206] R1 = K R ×R0 formula (8)

[0207] G1 = K G ×G0 formula (9)

[0208] B1 = K B ×B0 formula(10)

[0209] Where R0 is the R pixel value of the pixel before the terminal adjusts the pixel value of the pixel, G0 is the G pixel value of the pixel before adjustment, and B0 is the B pixel value of the pixel before adjustment. Based on formulas (8) to (10), it can be seen that the terminal can multiply the pixel value of each pixel with the color temperature adjustment coefficient of the corresponding channel, thereby realizing the adjustment of the pixel value of the pixel.

[0210] As described above, when the terminal adjusts the color temperature of at least one video frame in each target video segment, the color temperature adjustment coefficient (i.e., K) used is... R K G and K B The color temperature adjustment coefficients are the same for all target video segments, but different target video segments use different coefficients for their video frames. Since the color temperature adjustment coefficient for each target video segment is determined based on the average color temperature of at least one video frame within that segment and the average color balance correction coefficient, using this coefficient to adjust the color temperature of each video frame within the target video segment ensures a better adjustment effect for that segment, thus ensuring a better display effect for the adjusted target video segment.

[0211] Example, reference Figure 4 For each of the m target video segments, the terminal can adjust the color temperature of the target video segment based on the color temperature of the target video segment determined by color temperature estimation in steps 203 to 205 above, and the color balance correction coefficient of the target video segment determined in step 207 above.

[0212] Step 209: In accordance with the playback order, splice together at least one target video segment after color temperature adjustment, and the video segments other than at least one target video segment, to obtain the target video.

[0213] Example, reference Figure 4 The terminal can splice together m target video segments with adjusted color temperature and n video segments other than the m target video segments in the playback order to obtain the target video.

[0214] It is understood that the order of the steps in the video processing method provided in this application embodiment can be appropriately adjusted, and the steps can be added or removed as needed. For example, step 206 can be deleted as needed; that is, the terminal can adjust the color temperature of multiple video segments in the initial video directly without selecting a target video segment. Alternatively, it can be understood that each video segment in the initial video is a target video segment. Any variations that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application, and therefore will not be elaborated further.

[0215] In summary, this application provides a video processing method in which a terminal can divide an initial video into multiple video segments and calculate the color temperature of each video segment. For each target video segment, the terminal can determine a color temperature adjustment coefficient based on the color temperature and color balance correction coefficient of the target video segment, and then use this color temperature adjustment coefficient to adjust the color temperature of at least one video frame in the target video segment. This ensures a good color temperature adjustment effect for each video segment, thereby ensuring a good display effect for the processed video segment.

[0216] Figure 6 This is a schematic diagram of the structure of a video processing device provided in an embodiment of this application. This video processing device can be deployed in... Figure 1 In the scenario shown, the server 110 or terminal 120 can implement the video processing method provided in the above-described method embodiments, such as... Figure 6 As shown, the device includes:

[0217] The segmentation module 301 is used to divide the multiple video frames included in the initial video into multiple video segments, each video segment including one or more video frames, wherein the multiple video frames are consecutive.

[0218] The first determining module 302 is used to determine the color temperature of a video segment for each of a plurality of video segments based on the color temperature of at least one video frame in the video segment.

[0219] The second determining module 303 is used to determine the color temperature adjustment coefficient of each target video segment in a plurality of video segments based on the color temperature of the target video segment and the color balance correction coefficient of the target video segment. The color temperature adjustment coefficient is positively correlated with both the color temperature and the color balance correction coefficient. The number of target video segments included in the initial video is less than or equal to the number of multiple video segments.

[0220] The processing module 304 is used to adjust the color temperature of each video frame in the target video segment by using the color temperature adjustment coefficient of the target video segment.

[0221] The splicing module 305 is used to splice at least one target video segment after color temperature adjustment, as well as video segments other than the at least one target video segment, in the order of playback to obtain the target video.

[0222] Optionally, the first determining module 302 is configured to: for each of the multiple video segments, perform spatial transformation on each video frame in the video segment; determine the color temperature of the video frame based on the average value of each channel of the multiple pixels included in each video frame after spatial transformation; and determine the color temperature of the video segment based on the average value of the color temperature of at least one video frame in the video segment.

[0223] Optionally, the first determining module 302 is configured to: for each of the multiple video segments, convert each video frame in the video segment from the RGB color space to the HSV color space and the YUV color space respectively.

[0224] For each video frame in the video segment, the color temperature of the video frame is determined based on the average values ​​of the multiple pixels included in the video frame in the Y-channel, U-channel, and V-channel of the YUV color space, and the average value of the multiple pixels included in the HSV color space in the S-channel. The color temperature of the video frame is positively correlated with the average values ​​of the multiple pixels included in the video frame in the Y-channel and U-channel, and negatively correlated with the average values ​​of the multiple pixels included in the video frame in the V-channel and S-channel.

[0225] Optionally, for each of the multiple video segments, the color temperature C7 of each video frame in that video segment satisfies:

[0226]

[0227] Where α is a preset gain coefficient, Y mean U is the mean value on the Y channel. mean V is the mean value on channel U. mean C is the mean value on the V channel. minus For U mean and V mean The absolute value of the difference, S mean This is the mean value on the S channel.

[0228] Optionally, multiple video frames in the initial video are HDR images, and the value of α is 5000.

[0229] Figure 7This is a schematic diagram of another video processing device provided in an embodiment of this application, referred to... Figure 7 The video processing apparatus may further include a third determining module 306, which is used to determine at least one target video segment from the multiple video segments based on the color temperature of the multiple video segments.

[0230] The color temperature of each target video segment is outside the preset color temperature range.

[0231] Optionally, the color temperature range is 6000K to 7000K.

[0232] Optionally, for each target video segment among multiple video segments, the color temperature adjustment coefficient for that target video segment includes: R channel adjustment coefficient K. R G-channel adjustment coefficient K G and the B-channel adjustment coefficient K B ;

[0233] Among them, the R-channel adjustment coefficient K R Satisfy: K R =CT`×β1×Avg_gain′ R ;

[0234] G-channel adjustment coefficient K G Satisfy: K G =CT`×β2×Avg_gain′ G ;

[0235] Channel B adjustment coefficient K B Satisfy: K B =CT`×β3×Avg_gain′ B ;

[0236] Where β1, β2, and β3 are preset color temperature reference coefficients, CT′ is the color temperature of the target video segment, and Avg_gain′ is the color temperature of the target video segment. R Avg_gain′ is the color balance correction factor for the target video clip in the R channel. G Avg_gain′ is the color balance correction factor for the target video clip in the G channel. B The color balance correction factor for the target video clip in the B channel.

[0237] Optionally, the processing module 304 is used for:

[0238] For each video frame in the target video segment, the R-channel adjustment factor K is used. R The R pixel value of each pixel in the video frame is adjusted.

[0239] Using the G-channel adjustment coefficient K GThe G-pixel value of each pixel in the video frame is adjusted.

[0240] Using the B-channel adjustment coefficient K B The B-pixel value of each pixel in the video frame is adjusted.

[0241] Optionally, Avg_gain′ R This is the mean of the color balance correction coefficients on the R channel for at least one video frame in the target video segment. (Avg_gain′) G Avg_gain′ is the mean of the color balance correction coefficients on the G channel for at least one video frame in the target video segment. B The mean of the color balance correction coefficients on the B channel for at least one video frame in the target video segment.

[0242] Optionally, the color balance correction factor gain on the R channel for each video frame. R Satisfy: gain R =K / R avg +a;

[0243] The color balance correction factor gain for each video frame on the G channel. B Satisfy: gain B =K / G avg +b;

[0244] The color balance correction factor gain for each video frame in the B channel. B Satisfy: gain B =K / B avg +c;

[0245] Among them, R avg G is the average of the R pixel values ​​of multiple pixels in a video frame. avg G is the average of the G pixel values ​​of multiple pixels in a video frame, and B is the average of the G pixel values ​​of multiple pixels in a video frame. avg Let B be the mean value of multiple pixels in the video frame, a, b, and c be preset reference deviation values, and K be R. avg G avg and B avg The mean.

[0246] Optionally, multiple video frames in the initial video are HDR images, with a value of 20, b value of 10, and c value of 0.

[0247] Optionally, refer to Figure 7 The partitioning module 301 is used for:

[0248] Calculate the similarity between each video frame and the previous video frame in the order in which the initial video is played.

[0249] Based on the calculated similarity between each pair of adjacent video frames, the initial video is divided into multiple video segments.

[0250] Optionally, each video frame in the initial video includes multiple image blocks, and the segmentation module 301 is used for:

[0251] In each video frame of the initial video, at least one target image patch is identified, and the number of at least one target image patch is less than the number of multiple image patches.

[0252] Based on the playback order of the multiple video frames included in the initial video, the similarity between at least one target image patch in each video frame and at least one target image patch in the previous video frame is calculated sequentially.

[0253] In each video frame, the position of at least one target image block in the video frame is the same as the position of at least one target image block in the previous video frame.

[0254] Optionally, refer to Figure 7 The video processing device may further include: a dimensionality reduction module 307, which is used to perform dimensionality reduction processing on each initial video frame in the initial video before the segmentation module 301 calculates the similarity between each video frame and the previous video frame in the order of playback of the multiple video frames included in the initial video, so as to obtain multiple video frames.

[0255] Optionally, the segmentation module 301 is used to: determine the structural similarity between a video frame and a previous video frame based on the mean of the image data of each video frame and the mean of the image data of the previous video frame, the standard deviation of the image data of the video frame and the standard deviation of the image data of the previous video frame, and the covariance of the image data of the video frame and the image data of the previous video frame; and determine the similarity between the video frames based on the structural similarity between the video frames and the previous video frames.

[0256] In summary, this application provides a video processing apparatus capable of dividing an initial video into multiple video segments and calculating the color temperature of each video segment. For each target video segment, the apparatus determines a color temperature adjustment coefficient based on the target video segment's color temperature and color balance correction coefficient, and then adjusts the color temperature of at least one video frame within that target video segment using this adjustment coefficient. This ensures a good color temperature adjustment effect for each video segment, thereby ensuring a good display effect for the processed video.

[0257] It is understood that the video processing apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0258] Furthermore, the video processing apparatus and video processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0259] This application also provides a display device, which can be a computer device, for example, a computer device. Figure 1 The scenario shown includes a server 110 or a terminal 120. Furthermore, the display device may include the video processing apparatus provided in the above-described device embodiments.

[0260] like Figure 8 As shown, the display device may include a display screen 401, a processor 402, and a memory 403. The memory 403 stores instructions that are loaded and executed by the processor 402 to implement the video processing method provided in the above-described method embodiments (e.g., Figure 2 and Figure 3 (The method shown). The display screen 401 can then play the processed video.

[0261] Embodiments of this application also provide a computer-readable storage medium storing instructions that are loaded and executed by a processor to implement the video processing method provided in the above-described method embodiments (e.g., Figure 2 and Figure 3 (The method shown).

[0262] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions loaded and executed by a processor to implement the video processing method provided in the above method embodiments (e.g., Figure 2 and Figure 3 (The method shown).

[0263] It is understood that in this application, the term "at least one" means one or more, and "multiple" means two or more.

[0264] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "n," nor is there any limitation on the quantity or execution order.

[0265] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0266] The above are merely exemplary embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A video processing method, characterized in that, The method includes: The initial video is divided into multiple video segments, each of which includes one or more video frames, wherein the multiple video frames are consecutive. For each of the plurality of video segments, the color temperature of the video segment is determined based on the color temperature of at least one video frame in the video segment; For each target video segment among the plurality of video segments, a color temperature adjustment coefficient for the target video segment is determined based on the color temperature of the target video segment and the color balance correction coefficient of the target video segment. The color temperature adjustment coefficient is positively correlated with both the color temperature and the color balance correction coefficient. The number of target video segments included in the initial video is less than or equal to the number of the plurality of video segments. For each target video segment, the color temperature of each video frame in the target video segment is adjusted using the color temperature adjustment coefficient of the target video segment; The target video is obtained by splicing together the at least one target video segment with adjusted color temperature and other video segments in the playback order.

2. The method according to claim 1, characterized in that, Determining the color temperature of a video segment based on the color temperature of at least one video frame in the video segment includes: Perform spatial transformation on each video frame in the video segment; The color temperature of the video frame is determined based on the average value of each channel of the multiple pixels included in each video frame after spatial transformation. The color temperature of the video segment is determined based on the average color temperature of at least one video frame in the video segment.

3. The method according to claim 2, characterized in that, The spatial transformation of each video frame in the video segment includes: Each video frame in the video segment is converted from the RGB color space to the HSV color space and the YUV color space, respectively. Determining the color temperature of a video frame based on the average value of each channel of multiple pixels included in each video frame after spatial transformation includes: For each video frame in the video segment, the color temperature of the video frame is determined based on the average values ​​of the multiple pixels included in the video frame in the Y-channel, U-channel, and V-channel of the YUV color space, and the average value of the multiple pixels included in the HSV color space in the S-channel. The color temperature of the video frame is positively correlated with the average values ​​of the multiple pixels included in the video frame in the Y-channel and U-channel, and negatively correlated with the average values ​​of the multiple pixels included in the video frame in the V-channel and S-channel.

4. The method according to claim 3, characterized in that, The color temperature of the video frame CT satisfy: in, The preset gain coefficient, The mean value on the Y channel. The mean value on the U channel. The mean value over the V channel. For the and stated The absolute value of the difference The mean value on the S channel.

5. The method according to any one of claims 1 to 4, characterized in that, Before determining the color temperature adjustment coefficient of the target video segment, the method further includes: Based on the color temperature of the plurality of video segments, at least one target video segment is determined from the plurality of video segments; The color temperature of each target video segment is outside the preset color temperature range.

6. The method according to any one of claims 1 to 4, characterized in that, The color temperature adjustment coefficient of the target video segment includes: R channel adjustment coefficient. G-channel adjustment coefficient and B-channel adjustment coefficient ; Wherein, the R-channel adjustment coefficient satisfy: The G-channel adjustment coefficient satisfy: The B-channel adjustment coefficient satisfy: in, , and All are preset color temperature reference coefficients. The color temperature of the target video segment. The color balance correction coefficient for the target video segment in the R channel. The color balance correction coefficient for the target video segment in the G channel. The color balance correction coefficient for the target video segment in the B channel.

7. The method according to claim 6, characterized in that, The step of adjusting the color temperature of each video frame in the target video segment using the color temperature adjustment coefficient of the target video segment includes: For each video frame in the target video segment, the R-channel adjustment coefficient is used. The R pixel value of each pixel in the video frame is adjusted. Using the G-channel adjustment coefficient The G-pixel value of each pixel in the video frame is adjusted. Using the B-channel adjustment coefficient The B-pixel value of each pixel in the video frame is adjusted.

8. The method according to claim 6, characterized in that, The The mean value of the color balance correction coefficients on the R channel for at least one video frame in the target video segment; The The mean value of the color balance correction coefficients on the G channel for at least one video frame in the target video segment; The It is the mean of the color balance correction coefficients on the B channel for at least one video frame in the target video segment.

9. The method according to claim 8, characterized in that, Color balance correction factor for each video frame on the R channel satisfy: Color balance correction factor for each video frame on the G channel satisfy: Color balance correction factor for each video frame on the B channel satisfy: in, The mean R-pixel value of multiple pixels in the video frame. The average G-pixel value of multiple pixels in the video frame. The average of the B-pixel values ​​of multiple pixels in the video frame. , and The preset reference deviation value, K For the The and stated The mean.

10. The method according to any one of claims 1 to 4, characterized in that, The process of dividing the initial video, which comprises multiple video frames, into multiple video segments includes: According to the playback order of the multiple video frames included in the initial video, the similarity between each video frame and the previous video frame is calculated sequentially. Based on the calculated similarity between each pair of adjacent video frames, the initial video is divided into multiple video segments.

11. The method according to claim 10, characterized in that, Each video frame in the initial video includes multiple image blocks; the step of calculating the similarity between each video frame and the previous video frame in the playback order of the multiple video frames included in the initial video includes: In each video frame of the initial video, at least one target image patch is identified, and the number of the at least one target image patch is less than the number of the plurality of image patches; According to the playback order of the multiple video frames included in the initial video, the similarity between at least one target image block in each video frame and at least one target image block in the previous video frame is calculated sequentially. In each of the video frames, at least one target image block is positioned in the same location as at least one target image block in the previous video frame.

12. The method according to claim 10, characterized in that, Before calculating the similarity between each video frame and the previous video frame in the order of playback of the multiple video frames included in the initial video, the method further includes: Each initial video frame in the initial video is subjected to dimensionality reduction processing to obtain the plurality of video frames.

13. The method according to claim 10, characterized in that, The step of sequentially calculating the similarity between each video frame and the previous video frame includes: For each of the plurality of video frames, the structural similarity between the video frame and the previous video frame is determined based on the mean of the image data of the video frame and the mean of the image data of the previous video frame, the standard deviation of the image data of the video frame and the standard deviation of the image data of the previous video frame, and the covariance of the image data of the video frame and the image data of the previous video frame. The similarity between the video frame and the previous video frame is determined based on the structural similarity between the video frame and the previous video frame.

14. A video processing apparatus, characterized in that, The device includes: A segmentation module is used to divide the initial video, which includes multiple video frames, into multiple video segments, each of which includes one or more video frames, wherein the multiple video frames are consecutive. The first determining module is used to determine the color temperature of each of the plurality of video segments based on the color temperature of at least one video frame in the video segment; The second determining module is used to determine the color temperature adjustment coefficient of each target video segment among the plurality of video segments, based on the color temperature of the target video segment and the color balance correction coefficient of the target video segment, wherein the color temperature adjustment coefficient is positively correlated with both the color temperature and the color balance correction coefficient, wherein the number of target video segments included in the initial video is less than or equal to the number of the plurality of video segments; The processing module is used to adjust the color temperature of each video frame in the target video segment by using the color temperature adjustment coefficient of the target video segment; The splicing module is used to splice the at least one target video segment after color temperature adjustment, as well as other video segments besides the at least one target video segment, in the order of playback to obtain the target video.

15. A display device, characterized in that, The display device includes: a display screen, a processor, and a memory, wherein the memory stores instructions that are loaded and executed by the processor to implement the video processing method as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, The storage medium stores instructions that are loaded and executed by a processor to implement the video processing method as described in any one of claims 1 to 13.

17. A computer program product, characterized in that, The computer program product includes computer instructions that are loaded and executed by a processor to implement the video processing method as described in any one of claims 1 to 13.

Images