Video adjustment method and device, electronic equipment and readable storage medium

Abstract

The application discloses a video adjusting method and device, electronic equipment and readable storage medium, and relates to the technical field of video, which comprises the following steps: collecting first video content through an image collection module of a video collection end in the case that there is a light source emitting light in the surrounding environment of the video collection end; collecting spectral data of the light source occurring frequency flicker through a color temperature sensor of the video collection end; and adjusting the first video content based on the spectral data to obtain second video content reducing the frequency flicker interference of the light source. The first video content collected is adjusted through the spectral data, and then the frequency flicker interference of the light source is reduced, so that the color restoration degree of the obtained second video content is improved.

Description

Technical Field

[0001] This application relates to the field of video technology, and more specifically, to a video adjustment method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] Currently, with the development of electronic information technology, video content can be captured using devices. However, during the video capture process, there may be light sources around the device, causing flickering interference in the captured video, resulting in poor video quality and reduced user experience. Summary of the Invention

[0003] This application discloses a video adjustment method, apparatus, electronic device, and readable storage medium.

[0004] In a first aspect, embodiments of this application provide a video adjustment method, comprising: acquiring first video content through an image acquisition module of the video acquisition terminal when there is a light source in the environment surrounding the video acquisition terminal; acquiring spectral data of the light source flickering through a color temperature sensor of the video acquisition terminal; and adjusting the first video content based on the spectral data to obtain second video content with reduced flicker interference from the light source.

[0005] Secondly, embodiments of this application also provide a video adjustment device, including: a first acquisition unit, a second acquisition unit, and an adjustment unit. The first acquisition unit is used to acquire first video content through the image acquisition module of the video acquisition terminal when a light source is present in the environment surrounding the video acquisition terminal; the second acquisition unit is used to acquire spectral data of the flickering light source through the color temperature sensor of the video acquisition terminal; and the adjustment unit is used to adjust the first video content based on the spectral data to obtain second video content with reduced flickering interference from the light source.

[0006] Thirdly, embodiments of this application also provide an electronic device, including: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the one or more processors, and the one or more application programs are configured to perform the method described in the first aspect.

[0007] Fourthly, embodiments of this application also provide a computer-readable storage medium storing program code that can be invoked by a processor to execute the method described in the first aspect above.

[0008] The video adjustment method, apparatus, electronic device, and readable storage medium provided in this application first acquire first video content through the image acquisition module of the video acquisition terminal in the presence of a light source in the environment surrounding the video acquisition terminal; then, the spectral data of the flickering light source is acquired through the color temperature sensor of the video acquisition terminal; thereby, the first video content is adjusted based on the spectral data to obtain second video content with reduced flickering interference from the light source. If video content is directly acquired in the presence of a light source in the environment, the obtained video content will be subject to significant flickering interference from the light source. However, in the solution of this application, the acquired first video content can be adjusted through spectral data, thereby reducing flickering interference from the light source and improving the color reproduction of the obtained second video content.

[0009] Other features and advantages of the embodiments of this application will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the embodiments of this application. The objects and other advantages of the embodiments of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 The following diagram illustrates an application scenario of the video adjustment method provided in this application embodiment;

[0012] Figure 2 A flowchart of the video adjustment method provided in an embodiment of this application is shown;

[0013] Figure 3 A flowchart of a video adjustment method according to another embodiment of this application is shown;

[0014] Figure 4 A flowchart of a video adjustment method according to another embodiment of this application is shown;

[0015] Figure 5 A flowchart of a video adjustment method provided in another embodiment of this application is shown;

[0016] Figure 6 A structural block diagram of the video adjustment device provided in an embodiment of this application is shown;

[0017] Figure 7A structural block diagram of the electronic device provided in an embodiment of this application is shown;

[0018] Figure 8 A structural block diagram of a computer-readable storage medium provided in an embodiment of this application is shown. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. The components of the embodiments of the present application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without inventive effort are within the scope of protection of the present application.

[0020] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0021] Currently, with the development of electronic information technology, video content can be captured using devices. However, during the video capture process, there may be light sources around the device, causing flickering interference in the captured video, resulting in poor video quality and reduced user experience. How to reduce the degree of flickering interference in the captured video is a problem that urgently needs to be solved.

[0022] Currently, traditional white balance algorithms are generally used to capture video content. However, the inventors discovered in their research that traditional white balance algorithms suffer from severe flicker interference when encountering ambient light sources, leading to color fluctuations in the captured video content and reducing its color fidelity.

[0023] Therefore, in order to solve or partially solve the above problems, this application provides a video adjustment method, apparatus, electronic device, and readable storage medium.

[0024] Please see Figure 1 , Figure 1The illustration shows an application scenario diagram of the video adjustment method provided in the embodiments of this application, namely video adjustment scenario 100, which includes a video acquisition terminal 110 and a light source 120.

[0025] exist Figure 1 The video capture terminal 110 shown is a smartphone, which users can use to capture video content. In some embodiments, the video capture terminal 110 may include an image acquisition module and a color temperature sensor. The image acquisition module can capture video content, and the color temperature sensor can acquire spectral data. The acquired video content is then adjusted using the spectral data to obtain adjusted video content. Detailed descriptions can be found in subsequent embodiments.

[0026] Please see Figure 2 , Figure 2 A flowchart illustrating a video adjustment method provided in an embodiment of this application is shown. This video adjustment method can be applied to... Figure 1 In the illustrated text generation scenario, the video capture terminal can specifically be configured with its processor as the execution entity for the video adjustment method. This video adjustment method may include steps S110 to S130.

[0027] Step S110: When there is a light source in the environment around the video acquisition terminal, the first video content is acquired by the image acquisition module of the video acquisition terminal.

[0028] The video capture terminal can be an electronic device with video capture capabilities, such as a smartphone, tablet, or laptop equipped with a camera. The surrounding environment of the video capture terminal characterizes the environment surrounding its location. For example, the room where the video capture terminal is located can be considered its surrounding environment; alternatively, a specified area around the location of the video capture terminal can be considered its surrounding environment. Specifically, for example, a spatial area with a specified radius, such as 2 meters or 3 meters, can be considered its surrounding environment, with the location of the video capture terminal as the origin. It should be noted that this embodiment does not specifically limit the surrounding environment of the video capture terminal.

[0029] It is understandable that there may be light sources in the surrounding environment, which can negatively impact the subsequently captured video content. Therefore, if there are light sources in the surrounding environment of the video capture terminal, the first video content can be captured through the image acquisition module of the video capture terminal.

[0030] The image acquisition module can be a camera, such as a red-green-blue (Red-Green-Blue) camera. Therefore, the first video content can include a Red-Green-Blue video stream.

[0031] Optionally, when acquiring the first video content through the image acquisition module, a high dynamic range (HDR) exposure strategy can be adopted, which can preserve more color details in the subsequent second video content.

[0032] Step S120: Collect spectral data of the flickering light source using the color temperature sensor of the video acquisition terminal.

[0033] Additionally, a color temperature sensor can be configured in the video acquisition unit to collect spectral data of the flickering light source. For example, the color temperature sensor can be equipped with a spectral sensor, which can capture the environmental spectral distribution, thus obtaining the spectral distribution within the environment. Since there are luminous light sources in the surrounding environment, the spectral distribution captured by the spectral sensor can be approximated as spectral data.

[0034] For example, this spectral data can be obtained from the International Commission on Illumination (ICIDI). The CIE XYZ chromaticity values ​​are used to characterize it.

[0035] Step S130: Adjust the first video content based on the spectral data to obtain a second video content with reduced flicker interference from the light source.

[0036] Furthermore, after acquiring the first video content and spectral data, the first video content can be adjusted based on the spectral data to obtain a second video content with reduced flicker interference from the light source.

[0037] In some implementations, the frequency of light source flickering in the surrounding environment can be obtained through spectral data analysis, and then the first video can be adjusted according to the frequency to reduce the flickering interference of the light source.

[0038] For example, after obtaining the frequency of the flickering of the light source in the surrounding environment, the flicker period and multiple luminous intensities within the flicker period can be obtained based on the frequency. Then, the frame images in the first video corresponding to each luminous intensity within each flicker period can be determined. Finally, the second video content can be obtained by combining the luminous intensities with the individual frame images. Detailed descriptions can be found in subsequent embodiments.

[0039] It is understood that the obtained second video content can be transmitted to electronic devices other than the video capture terminal, thereby enabling these devices to receive and play the second video content. Therefore, in some implementations, after obtaining the second video content, it can be encapsulated and encoded. Optionally, metadata information matching the second video content can also be obtained, and then the second video content and metadata information can be encapsulated together to obtain encoded data, which is then transmitted to electronic devices other than the video capture terminal.

[0040] Among them, electronic devices other than video capture terminals can be video playback terminals, such as smartphones, smart tablets, in-vehicle systems, laptops, etc.

[0041] Therefore, after obtaining the encapsulated data, the video playback terminal can decode the encapsulated data to obtain the second video content.

[0042] It should be noted that, due to the variety of video playback device models and display screen color parameters, playing the second video content directly on the video playback device without color adjustment may result in different colors of the second video content seen on different playback devices, leading to errors compared to actual colors. Therefore, in some implementations, after obtaining the encapsulated data, the video playback device can adjust the colors of the second video content to obtain the target video content, and then play the target video content. Detailed descriptions can be found in subsequent embodiments.

[0043] For example, the aforementioned video capture terminal and video playback terminal can be applied to online live streaming scenarios. Specifically, in an online live streaming scenario, the broadcaster captures second video content through the video capture terminal and encapsulates it into encapsulated data. This encapsulated data is then sent to the video playback terminals of various users via the network. Users then decode the encapsulated data on their video playback terminals, adjust the colors of the second video content to obtain the target video content, and subsequently play the target video content.

[0044] The video adjustment method provided in this application first acquires first video content through the image acquisition module of the video acquisition terminal in the presence of a light source in the environment surrounding the video acquisition terminal; then, it acquires spectral data of the flicker caused by the light source through the color temperature sensor of the video acquisition terminal; and adjusts the first video content based on the spectral data to obtain second video content with reduced flicker interference from the light source. If video content is directly acquired in the presence of a light source in the environment, the resulting video content will be subject to significant flicker interference. However, in the solution of this application, the acquired first video content can be adjusted through spectral data, thereby reducing flicker interference from the light source and improving the color reproduction of the obtained second video content.

[0045] Please see Figure 3 , Figure 3 A flowchart illustrating a video adjustment method provided in an embodiment of this application is shown. This video adjustment method can be applied to... Figure 1 In the illustrated text generation scenario, the video capture terminal can specifically be configured with its processor as the execution entity for the video adjustment method. This video adjustment method may include steps S210 to S260.

[0046] Step S210: When there is a light source in the environment around the video acquisition terminal, the first video content is acquired through the image acquisition module of the video acquisition terminal.

[0047] Step S220: Collect spectral data of the flickering light source using the color temperature sensor of the video acquisition terminal.

[0048] Steps S210 and S220 have been described in detail in the foregoing embodiments and will not be repeated here.

[0049] Step S230: Determine the frequency at which the light source flickers based on the spectral data.

[0050] Step S240: Determine multiple flicker cycles corresponding to the first video content based on the frequency, and determine multiple luminous intensities of the light source in each flicker cycle.

[0051] Step S250: Determine the frame image in the first video content corresponding to the light emission intensity in each flicker cycle.

[0052] Step S260: Based on each luminous intensity within the stroboscopic cycle and the corresponding frame image, fuse multiple frame images corresponding to the stroboscopic cycle to obtain the second video content.

[0053] After acquiring the first video content and spectral data, the flicker frequency of the light source can be determined based on the spectral data. For example, if the light source is a light-emitting diode (LED) lamp with a frequency of 100Hz, then the flicker frequency of the light source can be determined to be 100Hz. This frequency can be obtained directly by analyzing the spectral data.

[0054] Additionally, the flicker period of the light source can also be determined from the frequency. Continuing with the example of a 100Hz frequency, the flicker period is the reciprocal of the frequency, which is 10ms.

[0055] It's understandable that the light intensity emitted by the light source changes within a single flicker cycle, for example, the intensity increases from weak to strong and then decreases again. Furthermore, a flicker cycle is generally short, while the first video content is typically longer than a flicker cycle. Therefore, the first video can generally correspond to multiple flicker cycles.

[0056] Therefore, multiple flicker cycles corresponding to the first video content can be determined based on the frequency, and multiple luminous intensities of the light source can be determined in each flicker cycle. For example, the specific time position within a flicker cycle can be characterized by the phase. For instance, for a flicker cycle lasting 10ms, a phase of 0° corresponds to the 0ms moment in the cycle frequency, i.e., the start moment of the cycle frequency; a phase of 180° corresponds to the 5ms moment in the cycle frequency; and a phase of 360° corresponds to the 10ms moment in the cycle frequency, i.e., the end moment of the cycle frequency. Furthermore, the luminous intensity corresponding to the light source at a phase of 180° can be the peak intensity; while the luminous intensities corresponding to phases of 0° and 360° can be the valley intensities.

[0057] Optionally, in the embodiments of this application, the luminous intensities corresponding to phase 0°, phase 120° and phase 240° in the periodic frequency can be determined respectively as multiple luminous intensities corresponding to the periodic frequency.

[0058] Furthermore, a frame image from the first video content corresponding to the luminous intensity in each flicker cycle can be determined. For example, images corresponding to the first video content at phase 0°, phase 120°, and phase 240° in each flicker cycle can be determined as frame images from the first video content corresponding to the luminous intensity in each flicker cycle.

[0059] Therefore, for each flicker cycle, multiple frame images and the corresponding luminous intensity of the frame image can be obtained. Then, based on each luminous intensity within the flicker cycle and the corresponding frame image, the multiple frame images corresponding to the flicker cycle can be fused to obtain the second video content.

[0060] It should be noted that frame image fusion does not mean fusing all the frame images corresponding to all frequency periods together. Instead, it means fusing multiple frame images corresponding to each frequency period to obtain a fused frame image for each frequency period. The fused multiple frame images are then used to construct the second video content.

[0061] Specifically, step S260 may also include steps S261 to S263.

[0062] Step S261: Determine the weight of each frame image based on the luminous intensity corresponding to each frame image, wherein the weight is negatively correlated with the luminous intensity.

[0063] Step S262: Based on the weight of each frame image corresponding to the strobe cycle, adjust the specified parameters of each frame image within the strobe cycle.

[0064] Step S263: Fuse the frames within the strobe cycle after adjusting the specified parameters to obtain the second video content.

[0065] In some implementations, after obtaining multiple frame images and their corresponding luminous intensities through the aforementioned steps, the weight of each frame image can be determined based on its luminous intensity. The weight is negatively correlated with the luminous intensity. That is, a higher weight value can be assigned to frame images with stronger luminous intensities, while a lower weight value can be assigned to frame images with lower luminous intensities.

[0066] Therefore, based on the weight of each frame image corresponding to the strobe cycle, the specified parameters of each frame image within the strobe cycle can be adjusted; then, the frame images within the strobe cycle after adjusting the specified parameters are fused together to obtain the second video content.

[0067] The specified parameters can include at least one of brightness and color temperature. That is, the stronger the luminous intensity, the greater the weight of the corresponding frame image, and consequently, the greater the degree of adjustment to the specified parameters. Since stronger luminous intensity has a greater negative impact on the specified parameters of the corresponding frame image, this weighting can minimize the negative impact of ambient light on the specified parameters of the frame image, thereby resulting in better color and brightness in the obtained second video content.

[0068] Optionally, the fused frame images corresponding to each strobe cycle obtained by fusing the frames after adjusting the specified parameters within the strobe cycle can be stitched together using a multi-frame synthesis algorithm based on temporal analysis to obtain the second video content.

[0069] In the video adjustment method provided in this application embodiment, the frequency of flickering of the light source is determined based on the spectral data; then, multiple flicker periods corresponding to the first video content are determined based on the frequency, and multiple luminous intensities of the light source are determined in each flicker period; furthermore, frame images in the first video content corresponding to the luminous intensities in each flicker period are determined; thereby, based on each luminous intensity within the flicker period and the frame image corresponding to that luminous intensity, the multiple frame images corresponding to that flicker period are fused to obtain the second video content. By using dual-sensor (camera and color temperature sensor) fusion and a multi-frame synthesis algorithm, interference caused by the flickering of the light source is eliminated in real time, significantly improving the color reproduction accuracy of the second video content.

[0070] Please see Figure 4 , Figure 4 A flowchart illustrating a video adjustment method provided in an embodiment of this application is shown. This video adjustment method can be applied to... Figure 1 The video capture terminal in the illustrated text generation scenario can specifically be the processor of the video capture terminal as the execution entity for performing the video adjustment method. This video adjustment method may include steps S310 to S360.

[0071] Step S310: When there is a light source in the environment around the video acquisition terminal, the first video content is acquired by the image acquisition module of the video acquisition terminal.

[0072] Step S320: Collect spectral data of the flickering light source using the color temperature sensor of the video acquisition terminal.

[0073] Step S330: Adjust the first video content based on the spectral data to obtain a second video content with reduced flicker interference from the light source.

[0074] Steps S310 to S330 have been described in detail in the foregoing embodiments and will not be repeated here.

[0075] Step S340: Obtain metadata information matching the second video content, wherein the metadata information includes the original color gamut information corresponding to the image acquisition module, the reference white balance information corresponding to the second video content, and the timestamp corresponding to the second video content.

[0076] As described above, after obtaining the second video content, it can be further sent to the video playback terminal, so that the video playback terminal can end and play the second video content.

[0077] In some implementations, to ensure that the color and brightness parameters of the second video content can be reproduced during subsequent playback on a video playback device, metadata information matching the second video content can also be obtained. This metadata information may include the original color gamut information corresponding to the image acquisition module, the reference white balance information corresponding to the second video content, and the timestamp corresponding to the second video content.

[0078] It is understandable that color gamut information is used to characterize the color gamut information that a hardware device can support for display, and is generally determined by the hardware device. For example, color gamut information may include the color gamut range, color gamut representation method, etc. Therefore, this raw color gamut information is used to characterize the color gamut information of the image acquisition module.

[0079] Reference white balance information is used to characterize the white balance information corresponding to the second video content. Specifically, the white balance information ensures that relatively accurate colors can be obtained when capturing video content under different lighting conditions.

[0080] The timestamp corresponds to the second video content and is used to align the metadata with the second video content, ensuring that the second video content is synchronized with the metadata.

[0081] Specifically, step S340 may include steps S341 to S345.

[0082] Step S341: Generate initial white balance information based on the spectral data.

[0083] Step S342: Analyze the second video content based on the color gamut recognition model to determine the target color gamut range corresponding to the target object in the second video content.

[0084] Step S343: Adjust the initial white balance information based on the target color gamut range to obtain reference white balance information.

[0085] Step S344: Obtain the original color gamut information corresponding to the image acquisition module and the timestamp corresponding to the second video content.

[0086] Step S345: Use the reference white balance information, the original color gamut information corresponding to the image acquisition module, and the timestamp corresponding to the second video content as metadata information.

[0087] To obtain reference white balance information, initial white balance information can first be generated based on spectral data.

[0088] For example, the initial white balance information can be represented in the form of a white balance matrix, such as a 3x3 white balance matrix.

[0089] Then, the second video content can be analyzed to obtain the target color gamut range corresponding to the target object in the second video content. The target object can be used to represent the main object in the second video content. For example, if the second video content is a video of a person, then the target object in the second video content can be a person; or if the second video content is a video of an object, then the target object in the second video content can be an object.

[0090] The target color gamut range corresponding to the target object can be used to characterize the color gamut range corresponding to the target area of ​​the target object. Specifically, the target area of ​​the target object can be used to characterize the area where the target object is of highest importance in the second video content. For example, if the target object is a person, the target area can be the area where the person's skin is located, and the target color gamut range can then characterize the color gamut range corresponding to the person's skin. Similarly, if the target object is an object, the target area can be the area where the outer surface of the object is located, and the target color gamut range can then characterize the color gamut range corresponding to the outer surface of the object. It should be noted that the target object and its corresponding target color gamut range can be identified using a color gamut recognition model, and can be flexibly changed according to the second video content. The specific content shown above is merely an example and does not constitute a limitation on the embodiments of this application.

[0091] The color gamut recognition model can be a pre-trained model, which can be used to analyze the second video content to identify the target object in the second video content and the target color gamut range corresponding to the target object.

[0092] Furthermore, after obtaining the target color gamut range, the initial white balance information can be adjusted based on the target color gamut range to obtain reference white balance information. In other words, the adjusted reference white balance information can prioritize ensuring the color accuracy of the target area of ​​the target object. Therefore, when the target area of ​​the target object is used to characterize the area of ​​highest importance in the second video content, the reference white balance information can ensure the color accuracy of the area of ​​highest importance in the second video content.

[0093] Similar to the initial white balance information, the reference white balance information can also be represented in the form of a white balance matrix, for example, by a 3x3 white balance matrix.

[0094] Furthermore, the original color gamut information corresponding to the image acquisition module and the timestamp corresponding to the second video content can also be obtained. For example, the original color gamut information may include parameter values ​​for ultra-high-definition television system program production and international exchange (BT.2020).

[0095] For example, the original color gamut information can be directly read from the configuration file recorded at the video capture terminal.

[0096] For example, the presentation time stamp (PTS) of each frame in the second video content can be configured as the timestamp of the second video content.

[0097] Furthermore, after obtaining the reference white balance information, the original color gamut information, and the timestamp corresponding to the second video content, the reference white balance information, the original color gamut information corresponding to the image acquisition module, and the timestamp corresponding to the second video content can be used as metadata information.

[0098] Optionally, after obtaining the reference white balance information, the original color gamut information can be adjusted based on the reference white balance information to obtain adjusted original color gamut information, thereby avoiding oversaturation or distortion in subsequent processes. The metadata information can include the adjusted original color gamut information.

[0099] It should be noted that the timestamps corresponding to the second video content are actually used to align each frame image in the second video content with the original color gamut information and reference white balance information.

[0100] Step S350: Encode and encapsulate the second video content and the metadata information to obtain encoded data, and transmit the encoded data to the video playback terminal.

[0101] After obtaining the second video content and metadata information through the aforementioned steps, the second video content and the metadata information can be encoded and encapsulated to obtain encoded data, and the encoded data can be transmitted to the video playback terminal. In some implementations, step S350 may include steps S351 to S353.

[0102] Step S351: Convert the second video content into video stream data.

[0103] Step S352: Embed the metadata information into the supplementary enhancement information layer of the video stream data.

[0104] Step S353: Encode and encapsulate the video stream data with embedded metadata information to obtain encoded data, and transmit the encoded data to the video playback terminal.

[0105] First, the acquired second video content can be converted into video stream data, which can then be embedded with information. Specifically, the metadata information obtained through the aforementioned steps can be embedded into the Supplemental Enhancement Information (SEI) layer of the video stream data.

[0106] The supplementary enhancement layer can be used to add extra information or custom data to the video stream without affecting the video stream's data. Furthermore, the supplementary enhancement layer facilitates timestamp alignment.

[0107] Furthermore, the video stream data embedding the metadata information is encoded and encapsulated to obtain encoded data, and the encoded data is transmitted to the video playback terminal. In other words, what is transmitted to the video playback terminal is the encapsulated encoded data.

[0108] For example, encoding can be performed using the High Efficiency Video Coding (HEVC) Main 10 profile. Optionally, degree subsampling and quantization rounding can be disabled to further improve the accuracy of color in the video content.

[0109] Optionally, during the encoding process, mainstream encoding formats can be used, such as 4:4:4 chroma sampling, to retain more color information.

[0110] It should be noted that during the encoding process, timestamps can also be used to ensure that each frame in the second video content is aligned with the original color gamut information and the reference white balance information.

[0111] Optionally, lightweight JavaScript Object Notation (JSON) can be used to encapsulate the encoded content to obtain encoded data. As can be seen, in this embodiment, the encapsulated encoded data possesses the characteristics of lightweight data and has high compatibility, balancing compatibility and bandwidth efficiency.

[0112] The video capture terminal can establish a communication connection with the video playback terminal, allowing the video capture terminal to directly send the encoded data to the video playback terminal. In addition, both the video capture terminal and the video playback terminal can establish a communication connection with the cloud server, allowing the video capture terminal to send the encoded data to the cloud server, which in turn sends it to each video playback terminal.

[0113] Step S360: Control the video playback terminal to play target video content on the display screen of the video playback terminal based on the acquired encoded data, wherein the target video content includes the second video content that has been color-adjusted to match the display screen.

[0114] Thus, through the aforementioned steps, the video playback terminal can obtain the encoded data. To play the content encapsulated within this encoded data, the video playback terminal needs to decode it. In the embodiments provided in this application, the video playback terminal can be controlled to play target video content on its display screen based on the obtained encoded data. As described above, the model of the video playback terminal and the color parameters of the display screen can vary greatly. Therefore, after obtaining the encapsulated data, the video playback terminal can also adjust the colors of the second video content to obtain the target video content, and then play the target video content. In other words, the target video content includes the second video content that has been color-adjusted to match the display screen.

[0115] It should be noted that the control of the video playback terminal can be implemented by the video capture terminal, for example, the processing module of the video capture terminal can directly control the video playback terminal; or the control of the video playback terminal can be implemented by the cloud server, for example, after the cloud server detects that the encoded data has been sent to the video playback terminal, it can directly control the video playback terminal to play the target video content.

[0116] In some implementations, step S360 may include steps S361 to S365.

[0117] Step S361: Decode the encoded data to obtain the second video content, original color gamut information, reference white balance information, and timestamp.

[0118] Step S362: Obtain the terminal color gamut information corresponding to the display screen.

[0119] Step S363: Obtain a target lookup table for mapping the original color gamut information to the terminal color gamut information.

[0120] Step S364: Based on the initial color gamut information, the target lookup table, the reference white balance information, and the timestamp, adjust the color of the second video content to obtain the target video content that matches the display screen.

[0121] Step S365: Play the target video content on the display screen of the video playback terminal.

[0122] First, the video playback device needs to decode the acquired encoded data to obtain the second video content, original color gamut information, reference white balance information, and timestamp. Understandably, decoding requires a method corresponding to the aforementioned encoding. For example, if encoding is done using the High Efficiency Video Coding (HEVC) Main10 profile, then decoding can also be performed using the HEVC Main10 profile. Furthermore, during the decoding process, the video stream can be separated and additional enhancement information layers can be added.

[0123] Then, the terminal color gamut information corresponding to the display screen can be obtained. Similar to the original color gamut information, the terminal color gamut information can be directly read from the configuration file recorded on the video playback terminal. It can be understood that the terminal color gamut information represents the inherent parameters of the display screen and is related to the hardware configuration of the display screen.

[0124] To display the second video content as accurately as possible on the screen and improve color accuracy, the colors of the second video content need to be adjusted based on the terminal's color gamut information. In some implementations, a target lookup table can be obtained to map the original color gamut information to the terminal's color gamut information.

[0125] The target lookup table can be called a target LUT (LookUp Table). This lookup table allows the mapping of the original color gamut information in the encoded data to the terminal color gamut information displayed on the video playback terminal screen.

[0126] For example, a target lookup table for mapping the original color gamut information to the terminal color gamut information can be found in a pre-acquired color profile, wherein the color profile includes multiple reference color gamut information and a reference lookup table for mapping between any two reference color gamut information.

[0127] The pre-acquired color profile can be represented by the International Color Consortium Profile (ICC Profile). The color profile includes multiple reference color gamuts and a lookup table mapping between any two reference color gamuts. In other words, once it's determined that a mapping from one reference color gamut to another is needed, the lookup table can be used to determine how to map from one reference color gamut to another.

[0128] Therefore, we can look up reference color gamut information that is the same as the original color gamut information and reference color gamut information that is the same as the terminal color gamut information in the color profile respectively. Then, we can determine the reference lookup table that maps the reference color gamut information that is the same as the original color gamut information to the reference color gamut information that is the same as the terminal color gamut information. This reference lookup table can then be used as the target lookup table.

[0129] For example, the target lookup table can be a 17x17x17 matrix, which can be used to implement color mapping conversion.

[0130] The color profile can be pre-acquired and stored on a cloud server, so that the video playback terminal can directly retrieve the color profile from the cloud server.

[0131] Furthermore, based on the initial color gamut information, the target lookup table, the reference white balance information, and the timestamp, the second video content can be color-adjusted to obtain the target video content that matches the display screen.

[0132] For example, the initial color gamut information can be adjusted according to a target lookup table to compress colors exceeding the initial color gamut information based on the principle of perceptual consistency, ultimately compressing the initial color gamut information to conform to the terminal's color gamut information. Then, by combining reference white balance information and the mapped initial color gamut information, the colors of the second video content are adjusted to obtain the target video content. It is evident that even for display screens with different hardware configurations, color adjustment can achieve color consistency of the target video content as much as possible, improving the user experience.

[0133] This allows the target video content to be played on the display screen of the video playback terminal.

[0134] Understandably, the video playback device can adjust the display brightness of the screen while playing the target video content.

[0135] Specifically, step S365 may also include steps S366 and S367.

[0136] Step S366: Obtain the input-brightness correspondence relationship corresponding to the display screen, wherein the input-brightness correspondence relationship is used to characterize the mapping relationship between the voltage signal strength corresponding to the reference video content input to the display screen and the reference brightness displayed on the display screen.

[0137] Step S367: Adjust the display brightness of the target video content played on the display screen based on the input-brightness correspondence.

[0138] First, the input-brightness correspondence for the display screen can be obtained. This correspondence can be stored in the video playback device. Specifically, this correspondence characterizes the mapping between the voltage signal strength of the reference video content input to the display screen and the reference brightness displayed on the screen. Since the voltage signal strength of the target video content input to the display screen is inherently variable, the display brightness of the target video content played on the screen can be adjusted based on this input-brightness correspondence. In other words, the display brightness of the target video content played on the display screen can be adjusted in real-time according to the real-time voltage signal strength input to the display screen.

[0139] The video adjustment method provided in this application embodiment, when the target area of ​​the target object is used to characterize the target object as the area of ​​highest importance in the second video content, can ensure the color accuracy of the area of ​​highest importance in the second video content by referring to white balance information. Furthermore, in this application embodiment, the encapsulated encoded data has the characteristics of lightweight data and high compatibility, balancing compatibility and bandwidth efficiency. Moreover, after obtaining the encoded data, it can be sent to the video playback end, so that the video playback end can adjust the color of the second video content based on the initial color gamut information, the target lookup table, the reference white balance information, and the timestamp, to obtain the target video content matching the display screen; then, the target video content is played on the display screen of the video playback end. It can be seen that even for display screens with different hardware configurations, color adjustment can achieve color consistency of the target video content as much as possible, improving the user experience, that is, ensuring color consistency across devices. It also promotes the unification of industry color standards to a certain extent. This video adjustment method is also compatible with mainstream mobile electronic devices, covering a wider user group.

[0140] It should also be noted that in live streaming scenarios, such as when a host sells goods via live stream, the video adjustment method provided in this application can ensure a higher consistency between the colors of the goods seen by users on the video playback end and the actual colors of the goods. This significantly reduces the risk of returns due to color discrepancies, enhances the purchase message, and increases conversion and repurchase rates. Furthermore, it can improve the host's professional image, optimize operational efficiency, and enhance market competitiveness. Moreover, this video adjustment method can be adapted to Augmented Reality (AR) in the future, further promoting the upgrade of live streaming scenarios towards an immersive experience. Furthermore, this video adjustment solution can also be adapted to multi-screen synchronous live streaming in the future.

[0141] Please see Figure 5 , Figure 5 A flowchart illustrating a video adjustment method provided in an embodiment of this application is shown. This video adjustment method can be applied to... Figure 1 In the illustrated text generation scenario, the video capture terminal can specifically be configured with its processor as the execution entity for the video adjustment method. This video adjustment method may include steps S410 to S450.

[0142] Step S410: The video acquisition end obtains the second video content based on the image acquisition module and the color temperature sensor.

[0143] Step S420: Encode and encapsulate the second video content and metadata information to obtain encapsulated data, and send it to the video playback terminal.

[0144] Step S430: Decode the encapsulated data.

[0145] Step S440: Based on the color profile, find the target lookup table used to map the original color gamut information to the terminal color gamut information.

[0146] Step S450: Based on the initial color gamut information, the target lookup table, the reference white balance information, and the timestamp, adjust the color of the second video content to obtain the target video content that matches the display screen, and play the target video content on the display screen.

[0147] For a detailed description of steps S410 to S450, please refer to the foregoing embodiments; they will not be repeated here.

[0148] Please see Figure 6 , Figure 6 A structural block diagram of the video adjustment device provided in an embodiment of this application is shown. Figure 6 The video adjustment device 600 shown includes: a first acquisition unit 610, a second acquisition unit 620, and an adjustment unit 630.

[0149] The first acquisition unit 610 is used to acquire first video content through the image acquisition module of the video acquisition terminal when there is a light source in the environment around the video acquisition terminal.

[0150] The second acquisition unit 620 is used to acquire spectral data of the flickering light source through the color temperature sensor of the video acquisition terminal.

[0151] The adjustment unit 630 is used to adjust the first video content based on the spectral data to obtain a second video content with reduced flicker interference from the light source.

[0152] Optionally, the adjustment unit 630 can also be used to determine the frequency at which the light source flickers based on the spectral data; determine multiple flicker periods corresponding to the first video content based on the frequency, and determine multiple luminous intensities of the light source in each flicker period; determine the frame image in the first video content corresponding to the luminous intensity in each flicker period; and fuse the multiple frame images corresponding to the flicker period based on each luminous intensity and the frame image corresponding to that luminous intensity to obtain the second video content.

[0153] Optionally, the adjustment unit 630 can also be used to determine the weight of each frame image based on the luminous intensity corresponding to each frame image, wherein the weight is negatively correlated with the luminous intensity; adjust the specified parameters of each frame image within the stroboscopic period based on the weight of each frame image corresponding to the stroboscopic period; and fuse the frame images within the stroboscopic period after adjusting the specified parameters to obtain the second video content.

[0154] Optionally, the specified parameters include at least one of brightness and color temperature.

[0155] Optionally, the video adjustment device further includes an encoding and sending unit, which is used to acquire metadata information matching the second video content, wherein the metadata information includes the original color gamut information corresponding to the image acquisition module, the reference white balance information corresponding to the second video content, and the timestamp corresponding to the second video content; encode and encapsulate the second video content and the metadata information to obtain encoded data, and transmit the encoded data to the video playback terminal; control the video playback terminal to play target video content on the display screen of the video playback terminal based on the acquired encoded data, wherein the target video content includes the second video content that matches the display screen after color adjustment.

[0156] Optionally, the code sending unit is further configured to generate initial white balance information based on the spectral data; analyze the second video content based on the color gamut recognition model to determine the target color gamut interval corresponding to the target object in the second video content; adjust the initial white balance information based on the target color gamut interval to obtain reference white balance information; acquire the original color gamut information corresponding to the image acquisition module and the timestamp corresponding to the second video content; and use the reference white balance information, the original color gamut information corresponding to the image acquisition module, and the timestamp corresponding to the second video content as metadata information.

[0157] Optionally, the code sending unit is further configured to convert the second video content into video stream data; embed the metadata information into a supplementary enhancement information layer of the video stream data; encode and encapsulate the video stream data with the embedded metadata information to obtain encoded data, and transmit the encoded data to the video playback terminal.

[0158] Optionally, the code sending unit is further configured to decode the encoded data to obtain the second video content, original color gamut information, reference white balance information, and timestamp; obtain the terminal color gamut information corresponding to the display screen; obtain a target lookup table for mapping the original color gamut information to the terminal color gamut information; adjust the color of the second video content based on the initial color gamut information, the target lookup table, the reference white balance information, and the timestamp to obtain the target video content matching the display screen; and play the target video content on the display screen of the video playback terminal.

[0159] Optionally, the code sending unit is further configured to look up a target lookup table in a pre-acquired color profile for mapping the original color gamut information to the terminal color gamut information, wherein the color profile includes multiple reference color gamut information and a reference lookup table for mapping between any two reference color gamut information.

[0160] Optionally, the code sending unit is further configured to obtain the input-brightness correspondence relationship corresponding to the display screen, wherein the input-brightness correspondence relationship is used to characterize the mapping relationship between the voltage signal strength corresponding to the reference video content input to the display screen and the reference brightness displayed on the display screen; and to adjust the display brightness of the target video content played on the display screen based on the input-brightness correspondence relationship.

[0161] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described apparatus and unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0162] In the several embodiments provided in this application, the coupling between the units can be electrical, mechanical, or other forms of coupling. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0163] Please see Figure 7 , Figure 7 A structural block diagram of an electronic device provided in an embodiment of this application is shown. The electronic device 110 may be a cloud server. The electronic device 110 in this application may include one or more of the following components: a processor 111, a memory 112, and one or more application programs, wherein the processor 111 is electrically connected to the memory 112, and the one or more programs are configured to perform the methods as described in the foregoing embodiments.

[0164] Processor 111 may include one or more processing cores. Processor 111 connects to various parts within the electronic device 110 using various interfaces and lines, and performs various functions and processes data of the electronic device 110 by running or executing instructions, programs, code sets, or instruction sets stored in memory 112, and by calling data stored in memory 112. Optionally, processor 111 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 111 may integrate one or more of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and computer programs; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 111 and may be implemented separately using a communication chip. Specifically, the methods described in the foregoing embodiments can be executed by one or more processors 111.

[0165] In some implementations, memory 112 may include random access memory (RAM) or read-only memory (ROM). Memory 112 can be used to store instructions, programs, code, code sets, or instruction sets. Memory 112 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function, instructions for implementing the various method embodiments described below, etc. The data storage area may also store data created by the electronic device 110 during use.

[0166] Please see Figure 8 This diagram illustrates a structural block diagram of a computer-readable storage medium provided in an embodiment of this application. The computer-readable medium 800 stores program code that can be called by a processor to execute the methods described in the above method embodiments.

[0167] The computer-readable storage medium 800 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 800 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 810 that performs any of the method steps described above. This program code can be read from or written to one or more computer program products. The program code 810 may be compressed, for example, in a suitable form.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A video adjustment method, characterized in that, include: When there is a light source in the environment around the video acquisition terminal, the first video content is acquired through the image acquisition module of the video acquisition terminal; The color temperature sensor at the video acquisition terminal collects spectral data of the flickering light source. The first video content is adjusted based on the spectral data to obtain a second video content with reduced flicker interference from the light source.

2. The method according to claim 1, characterized in that, The step of adjusting the first video content based on the spectral data to obtain a second video content with reduced flicker interference from the light source includes: The frequency at which the light source flickers is determined based on the spectral data; Based on the frequency, a plurality of flicker cycles corresponding to the first video content are determined, and a plurality of luminous intensities of the light source are determined in each flicker cycle; Determine the frame image in the first video content corresponding to the luminous intensity in each flicker cycle; Based on each luminous intensity within a stroboscopic cycle and the corresponding frame image, multiple frame images corresponding to that stroboscopic cycle are fused to obtain the second video content.

3. The method according to claim 2, characterized in that, The second video content is obtained by fusing multiple frame images corresponding to the flicker period based on each luminous intensity within the flicker period and the corresponding frame image, including: The weight of each frame image is determined based on the luminous intensity corresponding to that frame image, wherein the weight is negatively correlated with the luminous intensity; Based on the weight of each frame image corresponding to the strobe cycle, adjust the specified parameters of each frame image within that strobe cycle; The images of each frame within the flicker period, after adjusting the specified parameters, are fused together to obtain the second video content.

4. The method according to claim 3, characterized in that, The specified parameters include at least one of brightness and color temperature.

5. The method according to claim 1, characterized in that, After adjusting the first video content based on the spectral data to obtain a second video content with reduced flicker interference from the light source, the method further includes: Obtain metadata information that matches the second video content, wherein the metadata information includes the original color gamut information corresponding to the image acquisition module, the reference white balance information corresponding to the second video content, and the timestamp corresponding to the second video content; The second video content and the metadata information are encoded and encapsulated to obtain encoded data, and the encoded data is transmitted to the video playback terminal. The video playback terminal is controlled to play target video content on its display screen based on the acquired encoded data, wherein the target video content includes the second video content that has been color-adjusted to match the display screen.

6. The method according to claim 5, characterized in that, The step of obtaining metadata information matching the second video content includes: Initial white balance information is generated based on the spectral data; The second video content is analyzed based on the color gamut recognition model to determine the target color gamut range corresponding to the target object in the second video content; The initial white balance information is adjusted based on the target color gamut range to obtain reference white balance information; Obtain the original color gamut information corresponding to the image acquisition module and the timestamp corresponding to the second video content; The reference white balance information, the original color gamut information corresponding to the image acquisition module, and the timestamp corresponding to the second video content are used as metadata information.

7. The method according to claim 5, characterized in that, The step of encoding and encapsulating the second video content and the metadata information to obtain encoded data, and transmitting the encoded data to the video playback terminal, includes: Convert the second video content into video stream data; Embed the metadata information into a supplementary enhancement information layer of the video stream data; The video stream data with embedded metadata information is encoded and encapsulated to obtain encoded data, and the encoded data is transmitted to the video playback terminal.

8. The method according to claim 5, characterized in that, The step of controlling the video playback terminal to play the target video content on the display screen of the video playback terminal based on the acquired encoded data includes: Decoding the encoded data yields the second video content, original color gamut information, reference white balance information, and timestamp. Obtain the terminal color gamut information corresponding to the display screen; Obtain a target lookup table for mapping the original color gamut information to the terminal color gamut information; Based on the initial color gamut information, the target lookup table, the reference white balance information, and the timestamp, the second video content is color-adjusted to obtain the target video content that matches the display screen; The target video content is played on the display screen of the video playback terminal.

9. The method according to claim 8, characterized in that, The step of obtaining a target lookup table for mapping the original color gamut information to the terminal color gamut information includes: The target lookup table for mapping the original color gamut information to the terminal color gamut information is searched in the pre-acquired color profile. The color profile includes multiple reference color gamut information and a reference lookup table for mapping between any two reference color gamut information.

10. The method according to claim 8, characterized in that, Playing the target video content on the display screen of the video playback terminal includes: Obtain the input-brightness correspondence relationship corresponding to the display screen, wherein the input-brightness correspondence relationship is used to characterize the mapping relationship between the voltage signal strength corresponding to the reference video content input to the display screen and the reference brightness displayed on the display screen; The display brightness of the target video content played on the display screen is adjusted based on the input-brightness correspondence.

11. A video adjustment device, characterized in that, include: The first acquisition unit is used to acquire first video content through the image acquisition module of the video acquisition terminal when there is a light source in the environment around the video acquisition terminal; The second acquisition unit is used to acquire spectral data of the flickering light source through the color temperature sensor of the video acquisition terminal; An adjustment unit is used to adjust the first video content based on the spectral data to obtain a second video content with reduced flicker interference from the light source.

12. An electronic device, characterized in that, include: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications being configured to perform the method as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be invoked by a processor to execute the method as described in any one of claims 1-10.

Images