Method and apparatus for conversion of video data, device, and medium

Abstract

Provided are a method and apparatus for conversion of video data, a device, and a medium. The method comprises: in response to determining that the color space of an original video is unspecified, setting the color space of the original video to a default value; in response to determining that the color range of the original video is unspecified, for a first pixel in a first frame among a plurality of frames of the original video, determining a first color value of the first pixel; determining the color range of the original video on the basis of the first color value; and converting the original video into a target video on the basis of the color space and the color range. By means of the implementation of the present disclosure, when an original video contains no color information or has part of color information that is missing, the color information can be simply and effectively determined, thereby converting the original video into a target video on the basis of the correct color information. In this way, it can be ensured that the colors of pixels in the target video are consistent with the colors of pixels in the original video.

Description

Methods, apparatus, devices and media for converting video data Technical Field

[0001] The implementation of this disclosure generally relates to video data conversion, and in particular to methods, apparatus, devices, and computer-readable storage media for converting video data based on color information. Background Technology

[0002] Currently, various video playback applications are available that can play videos of different formats. Videos can involve different color information (e.g., color spaces and color gamuts), and the corresponding color information is required during video encoding and / or decoding. If incorrect color information is used during video encoding and / or decoding, the colors of the displayed video may differ from those of the original video. Therefore, it is desirable to determine the video's color information in a more efficient way and to perform video conversion based on that information. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for converting video data is provided. In this method, in response to determining that the color space of the original video is empty, the color space of the original video is set to a default value. In response to determining that the color range of the original video is empty, a first color value is determined for a first pixel in a first frame of a plurality of frames in the original video. Based on the first color value, the color range of the original video is determined. Based on the color space and the color range, the original video is converted into a destination video.

[0004] In a second aspect of this disclosure, an apparatus for converting video data is provided. The apparatus includes: a setting module configured to set the color space of the original video to a default value in response to determining that the color space of the original video is empty; a value determination module configured to determine a first color value for a first pixel in a first frame of a plurality of frames in the original video in response to determining that the color range of the original video is empty; a range determination module configured to determine the color range of the original video based on the first color value; and a conversion module configured to convert the original video into a destination video based on the color space and the color range.

[0005] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes: at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to a first aspect of this disclosure when executed by the at least one processing unit.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, causes the processor to implement the method according to a first aspect of this disclosure.

[0007] In a fifth aspect of this disclosure, a computer program product is provided, including a computer program, wherein the computer program, when executed by a processor, implements the method according to a first aspect of this disclosure.

[0008] It should be understood that the content described in this content section is not intended to limit the key or essential features of the implementation of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] In the following detailed description, the above and other features, advantages, and aspects of the various implementations of this disclosure will become more apparent, taken in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1A shows a block diagram of a video processing procedure according to one implementation of the present disclosure;

[0011] Figure 1B shows a block diagram of a color space according to one implementation of the present disclosure;

[0012] Figure 2 shows a block diagram of video data conversion according to some implementations of this disclosure;

[0013] Figure 3 shows a block diagram of a process for converting video data according to some implementations of this disclosure;

[0014] Figure 4 shows a flowchart of a method for determining color information of video data according to some implementations of this disclosure;

[0015] Figure 5 shows a block diagram of a video live streaming process according to some implementations of this disclosure;

[0016] Figure 6 shows a flowchart of a method for video data conversion according to some implementations of this disclosure;

[0017] Figure 7 shows a block diagram of an apparatus for video data conversion according to some implementations of the present disclosure; and

[0018] Figure 8 shows a block diagram of a device capable of implementing various implementations of the present disclosure. Detailed Implementation

[0019] Implementations of this disclosure will now be described in more detail with reference to the accompanying drawings. While some implementations of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the implementations set forth herein. Rather, these implementations are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and implementations of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] In the description of the implementation methods disclosed herein, the term "comprising" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one implementation" or "the implementation" should be understood as "at least one implementation". The term "some implementations" should be understood as "at least some implementations". Other explicit and implicit definitions may also be included below. As used herein, the term "model" can represent the relationships between various data. For example, the aforementioned relationships can be obtained based on various currently known and / or future-developed technical solutions.

[0021] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0022] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0023] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0024] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, for example, via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0025] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0026] The term "in response to" as used herein refers to a state in which a corresponding event occurs or a condition is satisfied. It will be understood that the timing of subsequent actions performed in response to such event or condition is not necessarily strongly correlated with the time when the event occurs or the condition is met. For example, in some cases, subsequent actions may be performed immediately upon the occurrence of the event or the fulfillment of the condition; while in others, they may be performed some time after the occurrence of the event or the fulfillment of the condition.

[0027] Example Environment

[0028] Currently, various video playback applications are available that can play videos in different formats. Videos can involve different color information (e.g., color spaces and color ranges), and the corresponding color information needs to be used to perform operations during video encoding and / or decoding. Figure 1A shows a block diagram 1A of a video processing procedure according to an implementation of this disclosure. As shown in Figure 1, during video processing, video input 110 can be received. This video input 110 can be real-time captured video or stored video. For ease of description, the video conversion process is described in a live video environment. At this time, video input 110 can be received from a capture device (e.g., a camera); alternatively and / or additionally, screen recording data can be received from a computing device. Here, the input video is usually represented in RGB format, which includes three color channels: red, green, and blue, and can form video data visible to the human eye. Generally, to save transmission bandwidth, the capture device usually stores the raw data in JPEG or YUV format.

[0029] Pixel conversion 112 can be performed on the video to convert the input video data format to a specified format by the encoder 114. For example, videos of different formats can be converted to storage formats such as NV12 or V210. The encoder is a low-level algorithm module used to compress temporal redundancy, spatial redundancy, and coding redundancy information in the video data, thereby converting the large original video into a smaller destination video suitable for online storage and transmission. Subsequently, the destination video can be transmitted in a specified manner 116. For example, the destination video compressed by the encoder can be stored locally, uploaded to the cloud, or live-streamed.

[0030] In the above process, it is necessary to determine the color information of the original video. This color information may include the color space and color range. Due to hardware limitations of display devices such as screens, the color range displayed by different display devices varies. Several color space standards have been proposed to achieve consistent display specifications. Refer to Figure 1B, which describes the definition of a color space according to some implementations of this disclosure. Figure 1B shows a block diagram 100B of a color space according to one implementation of this disclosure. As shown, area 120 represents the Ultra High Definition Television (uHDTV) color space, which is mainly used in professional acquisition equipment, such as professional cameras, etc. Area 122 represents the High Definition Television (HDTV) color space, which is mainly used in non-professional acquisition equipment, such as home cameras, smart terminal devices, etc.

[0031] The two color spaces mentioned above specify the range of colors that a display device can display. If the acquisition device uses JPEG or YUV data format instead of RGB, appropriate color space parameters need to be used to perform the conversion during display; otherwise, the color of the displayed pixels may differ from the color of the pixels acquired by the acquisition device. Color information can further include color range, such as full range and limited range.

[0032] Generally, if the original video does not carry color information, preset values ​​can be used. However, these preset values ​​may be inconsistent with the color information of the captured original video, leading to color deviations or loss of contrast in the user-uploaded video stream data on the viewer's side. In other words, incorrect color information may cause the displayed video's colors to differ from the original video's colors. Therefore, it is desirable to determine the video's color information in a more efficient way and perform video conversion based on that color information.

[0033] Video Conversion Overview

[0034] To at least partially address the shortcomings of the prior art, a method for video data conversion is proposed according to one implementation of this disclosure. Referring to Figure 2, which describes an overview of one implementation of this disclosure, Figure 2 shows a block diagram 200 for video data conversion according to some implementations of this disclosure. As shown in Figure 2, raw video 210 can be received. Typically, the raw video 210 may include color information 230 (e.g., written by the acquisition device); however, sometimes the color information 230 may be empty. In this case, it is necessary to analyze the color of each pixel in the raw video 210 in order to determine the color information 230.

[0035] Color information 230 may include a color space 232 and a color range 234. In response to determining that the color space 232 of the original video 210 is empty, the color space of the original video can be set to the default value 234. It should be understood that, because professional acquisition equipment is sophisticated, it can automatically set the color space and color range for the acquired original video, and in this case, there will be no missing color information. Generally, missing color information only occurs in original videos acquired by non-professional acquisition equipment; therefore, the default value 234 can be set to HDTV.

[0036] The color range of the original video can be further determined. In response to the determination that the color range 234 of the original video is empty, a first color value is determined for a first pixel (e.g., pixel 242) in the first frame (e.g., video frame 240) of multiple frames in the original video. Based on the first color value, the color range of the original video is determined. Specifically, one or more pixels in one or more video frames of the original video can be identified, and the corresponding color range can be determined based on the color value of that pixel. For example, a subset of pixels in the original video can be selected, or all pixels can be traversed to determine the color range. The original video can be converted into a destination video based on the color space and color range.

[0037] Using the implementation method of this disclosure, when the original video does not contain color information or is missing some color information, the color information can be determined in a simple and effective way, and then the original video can be converted into a target video according to the correct color information. In this way, it can be ensured that the color of each pixel in the target video is consistent with the color of each pixel in the original video.

[0038] Detailed process of video conversion

[0039] Having described an overview of some implementations of this disclosure, further details regarding methods for converting video data will be described below. For ease of description, the specific process for converting video data is described in a live streaming environment. Figure 3 shows a block diagram 300 of the process for converting video data according to some implementations of this disclosure. As shown in Figure 3, a live streaming application can be deployed at a computing device 320, where an acquisition device 310 can acquire raw video and send the raw video to the computing device 320. It should be understood that although Figure 3 shows the raw video being acquired by a separate acquisition device 310, alternatively and / or additionally, the computing device 320 may include a built-in acquisition device. In this case, the computing device 320 can directly acquire the raw video. According to some implementations of this disclosure, the methods of this disclosure can be implemented in a live streaming application.

[0040] Individual video frames can be extracted from the original video, for example, video frame 330. Color information of the original video (340) can be set based on at least a portion of pixels in video frame 330 (e.g., pixel 332). Specifically, at box 342, the color range of the original video can be obtained. At box 344, if it is determined that the color range cannot be obtained, the color range can be detected from the individual video frames. At box 346, color information can be set based on the detected color range, and the color space in the color information can be set to a default value. Further, video preprocessing can be performed at box 350, and then the corresponding encoding parameters are provided to the encoder at box 360.

[0041] The system can identify the format of the original video. If the format is a YUV-related format (e.g., NV12, YV12, etc.), it can determine whether the original video carries color information (including color space and color range). If complete color information can be obtained, it can be used to convert the original video to the target video. Specifically, in response to determining that the video data includes a color space and color range, the original video is processed based on the color space and color range to generate the target video.

[0042] Alternatively and / or additionally, if the color space and / or color range cannot be obtained, the color space and / or color range can be set based on the original video. Specifically, if the color space is empty (i.e., the original video does not carry color space information), the color space can be set to the default value HDTV. Generally, non-professional capture equipment uses the HDTV color space and may lose color space information, thus allowing for accurate determination of the original video's color space.

[0043] According to some implementations of this disclosure, if the color range is empty (i.e., the original video does not carry information related to the color range), the color range of the original video can be determined based on the colors of each pixel in the original video. For ease of description, the process of determining the color range will be described below using the YUV format as an example only. As shown in Figure 3, pixel 332 may include three color channels: the Y channel, the U channel, and the V channel. In the context of this disclosure, a color channel may be simply referred to as a channel.

[0044] First, let's introduce the YUV format. The conversion from analog to digital signals involves sampling techniques. For a normal 8-bit image, each channel in YUV is represented by an 8-bit binary number, resulting in 256 grayscale levels. For example, for the Y channel, 255 represents white and 0 represents black. This results in 256 levels (0-255), known as the full range. In the 8-bit YUV format, the Y channel's value range is limited to 16-235, while the U and V channels range from 16-240. This method of representing color values ​​is commonly called restricted range. Different color ranges correspond to different pixel grayscale levels. If an incorrect color range is used during conversion, grayscale loss (applying restricted range conversion to original video captured at the full range) or color darkening may occur (applying full range conversion to original video captured at the restricted range).

[0045] According to some implementations of this disclosure, pixels, such as first pixels, can be determined from video frames of the original video. The first color value of the first pixel may include multiple first channel values ​​corresponding to multiple color channels. Furthermore, in determining the color range of the original video based on the first color values, the color range of the original video can be determined based on multiple first channel values. Using some implementations of this disclosure, the color range of the original video can be determined by analyzing the colors of each channel of each pixel in the original video, thereby determining the color range in a more accurate manner.

[0046] In the context of this disclosure, the color range of each pixel in each frame of the original video can be detected one by one to determine the color range of the original video. According to some implementations of this disclosure, each pixel in each video frame can be traversed, and the value of each channel of each pixel can be determined. For example, if it is determined that the value of each channel of each pixel in each video frame conforms to a restricted range, the color range of the original video can be determined to be a restricted range. Alternatively and / or additionally, a subset of pixels in a subset of video frames can be processed. When processing any channel of any pixel in any video frame, assuming that the value of that channel does not conform to a restricted range, the color range of the original video can be determined to be a complete range. In this case, the loop traversal process can be skipped, thereby reducing the computational and time resource overhead of the processing.

[0047] Specifically, each video frame in the original video can be extracted one by one, or a subset of video frames can be extracted at predetermined intervals (e.g., 2 frames, 3 frames, or other intervals). Furthermore, each pixel in the video frame can be extracted one by one, or a subset of pixels in the video frame can be extracted in a predetermined manner. For example, a region of interest can be specified in the video frame, such as the area in the center of the video frame, the brightest area, the darkest area, or an area with a prominent color, etc. Furthermore, more video frames can be captured in the region of interest, and fewer video frames can be captured in other areas. In this way, the number of pixels to be processed can be reduced, and the efficiency of the video processing can be improved.

[0048] According to some implementations of this disclosure, in the process of determining the color range of the original video based on multiple first channel values, for the first channel among the multiple color channels, in response to determining that the first channel value corresponding to the first channel is outside the first channel value range, the color range of the original video is set to the complete range, where the first channel value range corresponds to the first channel. It should be understood that each channel has its own channel value range; if a channel value is outside its corresponding channel value range, the color range can be directly determined to be the complete range.

[0049] According to some implementations of this disclosure, the first channel is the Y channel, and the value range of the first channel is [16, 235]. In this case, the value of the first channel is the value of the Y channel. If this value is outside [16, 235], the color range of the original video can be directly determined as the complete range. Using some implementations of this disclosure, it is not necessary to extract the values ​​of other channels (e.g., U and V channels), but the color range can be determined solely based on the value of the Y channel.

[0050] According to some implementations of this disclosure, the first channel may include either a U channel or a V channel, and the value range of the first channel is [16, 240]. For example, the value of the first channel may be a value of the U channel. If this value is outside [16, 240], the color range of the original video can be directly determined to be the complete range. Using some implementations of this disclosure, it is not necessary to extract the values ​​of other channels (e.g., Y, V channels), but the color range can be determined based solely on the value of the U channel. Again, for example, the value of the first channel may be a value of the V channel. If this value is outside [16, 240], the color range of the original video can be directly determined to be the complete range. Using some implementations of this disclosure, it is not necessary to extract the values ​​of other channels (e.g., Y, U channels), but the color range can be determined based solely on the value of the V channel.

[0051] According to some implementations of this disclosure, the color range of the original video can be set to a full range as long as the value of any channel exceeds the corresponding color range. Alternatively and / or additionally, the color range of the original video can be determined to be a restricted range only if the values ​​of all channels are within the corresponding color range. Specifically, in response to determining that the value of the first channel of the first channel is within the range of the first channel values, the value of the second channel of the first pixel is determined. Here, the second channel includes all channels other than the first channel. In response to determining that the value of the second channel is within the range of the second channel values, the color range of the original video is set to a restricted range, where the range of the second channel values ​​corresponds to the second channel.

[0052] For example, if the Y channel value is determined to satisfy [16, 235], it can be determined whether both the U and V channels satisfy [16, 240]. If both the U and V channels satisfy [16, 240], then the color range is determined to be a restricted range. Again, for example, if the U channel value is determined to satisfy [16, 240], it can be determined whether the Y channel and V channels satisfy [16, 240]. If both the Y and V channels satisfy [16, 235], then the color range is determined to be a restricted range. Yet again, for example, if the V channel value is determined to satisfy [16, 240], it can be determined whether the Y channel and U channels satisfy [16, 240]. If both the Y and U channels satisfy [16, 235], then the color range is determined to be a restricted range. Using some of the implementations disclosed herein, color ranges can be determined in a simpler and more accurate manner.

[0053] According to some implementations of this disclosure, the color distribution of pixels in different video frames of the original video may differ. The fact that all pixels in one video frame satisfy the constrained range does not mean that all pixels in all video frames of the original video satisfy the constrained range. If the color range is determined to be constrained, the process described above needs to be performed on subsequent video frames to determine if there are any pixels that do not satisfy the constrained range. If there are pixels that do not satisfy the constrained range, the color range of the original video can be updated to the complete range. Using some implementations of this disclosure, the color range can be updated as more video frames are received, so that the determined color range can correctly match the original video.

[0054] According to some implementations of this disclosure, in response to determining that the color gamut of the original video is a limited range, the third channel value of the first channel of the second pixel in the second frame of the multiple frames in the original video can be determined. Further, in response to determining that the third channel value is outside the range of the first channel values, the color gamut of the original video is updated to a complete range. The first channel may include any one of the Y channel, U channel, and V channel. As long as the value of any channel exceeds the corresponding value range, the color gamut of the original video can be determined to be a complete range. Here, the second frame can be determined in various ways. The frame immediately following the first frame can be determined as the second frame. Alternatively and / or additionally, the second frame can be selected according to predetermined conditions, such as 2 frames, 3 frames, or other intervals, where the interval between the second frame and the first frame satisfies the predetermined conditions. In this way, it is not necessary to analyze each video frame individually, thereby reducing the computational resource overhead and time overhead in the video processing process.

[0055] According to some implementations of this disclosure, the value of the fourth channel of the second channel of the second pixel can be further determined, and the second channel can be any channel different from the first channel. In response to determining that the value of the fourth channel is outside the range of the second channel values, the color range of the original video is updated to the full range. Specifically, the second pixel in the second frame can be processed in a manner similar to that described above for processing the first pixel in the first frame. Using some implementations of this disclosure, the color range of the original video can be determined one by one based on the values ​​of each channel.

[0056] According to some implementations of this disclosure, the original video can be processed based on a defined color space and color range to generate a target video. It should be understood that the color space and color range here can be extracted from the original video, or it can be determined based on the methods described above. Using some implementations of this disclosure, a more accurate color space and color range can be determined, thereby ensuring that the colors of the subsequently displayed target video are consistent with the colors of the original video captured by the acquisition device.

[0057] The various steps of processing video data have been described. Hereinafter, referring to Figure 4, a complete flow for determining color information is described, illustrating a flowchart of a method 400 for determining color information of video data according to some implementations of this disclosure. As shown in Figure 4, at box 410, the video to be processed can be received; for example, the computing device can receive the video from a standalone acquisition device or a built-in acquisition device. At box 412, video data in YUV format can be received. If the received video data involves other formats, the received video data can be converted to YUV format based on a format conversion process. At box 414, the color information of the video data can be queried. For example, whether the video data carries color information can be queried via an application programming interface matching the video format. Here, the color information may include a color space and a color range.

[0058] At box 416, if the color space and color range can be successfully acquired, it indicates that the color information is complete, and method 400 can proceed to box 430 to set video processing parameters based on the acquired color space and color information. At box 416, if it is determined that the color space and color range were not successfully acquired, method 400 proceeds to box 418. At box 418, if the color space was not successfully acquired, the color space can be set to the default parameter (e.g., HDTV). Furthermore, if the color range was not successfully acquired, the range of each pixel in the video can be detected according to the method described above to determine the specific type of the color range.

[0059] Specifically, a subset of video frames can be selected, or all video frames can be traversed. Further, a subset of pixels within a video frame can be selected, or all pixels within that video frame can be traversed. For a given pixel, at box 420, it can be determined whether the values ​​of the Y, U, and V channels of each pixel satisfy their respective value ranges. Specifically, if it is determined that all values ​​of the Y channel satisfy [16, 235], and all values ​​of the U and V channels satisfy [16, 235], method 400 proceeds to box 424 and the color range can be set to a restricted range. Alternatively and / or additionally, to improve accuracy, the values ​​of the YUV channels in subsequent video frames can be detected so that if pixels that do not satisfy the value range are found, the color range can be updated to the full range.

[0060] At box 426, it can be determined whether all values ​​of the Y channel satisfy [16, 235], and whether all values ​​of the U and V channels satisfy [16, 235]. If the determination result is "No", method 400 proceeds to box 422 to set the color range to the full range. Then, at box 428, preprocessing can be performed on the video according to the determined color space and color range. Preprocessing may include, but is not limited to, format conversion processing, effects processing, etc. Specifically, the video can be converted to the desired format, or effects such as text, stickers, and style transitions can be added to the video. At box 430, encoding processing can be performed on the preprocessed video, thereby generating a compressed encoded video. If the determination result at box 426 is "Yes", method 400 returns to box 424. At this time, the color range continues to remain within the limited range, and individual pixels in subsequent video frames can continue to be checked in a similar manner.

[0061] It should be understood that although the process of determining the color range was described above using YUV format as an example, alternatively and / or additionally, pixels can be represented in RGB format, and the corresponding YUV format can be determined based on the conversion formula between RGB and YUV formats, and then the color range can be determined according to the process described above.

[0062] According to some implementations of this disclosure, the technical solutions described above can be executed in a live video environment. See Figure 5 for further details, which shows a block diagram 500 of a live video streaming process according to some implementations of this disclosure. As shown in Figure 5, video 512 can be acquired by an application 510 providing live video streaming functionality, and further, specific parameters 514 of the encoder can be configured. It should be understood that the acquired data can involve both audio and video data, and the two types of data can be processed separately in audio processing branches and video processing branches.

[0063] In the audio branch, audio data can be acquired at box 520, audio preprocessing can be performed at box 522, and audio encoding can be performed at box 524. In the video processing branch, video data can be acquired at box 530. At box 532, the methods described above can be performed to determine the color information of the acquired video data. At box 534, preprocessing can be performed on the video data, and at box 536, video encoding can be performed. At box 550, the output data from the audio processing branch and the video processing branch can be merged to generate a live data stream including audio and video components. At boxes 552 and 554, the generated data stream can be encapsulated according to a predetermined format and protocol for transmission to other devices at box 556.

[0064] In the context of this disclosure, if the color gamut of the captured video image cannot be obtained, the color gamut can be adaptively detected based on the process described above. In this way, a video with colors as consistent as possible with the original captured video can be presented. Furthermore, in the above process, there is no distinction between on-demand and live streaming; color information can be determined using the method described above in all video playback scenarios. Furthermore, in the above process, no manual operation from the video creator and / or broadcaster is required; color information can be determined adaptively.

[0065] Using the implementation method of this disclosure, when the original video does not contain color information or is missing some color information, the color information can be determined in a simple and effective way, and then the original video can be converted into a target video according to the correct color information. In this way, it can be ensured that the color of each pixel in the target video is consistent with the color of each pixel in the original video.

[0066] Example process

[0067] Figure 6 illustrates a flowchart of a method 600 for converting video data according to some implementations of this disclosure. At block 610, in response to determining that the color space of the original video is empty, the color space of the original video is set to a default value. At block 620, in response to determining that the color range of the original video is empty, a first color value for a first pixel in a first frame of a plurality of frames in the original video is determined. At block 630, the color range of the original video is determined based on the first color value. At block 640, the original video is converted into a destination video based on the color space and color range.

[0068] According to some implementations of this disclosure, the first color value includes multiple first channel values ​​corresponding to multiple color channels, and determining the color range of the original video based on the first value includes: determining the color range of the original video based on the multiple first channel values.

[0069] According to some implementations of this disclosure, determining the color range of the original video based on multiple first channel values ​​includes: for a first channel among multiple color channels, in response to determining that the first channel value corresponding to the first channel is outside the range of first channel values, setting the color range of the original video to a complete range, wherein the range of first channel values ​​corresponds to the first channel.

[0070] According to some implementations of this disclosure, determining the color range of the original video further includes: in response to determining that the first channel value of the first channel is within the first channel value range, determining the second channel value of the second channel of the first pixel, the second channel including any channel different from the first channel; and in response to determining that the second channel value is within the second channel value range, setting the color range of the original video to a restricted range, the second channel value range corresponding to the second channel.

[0071] According to some implementations of this disclosure, determining the color range of the original video further includes: in response to determining that the color range of the original video is a limited range, determining the third channel value of the first channel of the second pixel in the second frame of the multiple frames in the original video; and in response to determining that the third channel value is outside the range of the first channel value, updating the color range of the original video to a complete range.

[0072] According to some implementations of this disclosure, determining the color range of the original video further includes: determining the fourth channel value of the second channel of the second pixel; and updating the color range of the original video to the full range in response to determining that the fourth channel value is outside the range of the second channel values.

[0073] According to some implementations of this disclosure, the interval between the second frame and the first frame satisfies predetermined conditions.

[0074] According to some implementations of this disclosure, the first channel is a Y channel, and the value range of the first channel is [16, 235].

[0075] According to some implementations of this disclosure, the first channel includes either the U channel or the V channel, and the numerical range of the first channel is [16, 240].

[0076] According to some implementations of this disclosure, the original video includes live video, and the method is implemented in an application used to provide live streaming services.

[0077] Example devices and equipment

[0078] Figure 7 shows a block diagram of an apparatus 700 for converting video data according to some implementations of the present disclosure. The apparatus includes: a setting module 710 configured to set the color space of the original video to a default value in response to determining that the color space of the original video is empty; a value determination module 720 configured to determine a first color value for a first pixel in a first frame of a plurality of frames in the original video in response to determining that the color range of the original video is empty; a range determination module 730 configured to determine the color range of the original video based on the first color value; and a conversion module 740 configured to convert the original video into a destination video based on the color space and the color range.

[0079] According to some implementations of this disclosure, the first color value includes multiple first channel values ​​corresponding to multiple color channels, and the range determination module is further configured to: determine the color range of the original video based on the multiple first channel values.

[0080] According to some implementations of this disclosure, the range determination module is further configured to: for a first channel among a plurality of color channels, in response to determining that the value of the first channel corresponding to the first channel is outside the range of the first channel values, set the color range of the original video to a full range, wherein the range of the first channel values ​​corresponds to the first channel.

[0081] According to some implementations of this disclosure, the range determination module is further configured to: in response to determining that the first channel value of the first channel is within the range of the first channel value, determine the second channel value of the second channel of the first pixel, the second channel including any channel different from the first channel; and in response to determining that the second channel value is within the range of the second channel value, set the color range of the original video to a restricted range, the range of the second channel value corresponding to the second channel.

[0082] According to some implementations of this disclosure, the range determination module is further configured to: in response to determining that the color range of the original video is a limited range, determine the third channel value of the first channel of the second pixel in the second frame of a plurality of frames in the original video; and in response to determining that the third channel value is outside the range of the first channel value, update the color range of the original video to a full range.

[0083] According to some implementations of this disclosure, the range determination module is further configured to: determine the fourth channel value of the second channel of the second pixel; and update the color range of the original video to the full range in response to determining that the fourth channel value is outside the range of the second channel values.

[0084] According to some implementations of this disclosure, the interval between the second frame and the first frame satisfies predetermined conditions.

[0085] According to some implementations of this disclosure, the first channel is a Y channel, and the value range of the first channel is [16, 235].

[0086] According to some implementations of this disclosure, the first channel includes either the U channel or the V channel, and the numerical range of the first channel is [16, 240].

[0087] According to some implementations of this disclosure, the original video includes live video, and the method is implemented in an application used to provide live streaming services.

[0088] Figure 8 shows a block diagram of a device 800 capable of implementing various implementations of the present disclosure. It should be understood that the computing device 800 shown in Figure 8 is merely exemplary and should not constitute any limitation on the functionality and scope of the implementations described herein. The computing device 800 shown in Figure 8 can be used to implement the methods described above.

[0089] As shown in Figure 8, the computing device 800 is in the form of a general-purpose computing device. Components of the computing device 800 may include, but are not limited to, one or more processors or processing units 810, memory 820, storage devices 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. The processing unit 810 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 820. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of the computing device 800.

[0090] Computing device 800 typically includes multiple computer storage media. Such media can be any available media accessible to computing device 800, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 820 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 830 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within computing device 800.

[0091] The computing device 800 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG8, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks may be provided. In these cases, each drive may be connected to a bus (not shown) via one or more data media interfaces. The memory 820 may include a computer program product 825 having one or more program modules configured to perform various methods or actions of various implementations of the present disclosure.

[0092] The communication unit 840 enables communication with other computing devices via a communication medium. Additionally, the components of the computing device 800 can function as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the computing device 800 can operate in a networked environment using logical connections to one or more other servers, networked personal computers (PCs), or another network node.

[0093] Input device 850 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 860 can be one or more output devices, such as a monitor, speaker, printer, etc. Computing device 800 can also communicate as needed with one or more external devices (not shown) via communication unit 840. These external devices, such as storage devices, display devices, etc., can communicate with one or more devices that enable user interaction with computing device 800, or with any device (e.g., network card, modem, etc.) that enables computing device 800 to communicate with one or more other computing devices. Such communication can be performed via input / output (I / O) interfaces (not shown).

[0094] According to an implementation of this disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, which are executed by a processor to implement the method described above. According to an implementation of this disclosure, a computer program product is provided, on which a computer program is stored, which, when executed by a processor, implements the method described above.

[0095] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0096] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0097] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0099] Various implementations of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for converting video data, comprising: In response to determining that the color space of the original video is empty, the color space of the original video is set to the default value; In response to determining that the color range of the original video is empty, a first color value of the first pixel is determined for the first pixel in the first frame of the multiple frames in the original video; Based on the first color value, the color range of the original video is determined; as well as Based on the color space and the color range, the original video is converted into the target video.

2. The method according to claim 1, wherein the first color value includes a plurality of first channel values ​​corresponding to a plurality of color channels respectively, and determining the color range of the original video based on the first value includes: The color range of the original video is determined based on the multiple first channel values.

3. The method of claim 2, wherein determining the color range of the original video based on the plurality of first channel values ​​comprises: For a first channel among the plurality of color channels, in response to determining that the value of the first channel corresponding to the first channel is outside the range of the first channel values, the color range of the original video is set to the full range, wherein the range of the first channel values ​​corresponds to the first channel.

4. The method of claim 3, wherein determining the color range of the original video further comprises: In response to determining that the value of the first channel of the first channel is within the range of the first channel value, the value of the second channel of the first pixel is determined, wherein the second channel includes any channel different from the first channel; as well as In response to determining that the second channel value is within the range of the second channel values, the color range of the original video is set to a restricted range, the range of the second channel values ​​corresponding to the second channel.

5. The method of claim 4, wherein determining the color range of the original video further comprises: In response to determining that the color range of the original video is the restricted range, the third channel value of the first channel of the second pixel in the second frame of the plurality of frames in the original video is determined; as well as In response to determining that the third channel value is outside the range of the first channel value, the color range of the original video is updated to the full range.

6. The method of claim 5, wherein determining the color range of the original video further comprises: Determine the value of the fourth channel of the second channel of the second pixel; as well as In response to determining that the fourth channel value is outside the range of the second channel value, the color range of the original video is updated to the full range.

7. The method according to claim 5, wherein the interval between the second frame and the first frame satisfies a predetermined condition.

8. The method according to claim 3, wherein the first channel is a Y channel and the value range of the first channel is [16, 235].

9. The method of claim 3, wherein the first channel includes either a U channel or a V channel, and the value range of the first channel is [16, 240].

10. The method of claim 1, wherein the original video includes live video, and the method is implemented in an application for providing live streaming services.

11. An apparatus for converting video data, comprising: The settings module is configured to set the color space of the original video to a default value in response to determining that the color space of the original video is empty; A value determination module is configured to determine a first color value for a first pixel in a first frame of a plurality of frames in the original video in response to determining that the color range of the original video is empty. A range determination module is configured to determine the color range of the original video based on the first color value; as well as A conversion module is configured to convert the original video into a target video based on the color space and the color range.

12. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 10 when executed by the at least one processing unit.

13. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 10.

14. A computer instruction product comprising computer instructions, wherein the computer instructions, when executed by a processor, implement the method according to any one of claims 1 to 10.

Images