Video data processing method and device, encoder, network equipment and loader

By adjusting the video encoding parameters according to the target transmission rate, the problem of mismatch between the transmission rate and the video quality in the video streaming media is solved, and the synchronous improvement of the video quality and the transmission rate is achieved, and the user experience is improved.

CN118984393APending Publication Date: 2024-11-19CHINA MOBILE COMM LTD RES INST +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310575417.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-19
Publication Date
2024-11-19

AI Technical Summary

Technical Problem

There is a problem in the existing video encoding technology that the transmission rate does not match the video quality, resulting in a bandwidth step effect, and the video quality does not improve simultaneously with the increase of bandwidth within a certain rate range.

Method used

By receiving the target transmission rate between the network device and the target terminal, determine the encoding parameters of the video to be played, including the encoding parameters of the basic layer and the enhancement layer, adjust the video code stream to match the transmission rate, and ensure transmission during the playback of the video streaming media The rate matches the video quality.

Benefits of technology

It achieves the matching of the transmission rate and video quality during video streaming, and improves the synchronization of video quality and user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118984393A_ABST
    Figure CN118984393A_ABST
Patent Text Reader

Abstract

The invention provides a video data processing method and device, an encoder, network equipment and a loader, and relates to the technical field of wireless communication. The method comprises the following steps: receiving a target transmission rate sent by network equipment; determining a first parameter of a to-be-played video according to the target transmission rate, the first parameter including a first coding parameter of a base layer, or the first parameter including the first coding parameter of the base layer and a first number and a second coding parameter of enhancement layers; according to the first parameter, encoding the video to be played to obtain an output video code stream comprising at least one network abstraction layer unit (NALU); and sending the network abstraction layer unit to a loader. According to the scheme, the number of the enhancement layers and the coding parameters are determined according to the transmission rate between the network equipment and the terminal, the number of the enhancement layers and the coding parameters can be adjusted along with changes of the transmission rate, and it is guaranteed that the transmission rate is matched with the video quality in the playing process of the video streaming media.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a video data processing method, device, encoder, network equipment and loader. Background Art

[0002] Existing video coding has a serious step effect, that is, as the transmission rate increases, the video quality and user experience do not increase synchronously; instead, the video quality and user experience will only increase when the rate reaches the next resolution threshold. For example, a bandwidth of 2Mbps can watch 480p video, and 10M can watch 1080p video. However, in the range [2-10), even if the bandwidth increases, the video quality will not improve synchronously because it does not reach the bandwidth threshold of the next video quality level.

[0003] The bandwidth step effect is shown in Figure 2. Figure 1 As shown in the figure, as the data rate increases, the video resolution does not increase gradually, but increases in steps after reaching the minimum rate required for its transmission; similarly, when the transmission rate drops below the minimum rate, even if the transmission rate is much higher than the minimum rate required for the next level of transmission, only the next level of video can be played. Therefore, the existing technology has the problem of mismatch between the transmission rate and the video quality in video streaming playback. Summary of the invention

[0004] The purpose of the present invention is to provide a video data processing method, device, encoder, network equipment and loader to solve the problem of mismatch between transmission rate and video quality in video streaming media playback in the prior art.

[0005] To achieve the above object, an embodiment of the present invention provides a video data processing method, which is applied to an encoder, comprising:

[0006] Receiving a target transmission rate sent by a network device, wherein the target transmission rate is a transmission rate between the network device and a target terminal;

[0007] Determine a first parameter of the video to be played according to the target transmission rate, the first parameter comprising: a first encoding parameter of a base layer, or the first parameter comprising: a first encoding parameter of the base layer and a first number and a second encoding parameter of an enhancement layer;

[0008] According to the first parameter, the video to be played is encoded to obtain an output video code stream including at least one network abstraction layer unit NALU;

[0009] The network abstraction layer unit is sent to the loader.

[0010] Furthermore, the method for determining the first coding parameter of the base layer includes:

[0011] Determine a preset ratio of the target transmission rate as a first transmission rate of the base layer;

[0012] The first encoding parameter is determined according to the first transmission rate.

[0013] Furthermore, the method for determining the first number of enhancement layers and the second coding parameter includes:

[0014] Determining a first number of enhancement layers according to the target transmission rate and the first transmission rate;

[0015] The second encoding parameter is determined according to the target transmission rate, the first transmission rate, and the first number.

[0016] Furthermore, the first encoding parameter and the second encoding parameter include:

[0017] Resolution, frame rate, and encoding quality;

[0018] The highest level of the enhancement layer corresponds to the highest resolution of the video to be played.

[0019] Furthermore, the network abstraction layer includes at least one group of pictures GoP.

[0020] To achieve the above object, an embodiment of the present invention provides a video data processing method, which is applied to a network device, comprising:

[0021] Obtain status data of the target terminal;

[0022] Determining a target transmission rate between the network device and the target terminal according to the state data and the load data of the network device;

[0023] Sending the target transmission rate to an encoder, wherein the target transmission rate is used by the encoder to determine a first parameter of a video to be played;

[0024] Sending to the loader a transport block size TBS allocated for the video to be played;

[0025] Receives the Network Abstraction Layer Unit (NALU) sent by the loader.

[0026] Further, determining the target transmission rate according to the state data and the load data of the network device includes:

[0027] Determining an average transmission rate from the network device to the target terminal according to the state data and the load data of the network device;

[0028] The target transmission rate is determined according to the average transmission rate.

[0029] Further, the sending to the loader a transport block size TBS allocated for the video to be played includes:

[0030] Acquire a channel condition of the target terminal, the channel condition including: channel state information and a channel quality indicator;

[0031] According to the channel condition of the target terminal and the channel resource utilization rate of the network device, a transmission block size TBS allocated to each group of pictures GoP corresponding to the video to be played is determined.

[0032] Furthermore, the status data includes:

[0033] The transmission bandwidth, channel quality parameters and the highest modulation mode supported by the target terminal;

[0034] The load data includes:

[0035] The number of access terminals, the number of activated terminals and the air interface resource utilization rate of the network device.

[0036] To achieve the above object, an embodiment of the present invention provides a video data processing method, which is applied to a loader, comprising:

[0037] Receive a transmission block size TBS allocated for the video to be played sent by a network device;

[0038] Receiving at least one network abstraction layer unit NALU sent by the encoder;

[0039] The network abstraction layer unit NALU is sent to the network device according to the transport block size TBS.

[0040] Further, sending the network abstraction layer unit NALU to the network device according to the transmission block size TBS includes:

[0041] Determine whether to discard a network abstraction layer unit NALU according to a transmission block size TBS allocated by the network device for the video to be played;

[0042] If it is determined that the network abstraction layer unit NALU needs to be discarded, discard at least one target network abstraction layer unit NALU in the network abstraction layer unit;

[0043] The remaining network abstraction layer unit NALUs after discarding the target network abstraction layer unit NALU in the network abstraction layer unit NALU are sent to the network device.

[0044] Furthermore, the transmission block size TBS allocated for the video to be played includes:

[0045] The transport block size TBS allocated to each picture group corresponding to the video to be played.

[0046] Further, the determining whether it is necessary to discard a network abstraction layer unit NALU according to a transport block size TBS allocated by the network device to each picture group includes:

[0047] Determine the association relationship between each of the network abstraction layer units NALU;

[0048] And according to the association relationship and the transmission block size TBS allocated by the network device to each picture group, it is determined whether the network abstraction layer unit needs to be discarded.

[0049] Furthermore, the method for determining the target network abstraction layer unit NALU includes:

[0050] Determine the target network abstraction layer unit according to the second parameter of the network abstraction layer unit NALU;

[0051] The second parameter includes at least one of the following: a level of a corresponding enhancement layer, a degree of dependence on other network abstraction layer units, a corresponding region of interest, and a number of historical discards.

[0052] To achieve the above object, an embodiment of the present invention provides a video data processing device, applied to an encoder, comprising:

[0053] A first receiving module, used for receiving a target transmission rate sent by a network device;

[0054] A first determination module is used to determine a first parameter of the video to be played according to the target transmission rate, wherein the first parameter includes: a first encoding parameter of a base layer, or the first parameter includes: a first encoding parameter of the base layer and a first number and a second encoding parameter of an enhancement layer;

[0055] An encoding module, configured to encode the video to be played according to the first parameter to obtain an output video code stream including at least one network abstraction layer unit NALU;

[0056] The first sending module is used to send the network abstraction layer unit to the loader.

[0057] To achieve the above object, an embodiment of the present invention provides a video data processing device, which is applied to a network device, including:

[0058] An acquisition module, used to acquire the status data of the target terminal;

[0059] A second determination module, configured to determine a target transmission rate according to the state data and the load data of the network device;

[0060] A second sending module, used to send the target transmission rate to an encoder, where the target transmission rate is used by the encoder to determine a first parameter of a video to be played;

[0061] A third sending module, used for sending a transport block size TBS allocated for the video to be played to the loader;

[0062] The second receiving module is used to receive the network abstraction layer unit NALU sent by the loader.

[0063] To achieve the above object, an embodiment of the present invention provides a video data processing device, applied to a loader, comprising:

[0064] A third receiving module is used to receive a transmission block size TBS allocated for the video to be played and sent by a network device;

[0065] A fourth receiving module, configured to receive at least one network abstraction layer unit NALU sent by the encoder;

[0066] The fourth sending module is used to send the network abstraction layer unit NALU to the network device according to the transmission block size TBS.

[0067] To achieve the above object, an embodiment of the present invention provides an encoder, including a processor and a transceiver, wherein:

[0068] The transceiver is used to receive the target transmission rate sent by the network device;

[0069] The processor is used to determine a first parameter of the video to be played according to the target transmission rate, where the first parameter includes: a first encoding parameter of a base layer, or the first parameter includes: a first encoding parameter of the base layer and a first number and a second encoding parameter of an enhancement layer;

[0070] According to the first parameter, the video to be played is encoded to obtain an output video code stream including at least one network abstraction layer unit NALU;

[0071] The transceiver is further configured to send the network abstraction layer unit to a loader.

[0072] To achieve the above object, an embodiment of the present invention provides a network device, including a processor and a transceiver, wherein:

[0073] The transceiver is used to obtain status data of the target terminal;

[0074] The processor is used to determine a target transmission rate according to the state data and the load data of the network device;

[0075] The transceiver is also used to send the target transmission rate to the encoder, and the target transmission rate is used by the encoder to determine the first parameter of the video to be played;

[0076] Sending to the loader a transport block size TBS allocated for the video to be played;

[0077] Receives the Network Abstraction Layer Unit (NALU) sent by the loader.

[0078] To achieve the above object, an embodiment of the present invention provides a loader, including a processor and a transceiver, wherein:

[0079] The transceiver is used to receive a transmission block size TBS allocated for a video to be played and sent by a network device;

[0080] Receiving at least one network abstraction layer unit NALU sent by the encoder;

[0081] The network abstraction layer unit NALU is sent to the network device according to the transport block size TBS.

[0082] To achieve the above-mentioned purpose, an embodiment of the present invention provides a terminal device, including a transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; when the processor executes the program or instruction, the video data processing method as described above is implemented.

[0083] To achieve the above objective, an embodiment of the present invention provides a readable storage medium on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps in the video data processing method described above are implemented.

[0084] The beneficial effects of the above technical solution of the present invention are as follows:

[0085] In the video data processing method of the embodiment of the present invention, the encoder determines the first parameters of the video to be played including the first encoding parameter of the base layer, or the first encoding parameter of the base layer and the first number and second encoding parameter of the enhancement layer according to the received target transmission rate between the network device and the target terminal, and determines the video code stream including at least one network abstraction layer unit according to the first parameter. The scheme of the present invention determines the number and encoding parameters of the enhancement layers according to the transmission rate between the network device and the terminal, and can adjust the number and encoding parameters of the enhancement layers as the transmission rate changes, so as to ensure that the transmission rate matches the video quality during the playback of the video streaming media. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 Schematic diagram of bandwidth step effect;

[0087] Figure 2 It is a schematic diagram of the structure of scalable video coding;

[0088] Figure 3 A schematic diagram of a process of scalable video coding;

[0089] Figure 4 One of the structural diagrams of the network abstraction layer unit;

[0090] Figure 5 This is the second structural diagram of the network abstraction layer unit;

[0091] Figure 6 is a flowchart of a video data processing method applied to an encoder according to an embodiment of the present invention;

[0092] Figure 7 A flowchart of a video data processing method applied to a network device according to an embodiment of the present invention;

[0093] Figure 8 A flow chart of a method for processing video data applied to a loader according to an embodiment of the present invention;

[0094] Fig. 9 A logical diagram of video data processing according to an embodiment of the present invention;

[0095] Fig.10 A logical diagram of video data processing according to another embodiment of the present invention;

[0096] Fig.11 A schematic diagram of the structure of a video data processing device applied to an encoder according to an embodiment of the present invention;

[0097] Fig.12 A schematic diagram of the structure of a video data processing device applied to a network device according to an embodiment of the present invention;

[0098] Fig.13 It is a structural schematic diagram of a video data processing device applied to a loader according to an embodiment of the present invention;

[0099] Fig.14 is a schematic structural diagram of an encoder according to an embodiment of the present invention;

[0100] Fig.15 A schematic diagram of the structure of a network device according to an embodiment of the present invention;

[0101] Fig.16 is a schematic structural diagram of a loader according to an embodiment of the present invention;

[0102] Fig.17It is a schematic diagram of the structure of a terminal device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0103] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0104] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present invention. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0105] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0106] Additionally, the terms "system" and "network" are often used interchangeably herein.

[0107] In the embodiments provided in the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0108] It should be noted that the encoder of the embodiment of the present invention is a radio access network (RAN) encoder based on scalable video coding SVC and scalable high efficiency video coding (SHVC), and the encoder can communicate with a RAN side device (ie, a network device); a NALU loader that can selectively load the NALU output by the encoder to the RAN and can communicate with the network device.

[0109] like Figure 2 As shown in Figure 1, Scalable Video Coding (SVC) is a technology that can split a video stream into multiple spatial layers, quality and time layers. It is an extension of the H.264 video codec standard used by most video devices today. Video devices use SVC technology to send and receive multi-layer video streams consisting of a small base layer and multiple other optional layers (enhancement layers) that can improve resolution, frame rate and quality.

[0110] like Figure 3As shown in the figure, SVC adapts to channel changes by splitting high-quality video streams into multiple self-streams, which are divided into an independently decodable base layer (Base Layer, BL) and an enhancement layer (EnhanceLayer, EL) that depends on BL and bottom layer decoding. Starting from H.264AVC, video coding is divided into a two-layer structure of Video Coding Layer (VCL) and Network Abstraction Layer (NAL). VCL is responsible for encoding the video into a longer original stream, and then the original stream is segmented into shorter streams according to the slices of the image. NAL encapsulates the segmented stream into the Network Abstraction Layer Unit (NALU) and can be transmitted on the network.

[0111] NALU can be considered as the smallest video encoding unit transmitted on the network. Its structure is as follows Figure 4 and Figure 5 As shown in the figure. NALU is divided into a header (NAL Header) and a network raw byte sequence payload (RBSP, Raw Byte Sequence Payload), where RBSP is divided into three types: VCL NAL unit containing a picture parameter set (PPS), PS NAL unit containing a sequence parameter set (SPS), and SEI NAL unit containing supplemental enhancement information (SEI). Among them, a VCL NAL unit generally loads the coded data corresponding to a slice.

[0112] like Figure 6 As shown, a video data processing method according to an embodiment of the present invention is applied to an encoder, comprising the following steps:

[0113] Step 601: receiving a target transmission rate sent by a network device, where the target transmission rate is a transmission rate between the network device and a target terminal.

[0114] Step 602, determine the first parameter of the video to be played according to the target transmission rate, the first parameter includes: the first encoding parameter of the base layer, or the first parameter includes: the first encoding parameter of the base layer and the first number of enhancement layers and the second encoding parameter of each enhancement layer.

[0115] Step 603: Encode the video to be played according to the first parameter to obtain an output video code stream including at least one network abstraction layer unit NALU.

[0116] Step 604: Send the network abstraction layer unit to the loader.

[0117] Optionally, each of the network abstraction layer units corresponds to only a slice in an image.

[0118] Optionally, the average bit rate of the video to be played is determined according to the target transmission rate, and then the maximum resolution of the video to be played is determined.

[0119] In the video data processing method of the embodiment of the present invention, the highest resolution of the video to be played is slightly higher than the bit rate corresponding to the target transmission rate.

[0120] It should be noted that the first parameter is updated as the target transmission rate changes.

[0121] In one embodiment of the present invention, the encoder can determine the average bit rate of the video to be played through the target transmission rate sent by the network device, and then determine the highest resolution. The base layer is a mandatory option for encoding the video to be played, and the enhancement layer is an optional option for the video to be played. For example, if the transmission rate is too low, only the base layer data is encoded and transmitted, otherwise the highest resolution is the resolution corresponding to the highest enhancement layer, and the average bit rate of the corresponding video encoding at the highest level is slightly greater than the target transmission rate.

[0122] In the video data processing method of the embodiment of the present invention, the encoder determines the first parameters of the video to be played including the first encoding parameter of the base layer, or the first encoding parameter of the base layer and the first number and second encoding parameter of the enhancement layer according to the received target transmission rate between the network device and the target terminal, and determines the video code stream including at least one network abstraction layer unit according to the first parameter. The scheme of the present invention determines the number and encoding parameters of the enhancement layers according to the transmission rate between the network device and the terminal, and can adjust the number and encoding parameters of the enhancement layers as the transmission rate changes, so as to ensure that the transmission rate matches the video quality during the playback of the video streaming media.

[0123] Optionally, the method for determining the first encoding parameter of the base layer includes:

[0124] Determine a preset ratio of the target transmission rate as a first transmission rate of the base layer;

[0125] The first encoding parameter is determined according to the first transmission rate.

[0126] Optionally, the preset ratio ranges from 10% to 50%.

[0127] Optionally, when the target transmission rate is too low, only the base layer data is encoded and transmitted.

[0128] In one embodiment of the present invention, a preset ratio of the target transmission rate is used as the transmission rate of the base layer, and based on this, the coding parameters of the base layer are determined; then the number and coding parameters of the enhancement layers are determined according to the coding parameters of the base layer.

[0129] Optionally, the enhancement layer may be selected as an integer multiple, such as 40p, 480p, 960p, 1920p or 270p, 540p, 1080p, 2160p or 360p, 720p, 1440p.

[0130] Optionally, the manner of determining the first number of enhancement layers and the second coding parameter includes:

[0131] Determining a first number of enhancement layers according to the target transmission rate and the first transmission rate;

[0132] The second encoding parameter is determined according to the target transmission rate, the first transmission rate, and the first number.

[0133] Optionally, the first encoding parameter and the second encoding parameter include:

[0134] Resolution, frame rate, and encoding quality;

[0135] The highest level of the enhancement layer corresponds to the highest resolution of the video to be played.

[0136] Optionally, determining a first transmission rate, a resolution, a frame rate, and an encoding quality of the base layer according to the target transmission rate;

[0137] The resolution, frame rate and encoding quality of each enhancement layer are determined according to the target transmission rate, the first transmission rate and the first number.

[0138] Optionally, the encoder starts encoding after determining the first encoding parameter of the base layer, the number of enhancement layers and the second encoding parameter;

[0139] The network abstraction layer units output by the encoder are separated by group of pictures.

[0140] Optionally, the network abstraction layer includes at least one group of pictures GoP.

[0141] It should be noted that the frames and slices within the same GoP are related, but different GoPs can be considered unrelated.

[0142] The video data processing method of the embodiment of the present invention determines the encoding parameters of the base layer through the target transmission rate, thereby determining the number and encoding parameters of the enhancement layers. It can adjust the number and encoding parameters of the enhancement layers as the transmission rate changes, thereby ensuring that the transmission rate matches the video quality during the playback of the video streaming media.

[0143] like Figure 7 As shown, a video data processing method according to an embodiment of the present invention is applied to a network device, comprising the following steps:

[0144] Step 701, obtaining status data of the target terminal;

[0145] Step 702, determining a target transmission rate between the network device and the target terminal according to the state data and the load data of the network device;

[0146] Step 703, sending the target transmission rate to the encoder, where the target transmission rate is used by the encoder to determine a first parameter of the video to be played;

[0147] Step 704, sending the transport block size TBS allocated for the video to be played to the loader;

[0148] Step 705: Receive the network abstraction layer unit NALU sent by the loader.

[0149] In one embodiment of the present invention, the network device determines the expected average transmission rate based on the resource size allocated to the corresponding video service that can be provided to the target terminal in the future period of time, so that the encoder determines the encoding parameters according to the target transmission rate.

[0150] In one embodiment of the present invention, the baseband processing unit (Base Band Unite, BBU) of the network device determines the transport block size TBS allocated to each picture group according to its own current channel resource utilization and the channel condition (channel state information (Channel State Information, CSI) and channel quality indicator (Channel quality indicator, CQI)) of the target terminal.

[0151] The baseband processing unit BBU of the network device sends the loaded NALU. If the NALU corresponding to the GoP can be fully loaded in a scheduled transmission block, no additional operation is required; if multiple scheduling is required and some NALUs cannot be received correctly, the BBU needs to notify the NALU loader of the NALU information that cannot be accepted, and the NALU removes all NALUs associated with the NALUs that failed to send.

[0152] Optionally, the transport block size TBS may be the sum of transport block sizes sent to the target terminal in multiple times.

[0153] The video data processing method of the embodiment of the present invention can determine the target transmission rate between the network device and the target terminal by acquiring the status data of the target terminal and combining it with the load data of the network device; by sending the transmission block size TBS allocated to the picture group GoP to the loader, the loader can send the network abstraction layer unit to the network device according to the transmission block size, so that the network device sends the network abstraction layer unit to the target terminal for video playback.

[0154] Optionally, obtaining status data of the target terminal includes:

[0155] The transmission bandwidth, channel quality parameters and the highest modulation mode supported by the target terminal are obtained.

[0156] Optionally, the method for acquiring the load data of the network device includes:

[0157] Determine the terminals accessing the network device, and determine the number of terminals accessing the network device, the number of activated terminals accessing the network device, and the air interface resource utilization rate of each terminal.

[0158] Optionally, determining the target transmission rate according to the state data and the load data of the network device includes:

[0159] Determining an average transmission rate from the network device to the target terminal according to the state data and the load data of the network device;

[0160] The target transmission rate is determined according to the average transmission rate.

[0161] Optionally, the target transmission rate is equal to or slightly greater than the average transmission rate.

[0162] Optionally, the sending to the loader a transport block size TBS allocated for the video to be played includes:

[0163] Acquire a channel condition of the target terminal, the channel condition including: channel state information and a channel quality indicator;

[0164] According to the channel condition of the target terminal and the channel resource utilization rate of the network device, a transmission block size TBS allocated to each group of pictures GoP corresponding to the video to be played is determined.

[0165] Optionally, the status data includes:

[0166] The transmission bandwidth, channel quality parameters and the highest modulation mode supported by the target terminal;

[0167] The load data includes:

[0168] The number of access terminals, the number of activated terminals and the air interface resource utilization rate of the network device.

[0169] Optionally, the network device is mainly responsible for assisting the encoder in determining the number of enhancement layers and encoding parameters of each enhancement layer, and also needs to provide a TBS size expected to be allocated to each GoP;

[0170] For the loader, the network device also needs to ensure that the TB size indicated by the corresponding digital copyright unique identifier (DCI) in the physical downlink control channel (PDCCH) is as close as possible to the GoP TBS;

[0171] After the loader completes the loading of the NALU, the network device is responsible for sending the corresponding NALU to the target terminal.

[0172] like Figure 8 As shown, a video data processing method according to an embodiment of the present invention is applied to a loader and includes the following steps:

[0173] Step 801, receiving a transmission block size TBS allocated for a video to be played sent by a network device;

[0174] Step 802, receiving at least one network abstraction layer unit NALU sent by the encoder;

[0175] Step 803: Send the network abstraction layer unit NALU to the network device according to the transmission block size TBS.

[0176] Optionally, the loader loads the group of pictures according to the transport block size TBS.

[0177] In one embodiment of the present invention, the baseband processing unit (Base Band Unite, BBU) of the network device determines the transport block size TBS allocated to each picture group according to its own current channel resource utilization and the channel condition (channel state information (Channel State Information, CSI) and channel quality indicator (Channel quality indicator, CQI)) of the target terminal.

[0178] Optionally, sending the network abstraction layer unit NALU to the network device according to the transport block size TBS includes:

[0179] Determine whether to discard a network abstraction layer unit NALU according to a transmission block size TBS allocated by the network device for the video to be played;

[0180] If it is determined that the network abstraction layer unit NALU needs to be discarded, discard at least one target network abstraction layer unit NALU in the network abstraction layer unit;

[0181] The remaining network abstraction layer unit NALUs after discarding the target network abstraction layer unit NALU in the network abstraction layer unit NALU are sent to the network device.

[0182] Optionally, the transport block size TBS allocated for the video to be played includes:

[0183] The transport block size TBS allocated to each group of pictures GoP corresponding to the video to be played.

[0184] In one embodiment of the present invention, if the GoP TBS size is smaller than the total NALU size, a portion of the NALU is discarded to prioritize the base layer.

[0185] Optionally, the method further comprises:

[0186] Receiving a first network abstraction layer unit sent by a network device indicating failure in sending to the target terminal;

[0187] Determine and discard a second network abstraction layer unit according to the first network abstraction layer unit;

[0188] The second network abstraction layer unit is a network abstraction layer unit that is dependent on the first network abstraction layer unit and is among the network abstraction layer units that have not yet been sent to the network device.

[0189] Optionally, determining whether to discard a network abstraction layer unit NALU according to a transport block size TBS allocated by the network device to each picture group includes:

[0190] Determine the association relationship between each of the network abstraction layer units NALU;

[0191] And according to the association relationship and the transmission block size TBS allocated by the network device to each picture group, it is determined whether the network abstraction layer unit needs to be discarded.

[0192] Optionally, determining the association relationship between each of the network abstraction layer units NALU can be understood as follows: since one Slice corresponds to only one NALU, the association relationship between the NALUs is equivalent to the relationship between the corresponding Slices, and the Slice association information can be shown in Table 1:

[0193] Table 1

[0194]

[0195]

[0196] In one embodiment of the present invention, since Slice and its related information are also included in the NALU itself, the loader can parse the NALU by itself and obtain the Slice related information. If the encoder supports it, the Slice related information can also be displayed and passed to the loader to facilitate it to determine whether it needs to be discarded. After obtaining the Slice related information, the NALU loader loads it according to the TBS of the corresponding GoP indicated by the BBU. If the indicated GoP TBS is larger than the size of all NALUs of the corresponding GoP, the NALU is not discarded.

[0197] Optionally, the method for determining the target network abstraction layer unit NALU includes:

[0198] Determine the target network abstraction layer unit according to the second parameter of the network abstraction layer unit NALU;

[0199] The second parameter includes at least one of the following: a level of a corresponding enhancement layer, a degree of dependence on other network abstraction layer units, a corresponding region of interest, and a number of historical discards.

[0200] Optionally, the selection strategy of the target network abstraction layer unit is:

[0201] The enhancement layer with a higher level is discarded first;

[0202] Slices that are highly dependent on other Slices (NALU) are discarded first;

[0203] Non-regions of interest (ROI) are discarded first. In the absence of the ROI, the central area and the three-divided points are considered as the ROI;

[0204] Previously, the corresponding position of the Slice (NALU) that has been discarded is discarded first.

[0205] The video data processing method of the embodiment of the present invention determines the target network abstraction layer unit to be discarded in the network abstraction layer unit according to the second parameter of the network abstraction layer unit, thereby adapting to the change of bandwidth and the discarding strategy.

[0206] The logic diagram of the video data processing method according to an embodiment of the present invention is as follows: Fig. 9 As shown:

[0207] The network device (RAN) determines a target transmission rate according to the acquired target terminal status data and the load data of the network device; and sends the target transmission rate to the encoder (Codec);

[0208] The encoder determines, according to the target transmission rate, a first encoding parameter of a base layer corresponding to the video to be played, or the first encoding parameter and the number of enhancement layers and a second encoding parameter;

[0209] The loader (NALU Loader) receives the transmission block size allocated to the picture group sent by the network device; and receives at least one network abstraction layer unit sent by the encoder, and loads the network abstraction layer unit according to the transmission block size and the at least one network abstraction layer unit; at the same time, when the size of the transmission block is smaller than the total size of the network abstraction layer unit, selects the target network abstraction layer unit to be discarded according to the selection strategy of the network abstraction layer unit, and sends the network abstraction layer unit that has discarded the target network abstraction layer unit to the network device; and forwards it to the target terminal through the network device;

[0210] In the case that the target terminal cannot correctly receive all of the network abstraction layer units, the network device sends information (Slice identifier) ​​of the first network abstraction layer unit that was not correctly received to the loader, and the loader discards the first network abstraction layer unit that was not correctly received, and sends the network abstraction layer unit excluding the target network abstraction layer unit and the first network abstraction layer unit to the network device again.

[0211] The process of the video data processing method according to an embodiment of the present invention is as follows: Fig.10 As shown:

[0212] The encoder Codes encodes according to the target transmission rate, determines the base layer and the enhancement layer, as well as the resolution of the base layer (e.g., 720p), the number of enhancement layers and the resolution (e.g., two enhancement layers, whose resolutions are 1080p and 2160p respectively), encapsulates the base layer or the base layer and the enhancement layer through the network abstraction layer, and sends them to the loader Loader;

[0213] The loader receives the transmission block size of the picture group sent by the BBU of the network device, and receives the network abstraction layer unit sent by the encoder, and loads the network abstraction layer unit; wherein, when the channel quality (determined according to the target transmission rate) is good, all the data (all the network abstraction layer units) are loaded, and when the channel quality is general, part of the enhancement layer (the network abstraction layer unit corresponding to the enhancement layer) is selectively discarded, and when the channel quality is poor, only the data of the base layer (the network abstraction layer unit of the base layer) is loaded.

[0214] like Fig.11 As shown, a video data processing device 1100 according to an embodiment of the present invention is applied to an encoder, including:

[0215] The first receiving module 1101 is used to receive a target transmission rate sent by a network device;

[0216] A first determining module 1102 is configured to determine a first parameter of a video to be played according to the target transmission rate, wherein the first parameter includes: a first encoding parameter of a base layer, or the first parameter includes: a first encoding parameter of the base layer and a first number and a second encoding parameter of an enhancement layer;

[0217] The encoding module 1103 is used to encode the video to be played according to the first parameter to obtain an output video code stream including at least one network abstraction layer unit NALU;

[0218] The first sending module 1104 is configured to send the network abstraction layer unit to a loader.

[0219] like Fig.12 As shown, a video data processing device 1200 according to an embodiment of the present invention is applied to a network device, including:

[0220] The acquisition module 1201 is used to acquire the status data of the target terminal;

[0221] A second determination module 1202, configured to determine a target transmission rate according to the state data and the load data of the network device;

[0222] A second sending module 1203, configured to send the target transmission rate to an encoder, where the target transmission rate is used by the encoder to determine a first parameter of a video to be played;

[0223] The third sending module 1204 is used to send the transport block size TBS allocated for the video to be played to the loader;

[0224] The second receiving module 1205 is used to receive the network abstraction layer unit NALU sent by the loader.

[0225] like Fig.13 As shown, a video data processing device 1300 according to an embodiment of the present invention is applied to a loader, and includes:

[0226] The third receiving module 1301 is used to receive a transmission block size TBS allocated for the video to be played sent by a network device;

[0227] The fourth receiving module 1302 is used to receive at least one network abstraction layer unit NALU sent by the encoder;

[0228] The fourth sending module 1303 is used to send the network abstraction layer unit NALU to the network device according to the transmission block size TBS.

[0229] like Fig.14 As shown, an encoder 1400 according to an embodiment of the present invention includes a processor 1410 and a transceiver 1420, wherein:

[0230] The transceiver is used to receive the target transmission rate sent by the network device;

[0231] The processor is used to determine a first parameter of the video to be played according to the target transmission rate, where the first parameter includes: a first encoding parameter of a base layer, or the first parameter includes: a first encoding parameter of the base layer and a first number and a second encoding parameter of an enhancement layer;

[0232] According to the first parameter, the video to be played is encoded to obtain an output video code stream including at least one network abstraction layer unit NALU;

[0233] The transceiver is further configured to send the network abstraction layer unit to a loader.

[0234] like Fig.15 As shown, a network device 1500 according to an embodiment of the present invention includes a processor 1510 and a transceiver 1520, wherein:

[0235] The transceiver is used to obtain status data of the target terminal;

[0236] The processor is used to determine a target transmission rate according to the state data and the load data of the network device;

[0237] The transceiver is also used to send the target transmission rate to the encoder, and the target transmission rate is used by the encoder to determine the first parameter of the video to be played;

[0238] Sending to the loader a transport block size TBS allocated for the video to be played;

[0239] Receives the Network Abstraction Layer Unit (NALU) sent by the loader.

[0240] like Fig.16 As shown, a loader 1600 according to an embodiment of the present invention includes a transceiver 1620, wherein:

[0241] The transceiver is used to receive a transmission block size TBS allocated for a video to be played and sent by a network device;

[0242] Receiving at least one network abstraction layer unit NALU sent by the encoder;

[0243] The network abstraction layer unit NALU is sent to the network device according to the transport block size TBS.

[0244] A terminal device according to another embodiment of the present invention, such as Fig.17 As shown, it includes a transceiver 1710, a processor 1700, a memory 1720, and a program or instruction stored in the memory 1720 and executable on the processor 1700; when the processor 1700 executes the program or instruction, the above-mentioned method for processing video data is implemented.

[0245] The transceiver 1710 is used to receive and send data under the control of the processor 1700 .

[0246] Among them, Fig.17 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 1700 and various circuits of memory represented by memory 1720 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 1710 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. For different user devices, the user interface 1730 may also be an interface capable of externally and internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0247] The processor 1700 is responsible for managing the bus architecture and general processing, and the memory 1720 can store data used by the processor 1700 when performing operations.

[0248] A readable storage medium according to an embodiment of the present invention stores a program or instruction thereon. When the program or instruction is executed by a processor, the steps in the video data processing method described above are implemented and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0249] The processor is the processor in the terminal device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0250] It should be further explained that the terminals described in this specification include but are not limited to smart phones, tablet computers, etc., and many of the functional components described are called modules in order to more particularly emphasize the independence of their implementation methods.

[0251] In the embodiment of the present invention, module can be implemented with software so that it can be executed by various types of processors. For example, an executable code module of an identification can include one or more physical or logical blocks of computer instructions, for example, it can be constructed as an object, process or function. Nevertheless, the executable code of the identified module does not need to be physically located together, but can include different instructions stored in different positions, and when these instructions are logically combined together, it constitutes a module and realizes the specified purpose of the module.

[0252] In fact, executable code module can be a single instruction or many instructions, and can even be distributed on a plurality of different code segments, distributed among different programs, and distributed across a plurality of memory devices. Similarly, operating data can be identified in the module, and can be implemented and organized in the data structure of any appropriate type according to any appropriate form. The operating data can be collected as a single data set, or can be distributed in different locations (including on different storage devices), and can only be present on a system or network as an electronic signal at least in part.

[0253] When a module can be implemented by software, considering the level of existing hardware technology, a person skilled in the art can build a corresponding hardware circuit to implement the corresponding function of the module that can be implemented by software without considering the cost. The hardware circuit includes a conventional very large scale integration (VLSI) circuit or gate array and existing semiconductors such as logic chips, transistors, or other discrete components. The module can also be implemented by a programmable hardware device, such as a field programmable gate array, a programmable array logic, a programmable logic device, etc.

[0254] The above exemplary embodiments are described with reference to the accompanying drawings, and many different forms and embodiments are feasible without departing from the spirit and teachings of the present invention. Therefore, the present invention should not be constructed as a limitation of the exemplary embodiments proposed herein. More specifically, these exemplary embodiments are provided so that the present invention will be perfect and complete, and the scope of the present invention will be conveyed to those who are familiar with the technology. In these figures, the component sizes and relative sizes may be exaggerated for clarity. The terms used here are only based on the purpose of describing specific exemplary embodiments and are not intended to be limiting. As used herein, unless the text clearly indicates otherwise, the singular forms "one", "an" and "the" are intended to include these multiple forms. It will be further understood that the terms "including" and / or "comprising" when used in this specification indicate the presence of the features, integers, steps, operations, components and / or components, but do not exclude the presence or increase of one or more other features, integers, steps, operations, components, components and / or their groups. Unless otherwise indicated, when stated, a range of values ​​includes the upper and lower limits of that range and any subranges therebetween.

[0255] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A video data processing method, applied to an encoder, characterized in that: include: Receiving a target transmission rate sent by a network device, wherein the target transmission rate is a transmission rate between the network device and a target terminal; Determine a first parameter of the video to be played according to the target transmission rate, the first parameter comprising: a first encoding parameter of a base layer, or the first parameter comprising: a first encoding parameter of the base layer and a first number of enhancement layers and a second encoding parameter of each enhancement layer; According to the first parameter, the video to be played is encoded to obtain an output video code stream including at least one network abstraction layer unit NALU; The network abstraction layer unit is sent to the loader.

2. The video data processing method according to claim 1, characterized in that: The method for determining the first coding parameter of the base layer includes: Determine a preset ratio of the target transmission rate as a first transmission rate of the base layer; The first encoding parameter is determined according to the first transmission rate.

3. The video data processing method according to claim 2, characterized in that: The method for determining the first number and the second coding parameter of the enhancement layer includes: Determining a first number of enhancement layers according to the target transmission rate and the first transmission rate; The second encoding parameter is determined according to the target transmission rate, the first transmission rate, and the first number.

4. The video data processing method according to claim 1 or 2, characterized in that: The first encoding parameter and the second encoding parameter include: Resolution, frame rate, and encoding quality; The highest level of the enhancement layer corresponds to the highest resolution of the video to be played.

5. The video data processing method according to claim 1, characterized in that: The network abstraction layer comprises at least one group of pictures GoP.

6. A video data processing method, applied to a network device, characterized in that: include: Obtain status data of the target terminal; Determining a target transmission rate between the network device and the target terminal according to the state data and the load data of the network device; Sending the target transmission rate to an encoder, wherein the target transmission rate is used by the encoder to determine a first parameter of a video to be played; Sending to the loader a transport block size TBS allocated for the video to be played; Receives the Network Abstraction Layer Unit (NALU) sent by the loader.

7. The video data processing method according to claim 6, characterized in that: The determining the target transmission rate according to the state data and the load data of the network device includes: Determining an average transmission rate from the network device to the target terminal according to the state data and the load data of the network device; The target transmission rate is determined according to the average transmission rate.

8. The video data processing method according to claim 6, characterized in that: The sending to the loader a transport block size TBS allocated for the video to be played includes: Acquire a channel condition of the target terminal, the channel condition including: channel state information and a channel quality indicator; According to the channel condition of the target terminal and the channel resource utilization rate of the network device, a transmission block size TBS allocated to each group of pictures GoP corresponding to the video to be played is determined.

9. The video data processing method according to claim 6, characterized in that: The status data includes: The transmission bandwidth, channel quality parameters and the highest modulation mode supported by the target terminal; The load data includes: The number of access terminals, the number of activated terminals and the air interface resource utilization rate of the network device.

10. A video data processing method, applied to a loader, characterized in that: include: Receive a transmission block size TBS allocated for the video to be played sent by a network device; Receiving at least one network abstraction layer unit NALU sent by the encoder; The network abstraction layer unit NALU is sent to the network device according to the transport block size TBS.

11. The video data processing method according to claim 10, characterized in that: The sending the network abstraction layer unit NALU to the network device according to the transmission block size TBS includes: Determine whether to discard a network abstraction layer unit NALU according to a transmission block size TBS allocated by the network device for the video to be played; If it is determined that the network abstraction layer unit NALU needs to be discarded, discard at least one target network abstraction layer unit NALU in the network abstraction layer unit; The remaining network abstraction layer unit NALUs after discarding the target network abstraction layer unit NALU in the network abstraction layer unit NALU are sent to the network device.

12. The video data processing method according to claim 10 or 11, characterized in that: The transmission block size TBS allocated for the video to be played includes: The transport block size TBS allocated to each picture group corresponding to the video to be played.

13. The video data processing method according to claim 11, characterized in that: The determining whether to discard a network abstraction layer unit NALU according to a transport block size TBS allocated by the network device to each picture group includes: Determine the association relationship between each of the network abstraction layer units NALU; And according to the association relationship and the transmission block size TBS allocated by the network device to each picture group, it is determined whether the network abstraction layer unit needs to be discarded.

14. The video data processing method according to claim 11, characterized in that: The method for determining the target network abstraction layer unit NALU includes: Determine the target network abstraction layer unit according to the second parameter of the network abstraction layer unit NALU; The second parameter includes at least one of the following: a level of a corresponding enhancement layer, a degree of dependence on other network abstraction layer units, a corresponding region of interest, and a number of historical discards.

15. A video data processing device, applied to an encoder, characterized in that: include: A first receiving module, used for receiving a target transmission rate sent by a network device; A first determination module is used to determine a first parameter of the video to be played according to the target transmission rate, wherein the first parameter includes: a first encoding parameter of a base layer, or the first parameter includes: a first encoding parameter of the base layer and a first number and a second encoding parameter of an enhancement layer; An encoding module, configured to encode the video to be played according to the first parameter to obtain an output video code stream including at least one network abstraction layer unit NALU; The first sending module is used to send the network abstraction layer unit to the loader.

16. A video data processing device, applied to a network device, characterized in that: include: An acquisition module, used to acquire the status data of the target terminal; A second determination module, configured to determine a target transmission rate according to the state data and the load data of the network device; A second sending module, used to send the target transmission rate to an encoder, where the target transmission rate is used by the encoder to determine a first parameter of a video to be played; A third sending module, used for sending a transport block size TBS allocated for the video to be played to the loader; The second receiving module is used to receive the network abstraction layer unit NALU sent by the loader.

17. A video data processing device, applied to a loader, characterized in that: include: A third receiving module is used to receive a transmission block size TBS allocated for the video to be played and sent by a network device; A fourth receiving module, configured to receive at least one network abstraction layer unit NALU sent by the encoder; The fourth sending module is used to send the network abstraction layer unit NALU to the network device according to the transmission block size TBS.

18. An encoder, characterized in that: include: transceivers and processors; The transceiver is used to receive the target transmission rate sent by the network device; The processor is used to determine a first parameter of the video to be played according to the target transmission rate, where the first parameter includes: a first encoding parameter of a base layer, or the first parameter includes: a first encoding parameter of the base layer and a first number and a second encoding parameter of an enhancement layer; According to the first parameter, the video to be played is encoded to obtain an output video code stream including at least one network abstraction layer unit NALU; The transceiver is further configured to send the network abstraction layer unit to a loader.

19. A network device, characterized in that: include: transceivers and processors; The transceiver is used to obtain status data of the target terminal; The processor is used to determine a target transmission rate according to the state data and the load data of the network device; The transceiver is also used to send the target transmission rate to the encoder, and the target transmission rate is used by the encoder to determine the first parameter of the video to be played; Sending to the loader a transport block size TBS allocated for the video to be played; Receives the Network Abstraction Layer Unit (NALU) sent by the loader.

20. A loader, characterized in that: include: Transceiver; The transceiver is used to receive a transmission block size TBS allocated for a video to be played and sent by a network device; Receiving at least one network abstraction layer unit NALU sent by the encoder; The network abstraction layer unit NALU is sent to the network device according to the transport block size TBS.

21. A terminal device, comprising: A transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; characterized in that when the processor executes the program or instruction, the video data processing method as described in any one of claims 1 to 5 is implemented; or, the video data processing method as described in any one of claims 6 to 9 is implemented; or, the video data processing method as described in any one of claims 10 to 14 is implemented.

22. A readable storage medium having a program or instruction stored thereon, characterized in that: When the program or instruction is executed by the processor, it implements the steps in the video data processing method as described in any one of claims 1 to 5; or, implements the steps in the video data processing method as described in any one of claims 6 to 9; or, implements the steps in the video data processing method as described in any one of claims 10 to 14.