Subscription processing method, system, server and storage medium for a video

Abstract

The application relates to the technical field of data processing, and discloses a video subscription processing method and system, a server and a storage medium. After subscription requests with the same resolution and frame rate of a same video source are aggregated, a second subscription request is obtained, the bit rate corresponding to the second subscription request is determined according to a second subscription parameter, the video source is encoded according to the bit rate corresponding to the second subscription request to obtain video data, and when the video data is forwarded to a video receiving end, first video data is obtained according to the bit rate in the subscription request of the video receiving end and the real-time bit rate of the video stream. The application can consider the video quality of other video receiving ends, avoid the network of one video receiving end from affecting the picture quality of all video receiving ends, and further improve the picture quality of all video receiving ends.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a video subscription processing method, system, server, and storage medium. Background Technology

[0002] In the SFU architecture, the video receiver requests the desired video feed from the server via a subscription request. This subscription request includes the video sender's identifier, resolution, frame rate, and bitrate. During a multi-party conference, multiple video receivers may submit subscription requests to the server for the same video sender. Upon receiving these requests, the server forwards them all to the sending end of the video feed, which then determines which video feed to send based on these subscription requests.

[0003] Currently, the existing technical solution involves aggregating subscription requests for the same video source, resolution, and frame rate on the SFU server. After aggregation, the server only needs to forward one subscription request to the sending end of the video source. However, in this existing aggregation scheme, to ensure that the video stream emitted by the video source can be received by every video receiver, the minimum encoding bitrate is used for subscriptions to the same video source and resolution after aggregation. But since the video encoding bitrate is positively correlated with the video stream quality, this existing technical solution can lead to a situation where a poor network connection at one video receiver affects the picture quality of all video receivers. Summary of the Invention

[0004] This application provides a video subscription processing method, system, server, and storage medium. By aggregating subscription requests with the same resolution and frame rate from the same video source, a second subscription request is obtained. Then, based on the second subscription parameters, the bitrate corresponding to the second subscription request is determined, allowing the video source to encode according to the bitrate of the second subscription request to obtain video data. When forwarding the video data to the video receiver, the first video data is obtained according to the bitrate in the video receiver's subscription request and the real-time bitrate of the video stream. This application can take into account the video quality of other video receivers, avoiding the impact of a poor network connection at one video receiver on the picture quality of all video receivers, thereby improving the picture quality of all receivers.

[0005] The embodiments of this application provide the following technical solutions:

[0006] In a first aspect, embodiments of this application provide a video subscription processing method, applied to a server, wherein the server is communicatively connected to multiple video receivers and multiple video sources, and the method includes:

[0007] Obtain multiple first subscription requests sent by each video receiver, wherein each video receiver corresponds to a first subscription request;

[0008] Based on the first subscription parameters, multiple first subscription requests are classified to obtain multiple sets of first subscription requests;

[0009] Aggregate the first subscription requests in each set of first subscription requests to obtain multiple second subscription requests;

[0010] Based on the second subscription parameters, determine the bit rate corresponding to each second subscription request;

[0011] Multiple second subscription requests are sent to the video source so that the video source encodes the video data according to the second subscription requests and their corresponding bitrates, resulting in multiple encoded video data.

[0012] Receive encoded video data sent by the video source and obtain the real-time bit rate of each encoded video data;

[0013] Based on the real-time bitrate, each encoded video data is processed to obtain multiple first video data, and the first video data is forwarded to the video receiving end corresponding to the first subscription request.

[0014] In some embodiments, the first subscription parameters include resolution, frame rate, and video source identifier, with one video source corresponding to one video source identifier. Based on the first subscription parameters, multiple first subscription requests are categorized to obtain multiple sets of first subscription requests, including:

[0015] At fixed time intervals, based on the video source identifier corresponding to the first subscription request, multiple first subscription requests are divided into different video source sets, wherein the video source identifier corresponding to the first subscription requests in each video source set is the same;

[0016] Based on the resolution and frame rate corresponding to each first subscription request, the first subscription requests in the video source set are classified to obtain multiple sets of first subscription requests, wherein the first subscription requests in the first subscription request set have the same resolution and frame rate.

[0017] In some embodiments, the method further includes:

[0018] Get the subscription bit rate corresponding to the first subscription request in the first subscription request set;

[0019] The maximum value among the subscription bit rates corresponding to the first subscription request is determined as the maximum subscription bit rate, and the minimum value among the subscription bit rates corresponding to the first subscription request is determined as the minimum subscription bit rate;

[0020] The proportion that can be dropped is determined based on the resolution and frame rate corresponding to the first set of subscription requests;

[0021] The maximum supportable bit rate is calculated based on the minimum subscription bit rate and the discardable ratio, where the maximum supportable bit rate = minimum subscription bit rate / (1 - discardable ratio).

[0022] In some embodiments, the second subscription parameters include a maximum subscription bit rate and a maximum supported bit rate. Determining the bit rate corresponding to each second subscription request based on the second subscription parameters includes:

[0023] If the maximum bit rate that can be supported is greater than the maximum subscription bit rate, then the maximum subscription bit rate will be determined as the bit rate corresponding to the second subscription request.

[0024] If the maximum supported bit rate is less than or equal to the maximum subscription bit rate, then the maximum supported bit rate will be determined as the bit rate corresponding to the second subscription request.

[0025] In some embodiments, the bitrate of the encoded video data is less than or equal to the bitrate of the second subscription request. The first subscription request set includes multiple first subscription requests. The first subscription requests in the first subscription request set are aggregated to obtain the second subscription request, including:

[0026] All first subscription requests in each first subscription request set are aggregated into a second subscription request, wherein one second subscription request corresponds to one first subscription request set, and the resolution and frame rate of the second subscription request are the same as the resolution and frame rate of the first subscription requests in its corresponding first subscription request set.

[0027] In some embodiments, one encoded video data corresponds to one first subscription request. Each encoded video data is processed according to its real-time bitrate to obtain multiple first video data sets, including:

[0028] The difference between the real-time bitrate of the encoded video data and the subscription bitrate corresponding to the first subscription request for the video data is calculated and used as the first difference.

[0029] Based on the first difference, determine the video data that can be discarded in the encoded video data;

[0030] The first video data is obtained based on the discardable video data.

[0031] In some embodiments, the encoded video data includes several data packets, and the first video data is obtained based on the discardable video data, including:

[0032] Set up a sending channel, in which the server communicates with the video receiver through the sending channel. The sending channel is used to acquire data packets and send the data packets to the video receiver.

[0033] The data packets corresponding to the discardable video data are obtained through the transmission channel;

[0034] The priority of a data packet is determined based on its encoding level.

[0035] Based on the priority of the data packets, data packets are discarded in order of increasing priority until the real-time bit rate corresponding to the data packet in the current sending channel is less than or equal to the subscription bit rate corresponding to the first subscription request.

[0036] The data packets in the current transmission channel are obtained, and the video data corresponding to the data packets in the current transmission channel is identified as the first video data, thereby obtaining the first video data.

[0037] Secondly, embodiments of this application provide a server, including:

[0038] At least one processor; and

[0039] A memory that is communicatively connected to at least one processor; wherein,

[0040] The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform a video subscription processing method as described in the first aspect.

[0041] Thirdly, embodiments of this application provide a video subscription processing system, the system comprising:

[0042] Server, video receiver, and video source;

[0043] The video receiver connects to the server to send the first subscription request.

[0044] The server communicates with multiple video receivers and multiple video sources respectively to execute the video subscription processing method as described in any of the first aspects;

[0045] The video source communication connection is used to receive the second subscription request sent by the server; encode the video data according to the second subscription request and its corresponding bit rate to obtain the encoded video data; and send the encoded video data to the server.

[0046] Fourthly, embodiments of this application provide a non-volatile computer-readable storage medium storing computer-executable instructions for causing an electronic device to perform a video subscription processing method as described in the first aspect.

[0047] The beneficial effects of the embodiments of this application are as follows: Unlike the prior art, the embodiments of this application provide a video subscription processing method applied to a server. The server is communicatively connected to multiple video receiving ends and multiple video sources. The method includes: acquiring multiple first subscription requests sent by each video receiving end, wherein each video receiving end corresponds one-to-one with a first subscription request; classifying the multiple first subscription requests according to first subscription parameters to obtain multiple sets of first subscription requests; aggregating the first subscription requests in each set of first subscription requests to obtain multiple second subscription requests; determining the bitrate corresponding to each second subscription request according to second subscription parameters; sending the multiple second subscription requests to the video sources, so that the video sources encode video data according to the second subscription requests and their corresponding bitrates to obtain multiple encoded video data; receiving the encoded video data sent by the video sources and acquiring the real-time bitrate of each encoded video data; processing each encoded video data according to the real-time bitrate to obtain multiple first video data, and forwarding the first video data to the video receiving end corresponding to the first subscription request.

[0048] By aggregating subscription requests with the same resolution and frame rate from the same video source, a second subscription request is obtained. Then, based on the second subscription parameters, the bitrate corresponding to the second subscription request is determined so that the video source can encode according to the bitrate corresponding to the second subscription request to obtain video data. When forwarding the video data to the video receiver, the first video data is obtained according to the bitrate in the video receiver's subscription request and the real-time bitrate of the video stream. This application can take into account the video quality of other video receivers, avoid the poor network of one video receiver affecting the picture quality of all video receivers, and thus improve the picture quality of all receivers. Attached Figure Description

[0049] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0050] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0051] Figure 2 This is a schematic flowchart of a video subscription processing method provided in an embodiment of this application;

[0052] Figure 3 yes Figure 2 A detailed flowchart of step S202 in the process;

[0053] Figure 4 yes Figure 2 A detailed flowchart of step S203 in the process;

[0054] Figure 5 This is a schematic diagram of a process for calculating the maximum supported bit rate provided in an embodiment of this application;

[0055] Figure 6 yes Figure 2 A detailed flowchart of step S204 in the process;

[0056] Figure 7 yes Figure 2 A detailed flowchart of step S207 in the process;

[0057] Figure 8 yes Figure 7 A detailed flowchart of step S273 in the process;

[0058] Figure 9 This is a schematic diagram of the structure of a video subscription processing system provided in an embodiment of this application;

[0059] Figure 10 This is a schematic diagram of the structure of a server provided in an embodiment of this application.

[0060] Explanation of icon numbers:

[0061] label name label name 100 Application scenarios 10 Video receiver 20 server 30 Video source 901 Video subscription processing system 201 processor 202 memory Detailed Implementation

[0062] To facilitate understanding of this application, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that when an element is described as "fixed to" another element, it can be directly on the other element, or one or more intermediate elements may exist between them. When an element is described as "connected to" another element, it can be directly connected to the other element, or one or more intermediate elements may exist between them. The terms "vertical," "horizontal," "left," "right," and similar expressions used in this specification are for illustrative purposes only.

[0063] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0064] In the SFU architecture, the video receiver requests the desired video feed from the server via a subscription request. This subscription request includes the video sender's identifier, resolution, frame rate, and bitrate. During a multi-party conference, multiple video receivers may submit subscription requests to the server for the same video sender. Upon receiving these requests, the server forwards them all to the sending end of the video feed, which then determines which video feed to send based on these subscription requests.

[0065] Currently, the existing technical solution involves aggregating subscription requests for the same video source, resolution, and frame rate on the SFU server. After aggregation, the server only needs to forward one subscription request to the sending end of the video source. However, in this existing aggregation scheme, to ensure that the video stream emitted by the video source can be received by every video receiver, the minimum encoding bitrate is used for subscriptions to the same video source and resolution after aggregation. But since the video encoding bitrate is positively correlated with the video stream quality, this existing technical solution can lead to a situation where a poor network connection at one video receiver affects the picture quality of all video receivers.

[0066] To address the aforementioned deficiencies, this application provides a video subscription processing method. The method includes: acquiring multiple first subscription requests sent by various video receivers, wherein each video receiver corresponds one-to-one with a first subscription request; classifying the multiple first subscription requests according to first subscription parameters to obtain multiple sets of first subscription requests; aggregating the first subscription requests in each set of first subscription requests to obtain multiple second subscription requests; determining the bitrate corresponding to each second subscription request according to second subscription parameters; sending the multiple second subscription requests to a video source, so that the video source encodes video data according to the second subscription requests and their corresponding bitrates to obtain multiple encoded video data; receiving the encoded video data sent by the video source and acquiring the real-time bitrate of each encoded video data; processing each encoded video data according to the real-time bitrate to obtain multiple first video data, and forwarding the first video data to the video receiver corresponding to the first subscription request. By aggregating subscription requests with the same resolution and frame rate from the same video source, a second subscription request is obtained. Then, based on the second subscription parameters, the bitrate corresponding to the second subscription request is determined so that the video source can encode according to the bitrate corresponding to the second subscription request to obtain video data. When forwarding the video data to the video receiver, the first video data is obtained according to the bitrate in the video receiver's subscription request and the real-time bitrate of the video stream. This application can take into account the video quality of other video receivers, avoid the poor network of one video receiver affecting the picture quality of all video receivers, and thus improve the picture quality of all receivers.

[0067] The technical solution of this application is described in detail below with reference to the accompanying drawings:

[0068] Please see Figure 1 , Figure 1 This is a schematic diagram of an application scenario provided in an embodiment of this application.

[0069] like Figure 1 As shown, the application environment 100 includes: a video receiver 10, a server 20, and a video source 30.

[0070] The video receiver 10 can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, wearable device, ultra-mobile personal computer (UMPC), non-mobile electronic device, netbook, or personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, network attached storage (NAS), personal computer (PC), television (TV), etc.

[0071] The server 20 includes an SFU (Selective Forwarding Unit) server. An SFU is a network architecture commonly used in video conferencing and real-time communication systems, widely applied in scenarios such as multi-user video calls, live streaming, and online education. Its core function is to establish a connection with each client, receive audio and video data from them, and achieve one-to-many capability, i.e., forwarding the video / audio stream from one client to other clients. It can be understood that there can be one or more servers 102, and multiple servers can form a server cluster. For example, the server cluster may include: a first server, a second server, ..., an Nth server. Alternatively, the server cluster may be a cloud computing service center, which includes several servers. The servers in this embodiment include, but are not limited to, tower servers, rack servers, blade servers, and cloud servers. Preferably, the server is a cloud server (Elastic Compute Service, ECS).

[0072] The video source 30 includes, but is not limited to, cameras, video recorders, and video file servers; however, this application embodiment does not impose specific limitations on these.

[0073] In practical applications, this scenario can include multiple video receivers and multiple video sources. Figure 1 Taking a video receiver and a video source as an example, specifically, the video receiver 10 is connected to the server 20 via network communication, and the server 20 is connected to the video source 30 via network communication. This network includes wired and / or wireless networks. It can be understood that this network includes wireless networks such as 2G, 3G, 4G, 5G, Wi-Fi, and Bluetooth, and may also include wired networks such as serial cables and Ethernet cables.

[0074] In this embodiment, the video receiver 10 is used to send a first subscription request to the server; the server 20 is used to obtain the first subscription request sent by the video receiver; classify the first subscription request according to the first subscription parameters to obtain a first subscription request set; aggregate the first subscription requests in the first subscription request set to obtain a second subscription request; determine the bitrate corresponding to the second subscription request according to the second subscription parameters; send the second subscription request to the video source so that the video source encodes the video data according to the second subscription request and its corresponding bitrate to obtain encoded video data; receive the encoded video data sent by the video source and obtain the real-time bitrate of the encoded video data; process the encoded video data according to the real-time bitrate of the video data to obtain first video data, and forward the first video data to the video receiver corresponding to the first subscription request; the video source 30 is used to receive the second subscription request sent by the server; encode the video data according to the second subscription request and its corresponding bitrate to obtain encoded video data; and send the encoded video data to the server.

[0075] Please see Figure 2 , Figure 2 This is a flowchart illustrating a video subscription processing method provided in an embodiment of this application.

[0076] The video subscription processing method is applied to the server; specifically, the method is executed by one or at least two processors of the server.

[0077] like Figure 2 As shown, the subscription processing method for this video includes:

[0078] Step S201: Obtain multiple first subscription requests sent by each video receiving end.

[0079] Specifically, the server communicates with multiple video receivers and multiple video sources. The video receivers first send an initial subscription request to the server. Then, the server continuously receives initial subscription requests from different video receivers. There is a one-to-one correspondence between the video receivers and the initial subscription requests, and each initial subscription request corresponds to a video source, resolution, and frame rate. For example, if video receiver A sends an initial subscription request a to video source B, then the video receiver corresponding to initial subscription request a is A, and the video source corresponding to initial subscription request a is B.

[0080] Step S202: Based on the first subscription parameters, classify the multiple first subscription requests to obtain a set of multiple first subscription requests.

[0081] For details, please refer to [link / reference]. Figure 3 , Figure 3 yes Figure 2 A detailed flowchart of step S202 in the process.

[0082] like Figure 3 As shown, step S202: Based on the first subscription parameters, the multiple first subscription requests are classified to obtain a set of multiple first subscription requests, including:

[0083] Step S221: At fixed time intervals, based on the video source identifier corresponding to the first subscription request, divide the multiple first subscription requests into different video source sets.

[0084] Specifically, after receiving multiple first subscription requests from multiple video receivers, the server creates a subscription sending thread. This thread sets a subscription processing time window and categorizes the multiple first subscription requests into multiple sets. The subscription processing time window has a fixed time period, and at fixed intervals, the first subscription requests are divided into different video source sets based on the video source they correspond to. Each video source set contains the same video source for each first subscription request; that is, one video source corresponds to one video source identifier, and each first subscription request carries a video source identifier. The server retrieves the video source identifier from each first subscription request to identify the video source corresponding to the first subscription request. The system identifies video sources and categorizes first subscription requests based on these sources. First subscription requests with the same video source identifier are grouped into the same video source set. It's important to note that this fixed time period can be set according to actual needs; this application does not limit it. For example, the fixed time period can be set to 0.1 seconds. For instance, assuming the server receives three first subscription requests within the current fixed time period: first subscription request A, first subscription request B, and first subscription request C, where the video source corresponding to first subscription request A is D, the video source corresponding to first subscription request B is D, and the video source corresponding to first subscription request C is E, then first subscription requests A and B are grouped into the same video source set, and first subscription request C is grouped into another video source set, resulting in two video source sets.

[0085] Step S222: Classify the first subscription requests in the video source set according to the resolution and frame rate corresponding to each first subscription request to obtain multiple sets of first subscription requests.

[0086] Specifically, each first subscription request corresponds to a resolution and a frame rate. After dividing each first subscription request into different video source sets according to the video source corresponding to the first subscription request, each video source set needs to be further classified according to the resolution and frame rate corresponding to each first subscription request. For each set of video sources containing multiple first subscription requests, the server obtains the resolution and frame rate corresponding to each first subscription request. Based on the resolution and frame rate, the server categorizes the multiple first subscription requests in the video source set. First subscription requests with the same resolution and frame rate are grouped into the same first subscription request set, while first subscription requests with different resolutions and frame rates are grouped into different first subscription request sets, resulting in multiple first subscription request sets. For example, suppose a video source set has 3 first subscription requests: first subscription request A has a resolution of 1080P and a frame rate of 30 frames per second; first subscription request B has a resolution of 720P and a frame rate of 30 frames per second; and first subscription request C has a resolution of 720P and a frame rate of 50 frames per second. Since the resolutions and frame rates of these 3 first subscription requests are all different, the video source set is divided into 3 first subscription request sets, and these 3 first subscription requests are grouped into different first subscription request sets, thus obtaining 3 first subscription request sets.

[0087] In this embodiment of the application, by classifying the first subscription requests according to video source, resolution and frame rate to obtain a first subscription request set, this application can improve the efficiency of subsequent processing of subscription requests.

[0088] Step S203: Aggregate the first subscription requests in each first subscription request set to obtain multiple second subscription requests.

[0089] For details, please refer to [link / reference]. Figure 4 , Figure 4 yes Figure 2 A detailed flowchart of step S203 is shown.

[0090] like Figure 4 As shown, step S203: Aggregate the first subscription requests in each first subscription request set to obtain multiple second subscription requests, including:

[0091] Step S231: Aggregate all first subscription requests in each first subscription request set into a second subscription request, wherein one second subscription request corresponds to one first subscription request set, and the resolution and frame rate of the second subscription request are the same as the resolution and frame rate of the first subscription requests in its corresponding first subscription request set.

[0092] Specifically, after the above division based on video source, resolution, and frame rate, the video source, resolution, and frame rate corresponding to the first subscription requests in each first subscription request set are all the same. Therefore, all the first subscription requests in each first subscription request set can be aggregated into a second subscription request to reduce the number of subscription requests sent and improve the processing efficiency of the video source. Here, one second subscription request corresponds to one first subscription request set, and the resolution and frame rate of the second subscription request are the same as the resolution and frame rate of the first subscription requests in its corresponding first subscription request set.

[0093] In this embodiment of the application, by aggregating all first subscription requests in each first subscription request set into a second subscription request, the application can reduce the number of subscription requests, thereby improving the processing efficiency of the system.

[0094] In this embodiment of the application, a first subscription request set corresponds to a second subscription request. After obtaining multiple second subscription requests, it is also necessary to calculate the corresponding maximum support bit rate based on the minimum subscription bit rate and the discardable ratio corresponding to the first subscription request set. The maximum support bit rate refers to the maximum bit rate that can be achieved on the basis of meeting the minimum bit rate during the video encoding process. It can be understood that the maximum support bit rate is the highest video quality that can be provided without discarding any video data.

[0095] Please refer to the following: Figure 5 , Figure 5 This is a schematic diagram of a process for calculating the maximum supported bit rate provided in an embodiment of this application.

[0096] like Figure 5 As shown, the process for calculating the maximum supported bit rate includes:

[0097] Step S501: Obtain the subscription bit rate corresponding to the first subscription request in the first subscription request set.

[0098] Specifically, the subscription bitrate refers to the amount of data transmitted per unit of time. It is an important indicator of video stream quality; a higher bitrate generally indicates better video quality, but also consumes more network bandwidth. The first subscription requests in the first subscription request set are all subscription requests for the same video source at the same resolution and frame rate, differing only in their subscription bitrate. Each first subscription request corresponds to a specific subscription bitrate. After obtaining multiple first subscription request sets, the subscription bitrates corresponding to all first subscription requests in each set are retrieved to facilitate the subsequent determination of the maximum and minimum subscription bitrates for all first subscription requests within that set.

[0099] Step S502: Determine the maximum value among the subscription bit rates corresponding to the first subscription request as the maximum subscription bit rate, and determine the minimum value among the subscription bit rates corresponding to the first subscription request as the minimum subscription bit rate.

[0100] Specifically, the maximum and minimum subscription bitrates corresponding to all first subscription requests in the first subscription request set are obtained. The maximum subscription bitrate is determined as the maximum subscription bitrate, and the minimum subscription bitrate is determined as the minimum subscription bitrate. It should be noted that the maximum and minimum subscription bitrates refer to the maximum and minimum subscription bitrates corresponding to the first subscription requests in a set of first subscription requests. For example, if there are three sets of first subscription requests, there are three maximum and minimum subscription bitrates, meaning that each set of first subscription requests corresponds to one maximum and one minimum subscription bitrate.

[0101] Step S503: Determine the discardable ratio based on the resolution and frame rate corresponding to the first subscription request set.

[0102] Specifically, droppable ratios include, but are not limited to, SVC droppable ratios, SHVC droppable ratios, and AVC droppable ratios. SVC (Scalable Video Coding) is an extension of the H.264 standard that allows video streams to be encoded into multiple layers, each corresponding to different resolutions, qualities, and frame rates. SHVC (Scalable HEVC) is a further development of SVC, based on the HEVC (High Efficiency Video Coding) standard. SHVC also supports layered coding, meeting the needs of different network environments and terminal devices. AVC (Advanced Video Coding)... Advanced Video Coding (AVC) is a high-efficiency video compression standard jointly developed by ITU-T and ISO / IEC. It is widely used in various video application scenarios and achieves high compression ratios and excellent visual experiences through advanced technologies (such as motion compensation and transform coding). SVC droppable ratio refers to the proportion of frames or data that can be dropped or ignored in the total number of frames or data during SVC encoding. SHVC droppable ratio refers to the proportion of frames or data that can be dropped or ignored in the total number of frames or data during SHVC encoding. AVC droppable ratio refers to the proportion of frames or data that can be dropped or ignored in the total number of frames or data during AVC encoding. This selective dropping is to optimize coding efficiency, reduce transmission bit rate, or adapt to different network bandwidths.

[0103] In this embodiment of the application, before determining the dropable ratio, it is necessary to set a mapping relationship between the resolution and frame rate corresponding to the first subscription request set and the dropable ratio. This mapping relationship is used to directly map and obtain the SVC dropable ratio through the resolution and frame rate corresponding to the first subscription request set. For example, assume there are the following three mapping relationships: when the resolution is 1080P and the frame rate is 30 frames, the corresponding SVC dropable ratio is 0.5; when the resolution is 1080P and the frame rate is 60 frames, the corresponding SVC dropable ratio is 0.6; when the resolution is 1080P and the frame rate is 15 frames, the corresponding SVC dropable ratio is 0.8. If the resolution and frame rate corresponding to a certain first subscription request set are 1080P and 60 frames respectively, then the dropable ratio can be directly determined to be 0.6 through the above mapping relationship.

[0104] It should be noted that the mapping relationship can be set according to actual needs. For example, the mapping relationship can be set according to the current network conditions or encoding strategy. This application does not limit this.

[0105] Step S504: Calculate the maximum supportable bit rate based on the minimum subscription bit rate and the discardable ratio.

[0106] Specifically, during the encoding process, data in the media stream is prioritized according to importance. Discarding low-priority data does not affect the subsequent processing of high-priority data. For example, taking the SVC droppable ratio as an example, different SVC droppable ratios exist at different resolutions and frame rates to ensure video quality. The maximum supportable bitrate refers to the maximum bitrate achievable while meeting the minimum bitrate during video encoding. It is calculated based on the minimum subscription bitrate and the droppable ratio, where: Maximum Supportable Bitrate = Minimum Subscription Bitrate / (1 - Droppable Ratio). Here, "1 - Droppable Ratio" represents the proportion of non-droppable media. The formula for calculating the maximum supportable bitrate means dividing the minimum bitrate in the first subscription request set by the proportion of non-droppable media to obtain the maximum supportable bitrate.

[0107] For example, taking the SVC drop rate as an example, assuming the minimum bit rate in a certain first subscription request set is 1000kbps and the maximum subscription bit rate is 1800kbps, and the SVC drop rate obtained through the mapping relationship between resolution, frame rate and SVC drop rate is 0.5, then the maximum supportable bit rate calculated by the above formula is 1000kbps / (1-0.5) = 2000kbps; as another example, if the minimum bit rate in a certain first subscription request set is 1000kbps and the maximum subscription bit rate is 2200kbps, and the SVC drop rate obtained through the mapping relationship between resolution, frame rate and SVC drop rate is 0.6, then the maximum supportable bit rate calculated by the above formula is 1000kbps / (1-0.6) = 2500kbps.

[0108] In this embodiment of the application, by determining the discardable ratio based on the resolution and frame rate corresponding to the first subscription request set, this application can facilitate the subsequent discarding of low-priority video data.

[0109] Step S204: Determine the bit rate corresponding to each second subscription request based on the second subscription parameters.

[0110] For details, please refer to [link / reference]. Figure 6 , Figure 6 yes Figure 2 A detailed flowchart of step S204 is shown.

[0111] like Figure 6 As shown, step S204: Determine the bit rate corresponding to each second subscription request based on the second subscription parameters, including:

[0112] Step S241: Obtain the maximum subscription bit rate and the maximum supported bit rate.

[0113] Specifically, taking a second subscription request as an example, we obtain the set of first subscription requests corresponding to the second subscription request, and determine the maximum subscription bit rate and the maximum supportable bit rate corresponding to the set of first subscription requests. The maximum subscription bit rate is the maximum value of the subscription bit rate corresponding to the first subscription request in the set of first subscription requests, and the maximum supportable bit rate is calculated based on the minimum subscription bit rate and the discardable ratio.

[0114] Step S242: Determine whether the maximum subscription bit rate is greater than the maximum supported bit rate.

[0115] Specifically, determine whether the maximum subscription bit rate is greater than the maximum supported bit rate. If the maximum subscription bit rate is greater than the maximum supported bit rate, proceed to step S244; if the maximum subscription bit rate is less than or equal to the maximum supported bit rate, proceed to step S243.

[0116] Step S243: Determine the maximum subscription bit rate as the bit rate corresponding to the second subscription request.

[0117] Specifically, if the maximum subscription bit rate is less than or equal to the maximum supported bit rate, then the maximum subscription bit rate will be determined as the bit rate corresponding to the second subscription request.

[0118] Step S244: Determine the maximum supported bit rate as the bit rate corresponding to the second subscription request.

[0119] Specifically, if the maximum subscription bit rate is greater than the maximum supported bit rate, then the maximum supported bit rate will be determined as the bit rate corresponding to the second subscription request.

[0120] In this embodiment, the maximum support bitrate is obtained by amplifying the minimum subscription bitrate. If the maximum support bitrate is used directly, video receivers with requests containing the minimum subscription bitrate will need to encode the video before forwarding it to the corresponding video receiver during actual media transmission, depending on the subscription status. This results in poor video quality for the receiving video receiver. Therefore, it is necessary to determine whether the maximum subscription bitrate is greater than the maximum support bitrate. When the maximum subscription bitrate is greater than the maximum support bitrate, the maximum support bitrate is determined as the bitrate corresponding to the second subscription request; when the maximum subscription bitrate is less than or equal to the maximum support bitrate, the maximum subscription bitrate is determined as the bitrate corresponding to the second subscription request. It is understandable that by determining the bitrate corresponding to the second subscription request based on the maximum subscription bitrate and the maximum support bitrate, this application can avoid providing video at excessively high bitrates, which could cause video stuttering, while ensuring the picture quality for video receivers with lower subscription bitrates.

[0121] Step S205: Send multiple second subscription requests to the video source so that the video source encodes the video data according to the second subscription requests and their corresponding bitrates to obtain multiple encoded video data.

[0122] Specifically, after the server aggregates the second subscription requests and determines the bitrate corresponding to each second subscription request, it sends multiple second subscription requests to the video source. The video source generates video data based on these second subscription requests, enabling it to encode the video data according to the second subscription requests and their corresponding bitrates. In essence, each second subscription request includes multiple first subscription requests. The video source encodes the video data according to the bitrates corresponding to each first and second subscription request, resulting in multiple encoded video data sets. One first subscription request corresponds to one encoded video data set. During encoding, the bitrate corresponding to the second subscription request is used as the maximum bitrate limit for encoding. However, due to factors such as image complexity, the actual real-time bitrate of the encoded video stream is not a fixed value but fluctuates below the bitrate specified in the final subscription request. The encoding process converts the video data into a format suitable for network transmission and adjusts its quality to match the bitrate specified in the second subscription request. The bitrate of the encoded video data is less than or equal to the bitrate of the second subscription request. In the embodiments of this application, the video source refers to the device or system that provides the original video data. The video source can be an electronic device such as a mobile phone or computer, or a camera, video recorder, video file server, etc. This application does not limit the type of video source.

[0123] Step S206: Receive the encoded video data sent by the video source and obtain the real-time bit rate of each encoded video data.

[0124] Specifically, the server receives encoded video data from the video source. This encoded video data consists of several data packets. A sliding window is used to obtain the real-time bitrate of each encoded video data packet. The real-time bitrate refers to the transmission rate of the video data at a specific moment, reflecting the bandwidth usage of the video stream at that current time. The sliding window method is used to statistically analyze the real-time bitrate of the encoded video data. A sliding window is a data statistics method that uses a fixed-size time window to slide across the data stream to analyze the characteristics of the data within the window. First, the window value of the sliding window needs to be set. The window value refers to the size of the sliding window. As time progresses, the sliding window moves across the video stream. At each point in time, the window contains all video data packets received within the current window value. Then, the sizes (in bytes) of all video stream data packets within the window are summed to obtain the total amount of data within the window. Finally, the total amount of data within the window is divided by the window size to obtain the real-time bitrate, i.e., Real-time bitrate = Total amount of data within the window / Window value. Bitrate is usually expressed in units such as bits per second (bps), kilobits per second (kbps), or megabits per second (Mbps). For example, when the window value is 500 milliseconds, all encoded video data packets received within the last 500 milliseconds are summed in bytes, and then divided by 500 milliseconds to obtain the real-time bitrate of the encoded video data.

[0125] In this embodiment, the real-time bit rate of the encoded video data is statistically analyzed using a sliding window, which can reflect the bandwidth usage of the video stream at the current time point and provide important reference information for the transmission and control of the video stream.

[0126] Step S207: Process each encoded video data according to the real-time bitrate to obtain multiple first video data, and forward the first video data to the video receiving end corresponding to the first subscription request.

[0127] For details, please refer to [link / reference]. Figure 7 , Figure 7 yes Figure 2 A detailed flowchart of step S207 is shown.

[0128] like Figure 7 As shown, step S207: Based on the real-time bitrate, each encoded video data is processed to obtain multiple first video data, and the first video data is forwarded to the video receiving end corresponding to the first subscription request, including:

[0129] Step S271: Calculate the difference between the real-time bitrate of the encoded video data and the subscription bitrate corresponding to the first subscription request for the video data, and use it as the first difference.

[0130] Specifically, one encoded video data corresponds to one first subscription request. After obtaining the real-time bitrate of the encoded video data, the first difference is calculated based on the real-time bitrate of the encoded video data and the subscription bitrate of the first subscription request corresponding to the video data. The formula for calculating the difference between the real-time bitrate of the encoded video data and the subscription bitrate of the first subscription request corresponding to the video data is: First difference = Real-time bitrate of the encoded video data - Subscription bitrate of the first subscription request corresponding to the video data.

[0131] Step S272: Based on the first difference, determine the video data that can be discarded in the encoded video data.

[0132] Specifically, for a video receiver with a low subscription bitrate, it can only receive the bitrate carried in its first subscription request. When the real-time bitrate of the video data is higher than the bitrate carried in the first subscription request sent by the video receiver, the excess part needs to be discarded. In this embodiment, the video data that can be discarded in the encoded video data is determined according to the first difference. The first difference is the part of the real-time bitrate of the encoded video data that exceeds the subscription bitrate of the first subscription request corresponding to the video data. The video data that can be discarded is the video data corresponding to the first difference.

[0133] Step S273: Obtain the first video data based on the discardable video data.

[0134] In this embodiment, the video source is used to encode video content at multiple quality levels, enabling flexible transmission and playback of video data based on network bandwidth, terminal processing capabilities, or user needs. Taking SVC (Scalable Video Coding) as an example, SVC is a video coding standard. SVC provides temporally scalable, spatially scalable, and quality-scalable coding capabilities. Specifically, SVC encodes the video to generate a base layer, and then generates multiple enhancement layers according to settings. The base layer provides the most basic video quality, ensuring that all decoders can decode this content, which contains the core information of the video stream, guaranteeing a minimum acceptable viewing quality. Enhancement layers provide higher temporal, spatial, or quality resolution than the base layer. Each enhancement layer depends on all layers below it and can provide finer details and higher quality.

[0135] In this embodiment, based on the principle of the aforementioned video encoding mechanism, if a bitrate limitation occurs during transmission, the video stream generated by video encoding can discard the video stream of the enhancement layer according to priority to ensure network transmission. By determining the discardable video data in the encoded video data based on the first difference, and obtaining the first video data based on the discardable video data, wherein the first video data = encoded video data - discardable video data, this application can adaptively adjust the video data according to the subscription bitrate, thereby improving the smoothness of video playback.

[0136] For details, please refer to [link / reference]. Figure 8 , Figure 8 yes Figure 7 A detailed flowchart of step S273 is shown.

[0137] like Figure 8 As shown, step S273: Based on the discardable video data, obtain the first video data, including:

[0138] Step S2731: Set up the sending channel.

[0139] Specifically, a sending channel is set up, and all data packets from the video source are sent to the sending channel. The server communicates with the video receiver through the sending channel. The sending channel refers to the data transmission path between the server and the video receiver. It is mainly used to manage the encapsulation and scheduling of encoded video data and send the data to the video receiver according to a specific protocol. The type of the sending channel includes, but is not limited to, Real-time Streaming Protocol (RTSP / RTP), Real-time Messaging Protocol (RTMP), and HTTP Streaming (HLS / HTTP-FLV). In this embodiment, the sending channel is used to acquire data packets and send them to the video receiver.

[0140] Step S2732: Obtain the data packet corresponding to the encoded video data through the transmission channel.

[0141] Specifically, the data packets corresponding to the encoded video data are obtained through the transmission channel. These data packets include those corresponding to discardable video data. When the transmission channel detects that the real-time bit rate of the current video data is greater than the subscription bit rate of the first subscription request for the video data, a data packet discarding mechanism is triggered to obtain the data packets corresponding to the discardable video data.

[0142] Step S2733: Determine the priority of the data packets corresponding to the discardable video data based on the encoding level of the data packets corresponding to the discardable video data.

[0143] Specifically, the encoding level of the data packets corresponding to discardable video data refers to the encoding level pre-set during the encoding process of the video source. Taking SVC (Scalable Video Codec) as an example, the encoding level in SVC is set through three dimensions: temporal domain, spatial domain, and quality domain. This encoding level is positively correlated with priority. Among them, temporal scalability refers to achieving frame rate gradation by setting different reference frames. For example, a video stream can be encoded into multiple frame rate levels, such as full frame rate, half frame rate, 1 / 3 frame rate, etc. During transmission, different frame rate video streams can be sent according to network conditions or the capabilities of the video receiver. Spatial domain scalability refers to achieving spatial gradation by encoding sub-streams of different resolutions. The video stream can be encoded into multiple resolution levels, such as high definition, standard definition, low resolution, etc. The video receiver can select the appropriate resolution level for decoding and playback according to its display capabilities or network bandwidth. Quality domain scalability refers to achieving quality gradation by encoding video streams of different quality levels at the same resolution and frame rate. It allows the video receiver to adjust the video quality according to network conditions or user needs while maintaining the same resolution and frame rate.

[0144] In this embodiment, the video source can selectively discard parts of the bitstream during transmission or at the video receiver to adapt to different network conditions or terminal requirements. For example, when the network is congested, the encoding level of the video data is determined according to the frame rate, resolution, or quality level of the video data. Low frame rate, low resolution, or low quality levels are determined as low encoding levels, and high frame rate, high resolution, or high quality levels are determined as high encoding levels. Furthermore, the priority of data packets at low encoding levels is determined as low priority, and the priority of data packets at high encoding levels is determined as high priority. This allows for the subsequent discarding of low-priority video streams (such as low frame rate, low resolution, or low quality levels) based on the priority of the data packets, ensuring that high-priority video streams (such as keyframes, high resolution, or high quality levels) can be transmitted and played smoothly.

[0145] Step S2734: Discard data packets in order of priority from low to high according to their priority, until the real-time bit rate corresponding to the data packet in the current sending channel is less than or equal to the subscription bit rate corresponding to the first subscription request.

[0146] Specifically, the transmission channel discards data packets in ascending order of priority based on the priority of the data packets corresponding to the discardable video data, until the real-time bit rate corresponding to the data packets in the current transmission channel is less than or equal to the subscription bit rate corresponding to the first subscription request. For example, assuming that the priority of the data packets corresponding to the discardable video data is divided into three levels: A, B, and C, where level A is greater than level B, and level B is greater than level C, then when the real-time bit rate corresponding to the data packets in the transmission channel is greater than the subscription bit rate corresponding to the first subscription request, the data packets of level C are discarded first, then the data packets of level B are discarded, and then the data packets of level A are discarded, until the real-time bit rate corresponding to the data packets in the current transmission channel is less than or equal to the subscription bit rate corresponding to the first subscription request.

[0147] In this embodiment, the priority of data packets is determined according to their encoding level. Data packets are then discarded sequentially from low to high priority until the real-time bit rate of the data packets in the current transmission channel is less than or equal to the subscription bit rate of the first subscription request. This application can improve the quality and efficiency of video transmission.

[0148] Step S2735: Obtain the data packet in the current transmission channel, and determine the video data corresponding to the data packet in the current transmission channel as the first video data, thereby obtaining the first video data.

[0149] Specifically, if the real-time bit rate corresponding to the data packet in the current sending channel is less than or equal to the subscription bit rate corresponding to the first subscription request, then the data packet in the current sending channel is obtained, and the video data corresponding to the data packet in the current sending channel is determined as the first video data. The first video data is then forwarded to the video receiving end corresponding to the first subscription request. It can be understood that there is a one-to-one correspondence between the first subscription request and the video receiving end. When multiple first subscription requests are aggregated into a second subscription request and sent to the video source, the video source will obtain the encoded video data corresponding to the first subscription request based on the multiple first subscription requests in the second subscription request. That is, one first subscription request corresponds to one encoded video data. After the server processes each encoded video data, the resulting multiple first video data also have a one-to-one correspondence with the first subscription request. In other words, one first video data corresponds to one first subscription request. The server will then send the first video data to the video receiving end corresponding to the first subscription request to complete the video subscription process.

[0150] In this embodiment, a video subscription processing method is provided, applied to a server. The server is communicatively connected to multiple video receivers and multiple video sources. The method includes: acquiring multiple first subscription requests sent by each video receiver, wherein each video receiver corresponds one-to-one with a first subscription request; classifying the multiple first subscription requests according to first subscription parameters to obtain multiple sets of first subscription requests; aggregating the first subscription requests in each set of first subscription requests to obtain multiple second subscription requests; determining the bitrate corresponding to each second subscription request according to the second subscription parameters; sending the multiple second subscription requests to the video sources so that the video sources encode video data according to the second subscription requests and their corresponding bitrates to obtain multiple encoded video data; receiving the encoded video data sent by the video sources and acquiring the real-time bitrate of each encoded video data; processing each encoded video data according to the real-time bitrate to obtain multiple first video data, and forwarding the first video data to the video receivers corresponding to the first subscription requests.

[0151] By aggregating subscription requests with the same resolution and frame rate from the same video source, a second subscription request is obtained. Then, based on the second subscription parameters, the bitrate corresponding to the second subscription request is determined, so that the video source encodes according to the bitrate corresponding to the second subscription request to obtain video data. When forwarding the video data to the video receiver, the first video data is obtained according to the bitrate in the video receiver's subscription request and the real-time bitrate of the video stream. This application can take into account the video quality of other video receivers, avoid the poor network of one video receiver affecting the picture quality of all video receivers, and thus improve the picture quality of all video receivers.

[0152] Please refer to the following: Figure 9 , Figure 9 This is a schematic diagram of the structure of a video subscription processing system provided in an embodiment of this application.

[0153] like Figure 9 As shown, the video subscription processing system 901 includes a video receiver 10, a server 20, and a video source 30.

[0154] The video receiver 10 is connected to the server 20. The video receiver 10 is used to send a first subscription request to the server, or to receive the first video data sent by the server.

[0155] Specifically, the video receiver first sends a first subscription request to the server so that the server can continuously obtain first subscription requests sent by different video receivers. Each first subscription request corresponds to a video source, resolution, frame rate and subscription bit rate. The video receiver is also used to receive first video data sent by the server, where one first video data corresponds to one first subscription request.

[0156] Server 20 communicates with multiple video receivers 10 and multiple video sources 30. Taking a server connected to one video receiver and one video source as an example, server 20 obtains multiple first subscription requests sent by each video receiver, with each video receiver corresponding to a specific first subscription request. Based on the first subscription parameters, it categorizes the multiple first subscription requests to obtain multiple sets of first subscription requests. It aggregates the first subscription requests in each set to obtain multiple second subscription requests. Based on the second subscription parameters, it determines the bitrate corresponding to each second subscription request. It sends the multiple second subscription requests to the video sources, enabling the video sources to encode video data according to the second subscription requests and their corresponding bitrates, resulting in multiple encoded video data sets. It receives the encoded video data sent by the video sources and obtains the real-time bitrate of each encoded video data set. Based on the real-time bitrate, it processes each encoded video data set to obtain multiple first video data sets and forwards the first video data sets to the video receivers corresponding to the first subscription requests.

[0157] Specifically, the server receives the first subscription request sent by the video receiver. The first subscription parameters include resolution, frame rate, and video source identifier. The video source identifier refers to specific information or symbols used to identify the source of the video data. For example, assuming the video source is a display device, the video source identifier is the device number of that display device. At fixed time intervals, the server first divides the first subscription request into different video source sets based on the video source identifier corresponding to the first subscription request, resulting in multiple video source sets. Then, based on the resolution and frame rate, the server classifies the first subscription requests in each video source set to group first subscription requests with the same resolution and frame rate into the same first subscription. Within the request set, first subscription requests with different resolutions and frame rates are divided into different first subscription request sets, resulting in multiple first subscription request sets. After the above division based on video source identifier, resolution, and frame rate, the first subscription requests in each first subscription request set have the same video source identifier, resolution, and frame rate. Therefore, all first subscription requests in each first subscription request set can be aggregated into a single second subscription request to reduce the number of subscription requests sent and improve the processing efficiency of the video source. The subscription bitrate corresponding to the first subscription requests in the first subscription request set is obtained, and the maximum subscription bitrate among the first subscription requests is selected. The maximum subscription bitrate is determined, and the minimum subscription bitrate among the first subscription requests is determined as the minimum subscription bitrate. Based on the resolution and frame rate corresponding to the first set of subscription requests, the drop-through ratio is determined. Based on the minimum subscription bitrate and the drop-through ratio, the maximum supportable bitrate is calculated, where the maximum supportable bitrate = minimum subscription bitrate / (1 - drop-through ratio). The second subscription parameters include the maximum subscription bitrate and the maximum supportable bitrate. If the maximum subscription bitrate is less than or equal to the maximum supportable bitrate, then the maximum subscription bitrate is determined as the bitrate corresponding to the second subscription request. If the maximum subscription bitrate is greater than the maximum supportable bitrate, then the drop-through ratio is determined as the maximum supportable bitrate. The maximum bitrate is determined to be the bitrate corresponding to the second subscription request; the second subscription request is sent to the video source so that the video source encodes the video data according to the second subscription request and its corresponding bitrate to obtain encoded video data; the encoded video data sent by the video source is received, and the real-time bitrate of the encoded video data is obtained; a first difference is determined based on the real-time bitrate of the encoded video data and the subscription bitrate corresponding to the first subscription request; based on the first difference, discardable video data in the encoded video data is determined; based on the discardable video data, the first video data is obtained and sent to the video receiving end.

[0158] The video source 30 is connected to the server 20. The video source 30 is used to receive the second subscription request sent by the server; encode the video data according to the second subscription request and its corresponding bit rate to obtain the encoded video data; and send the encoded video data to the server.

[0159] Specifically, after receiving the second subscription request from the server, the video source encodes the video data according to the second subscription request and its corresponding bitrate to obtain encoded video data. During encoding, the bitrate corresponding to the second subscription request is used as the maximum bitrate limit for encoding. However, due to factors such as image complexity, the actual real-time bitrate of the encoded video stream is not a fixed value but fluctuates below the bitrate in the final subscription request. The encoding process converts the video data into a format suitable for network transmission and adjusts its quality to match the bitrate specified in the second subscription request. The bitrate of the encoded video data is less than or equal to the bitrate corresponding to the second subscription request. In this embodiment, the video source refers to a device or system that provides the original video data. The video source can be an electronic device such as a mobile phone or computer, or a camera, recorder, video file server, etc. This application does not limit the type of video source.

[0160] Please refer to the following: Figure 10 , Figure 10 This is a schematic diagram of the structure of a server provided in an embodiment of this application.

[0161] like Figure 10 As shown, the server 20 includes one or more processors 201 and a memory 202. Wherein, Figure 10 Take a processor 201 as an example.

[0162] The processor 201 and the memory 202 can be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0163] Processor 201 is used to provide computing and control capabilities to control the server to perform corresponding tasks, such as controlling the server to perform the above-mentioned tasks. Figure 2A video subscription processing method is disclosed, comprising: acquiring multiple first subscription requests sent by various video receivers, wherein each video receiver corresponds one-to-one with a first subscription request; classifying the multiple first subscription requests according to first subscription parameters to obtain multiple sets of first subscription requests; aggregating the first subscription requests in each set of first subscription requests to obtain multiple second subscription requests; determining the bitrate corresponding to each second subscription request according to the second subscription parameters; sending the multiple second subscription requests to a video source, so that the video source encodes video data according to the second subscription requests and their corresponding bitrates to obtain multiple encoded video data; receiving the encoded video data sent by the video source and acquiring the real-time bitrate of each encoded video data; processing each encoded video data according to the real-time bitrate to obtain multiple first video data, and forwarding the first video data to the video receiver corresponding to the first subscription request.

[0164] By aggregating subscription requests with the same resolution and frame rate from the same video source, a second subscription request is obtained. Then, based on the second subscription parameters, the bitrate corresponding to the second subscription request is determined, so that the video source encodes according to the bitrate corresponding to the second subscription request to obtain video data. When forwarding the video data to the video receiver, the first video data is obtained according to the bitrate in the video receiver's subscription request and the real-time bitrate of the video stream. This application can take into account the video quality of other video receivers, avoid the poor network of one video receiver affecting the picture quality of all video receivers, and thus improve the picture quality of all video receivers.

[0165] Processor 201 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0166] The memory 202, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the video subscription processing method in the embodiments of this application. The processor 201 can implement the video subscription processing method in any of the above method embodiments by running the non-transitory software programs, instructions, and modules stored in the memory 202. Specifically, the memory 202 may include volatile memory (VM), such as random access memory (RAM); the memory 202 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), or other non-transitory solid-state storage devices; the memory 202 may also include combinations of the above types of memory.

[0167] Memory 202 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 202 may optionally include memory remotely located relative to processor 201, and these remote memories may be connected to processor 201 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0168] One or more modules are stored in memory 202. When executed by one or more processors 201, they perform the video subscription processing method in any of the above method embodiments, for example, the method described above. Figure 2 The steps shown.

[0169] In this embodiment, the server 20 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 20 may also include other components for implementing device functions, which will not be described in detail here.

[0170] This application also provides a non-volatile computer-readable storage medium, such as a memory including program code, which can be executed by a processor to complete the video subscription processing method described above. For example, the non-volatile computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), magnetic tape, floppy disk, or optical data storage device, etc.

[0171] This application also provides a computer program product comprising one or more lines of program code stored in a non-volatile computer-readable storage medium. A processor of an electronic device reads the program code from the non-volatile computer-readable storage medium and executes the program code to complete the method steps of the video subscription processing method provided in the above embodiments.

[0172] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program or program code related to hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0173] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, the program can include the processes of the embodiments of the above methods. The non-volatile computer-readable storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0174] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations as described above in different aspects of this application, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A video subscription processing method, characterized in that, Applied to a server, wherein the server is communicatively connected to multiple video receivers and multiple video sources, the method includes: Obtain multiple first subscription requests sent by each of the video receiving terminals, wherein each video receiving terminal corresponds to one of the first subscription requests; Based on the first subscription parameters, multiple first subscription requests are classified to obtain multiple sets of first subscription requests; Aggregate the first subscription requests in each of the first subscription request sets to obtain multiple second subscription requests; Based on the second subscription parameters, determine the bit rate corresponding to each second subscription request; Multiple second subscription requests are sent to the video source, so that the video source encodes the video data according to the second subscription requests and their corresponding bitrates to obtain multiple encoded video data. Receive the encoded video data sent by the video source, and obtain the real-time bit rate of each encoded video data; Based on the real-time bitrate, each encoded video data is processed to obtain multiple first video data, and the first video data is forwarded to the video receiving end corresponding to the first subscription request.

2. The method according to claim 1, characterized in that, The first subscription parameters include resolution, frame rate, and video source identifier, where one video source corresponds to one video source identifier. The step of classifying multiple first subscription requests according to the first subscription parameters to obtain multiple sets of first subscription requests includes: At fixed time intervals, based on the video source identifier corresponding to the first subscription request, multiple first subscription requests are divided into different video source sets, wherein the video source identifier corresponding to the first subscription request in each video source set is the same; Based on the resolution and frame rate corresponding to each first subscription request, the first subscription requests in the video source set are classified to obtain multiple sets of first subscription requests, wherein the first subscription requests in the first subscription request sets have the same resolution and frame rate.

3. The method according to claim 2, characterized in that, The method further includes: Obtain the subscription bit rate corresponding to the first subscription request in the first subscription request set; The maximum value among the subscription bit rates corresponding to the first subscription request is determined as the maximum subscription bit rate, and the minimum value among the subscription bit rates corresponding to the first subscription request is determined as the minimum subscription bit rate; The proportion of requests that can be discarded is determined based on the resolution and frame rate corresponding to the first set of subscription requests. The maximum supportable bit rate is calculated based on the minimum subscription bit rate and the discardable ratio, wherein the maximum supportable bit rate = the minimum subscription bit rate / (1 - the discardable ratio).

4. The method according to claim 3, characterized in that, The second subscription parameters include a maximum subscription bit rate and a maximum supported bit rate. Determining the bit rate corresponding to each second subscription request based on the second subscription parameters includes: If the maximum supported bit rate is greater than the maximum subscription bit rate, then the maximum subscription bit rate is determined as the bit rate corresponding to the second subscription request; If the maximum supported bit rate is less than or equal to the maximum subscription bit rate, then the maximum supported bit rate is determined as the bit rate corresponding to the second subscription request.

5. The method according to claim 1, characterized in that, The bitrate of the encoded video data is less than or equal to the bitrate of the second subscription request. The first subscription request set includes multiple first subscription requests. The step of aggregating the first subscription requests in the first subscription request set to obtain the second subscription request includes: All first subscription requests in each first subscription request set are aggregated into a second subscription request, wherein one second subscription request corresponds to one first subscription request set, and the resolution and frame rate of the second subscription request are the same as the resolution and frame rate of the first subscription requests in its corresponding first subscription request set.

6. The method according to claim 1, characterized in that, One encoded video data corresponds to one first subscription request. The process involves processing each encoded video data according to its real-time bitrate to obtain multiple first video data sets, including: The difference between the real-time bitrate of the encoded video data and the subscription bitrate corresponding to the first subscription request for the video data is calculated and used as the first difference. Based on the first difference, determine the video data that can be discarded in the encoded video data; The first video data is obtained based on the discardable video data.

7. The method according to claim 6, characterized in that, The encoded video data includes several data packets, and obtaining the first video data based on the discardable video data includes: A sending channel is set up, wherein the server communicates with the video receiving end through the sending channel, and the sending channel is used to acquire data packets and send the data packets to the video receiving end; The data packets corresponding to the encoded video data are obtained through the transmission channel; The priority of the data packets corresponding to the discardable video data is determined based on the encoding level of the data packets corresponding to the discardable video data. According to the priority of the data packets, the data packets are discarded in order of increasing priority until the real-time bit rate corresponding to the data packets in the current sending channel is less than or equal to the subscription bit rate corresponding to the first subscription request. The data packets in the current transmission channel are obtained, and the video data corresponding to the data packets in the current transmission channel is determined as the first video data, thereby obtaining the first video data.

8. A server, characterized in that, include: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the video subscription processing method as described in any one of claims 1-7.

9. A video subscription processing system, characterized in that, The system includes: Server, video receiver, and video source; The video receiving end is communicatively connected to the server and is used to send a first subscription request to the server; The server is communicatively connected to multiple video receiving terminals and multiple video sources, respectively, for executing the video subscription processing method as described in any one of claims 1-7; The video source is communicatively connected to the server and is used to receive a second subscription request sent by the server; encode the video data according to the second subscription request and its corresponding bitrate to obtain encoded video data; and send the encoded video data to the server.

10. A non-volatile computer-readable storage medium, characterized in that, The non-volatile computer storage medium stores a computer program or instructions, which, when executed, implement the video subscription processing method as described in any one of claims 1-7.

Images