A video encoding method, device and apparatus

Abstract

The embodiments of this specification disclose a method, device and apparatus for video encoding. By obtaining uploaded videos from each publishing end, wherein the uploaded videos include multiple different encoding resolutions and bit rates; respectively determining the experience quality indicators corresponding to the uploaded videos at different encoding bit rates; obtaining the downlink bandwidth of the jth subscriber, wherein 1≤j≤M; determining the preferred subscription combination of the jth subscriber according to the experience quality indicator and the downlink bandwidth, wherein the preferred subscription combination includes N layers of uploaded videos corresponding to N publishing ends, the sum of the encoding bit rates of the N layers of uploaded videos does not exceed the downlink bandwidth, and the sum of the experience quality indicators corresponding to the N layers of uploaded videos is the largest, thereby achieving the selection of the optimal subscription combination based on the principle of maximizing the experience quality indicator under the condition of satisfying the downlink bandwidth.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a method, device and equipment for video encoding. Background Art

[0002] With the development of technology, multi-person conference or video technology has been widely used. In this type of scenario, each publisher can use multi-stream technology (such as simulcast technology or SVC technology) to publish video streams with different encoding resolutions and bit rates. At the same time, each subscriber can also subscribe to the video streams sent by multiple publishers. Then, how to select the bit rate combination suitable for each subscriber from the uploaded videos of multiple publishers has become a problem that needs to be solved.

[0003] Based on this, a control scheme for video encoding with better user experience is needed. Summary of the invention

[0004] The embodiments of this specification provide a video encoding control method, apparatus, device and storage medium to solve the following technical problem: a video encoding control solution with better user experience is needed.

[0005] To solve the above technical problems, one or more embodiments of this specification are implemented as follows:

[0006] In a first aspect, an embodiment of the present specification provides a method for controlling video encoding, comprising: obtaining uploaded videos from each publishing end, wherein the uploaded videos include multiple layers of different encoding bit rates; respectively determining experience quality indicators corresponding to the uploaded videos at different encoding bit rates; obtaining the downlink bandwidth of the j-th subscriber, wherein 1≤j≤M; determining a preferred subscription combination of the j-th subscriber based on the experience quality indicator and the downlink bandwidth, wherein the preferred subscription combination includes N layers of uploaded videos corresponding to N publishing ends, the sum of the encoding bit rates of the N layers of uploaded videos does not exceed the downlink bandwidth, and the sum of the experience quality indicators corresponding to the N layers of uploaded videos is the largest.

[0007] In a second aspect, an embodiment of the present specification provides a control device for video encoding, comprising: a video acquisition module, which acquires uploaded videos from each publishing end, wherein the uploaded videos include multiple layers of different encoding bit rates; an indicator determination module, which respectively determines the experience quality indicators corresponding to the uploaded videos at different encoding bit rates; a bandwidth acquisition module, which acquires the downlink bandwidth of the j-th subscriber end, wherein 1≤j≤M; and a subscription combination determination module, which determines the preferred subscription combination of the j-th subscriber end according to the experience quality indicator and the downlink bandwidth.

[0008] In a third aspect, one or more embodiments of this specification provide an electronic device, including:

[0009] at least one processor; and,

[0010] a memory communicatively connected to the at least one processor; wherein,

[0011] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to the first aspect.

[0012] In a fourth aspect, an embodiment of the present specification provides a non-volatile computer storage medium storing computer executable instructions. When a computer reads the computer executable instructions in the storage medium, the instructions enable one or more processors to execute the method described in the first aspect.

[0013] At least one of the above-mentioned technical solutions adopted in one or more embodiments of the present specification can achieve the following beneficial effects: by obtaining uploaded videos from each publishing end, wherein the uploaded videos include multiple layers of different encoding bit rates; respectively determining the experience quality indicators corresponding to the uploaded videos at different encoding bit rates; obtaining the downlink bandwidth of the j-th subscriber, wherein 1≤j≤M; determining the preferred subscription combination of the j-th subscriber according to the experience quality indicator and the downlink bandwidth, wherein the preferred subscription combination includes N layers of uploaded videos corresponding to N publishing ends, the sum of the encoding bit rates of the N layers of uploaded videos does not exceed the downlink bandwidth, and the sum of the experience quality indicators corresponding to the N layers of uploaded videos is the largest, thereby achieving the selection of the optimal subscription combination based on the principle of maximizing the experience quality indicator under the condition of satisfying the downlink bandwidth, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0015] Figure 1 A schematic diagram of the system architecture involved in the embodiments of this specification;

[0016] Figure 2 A schematic diagram of a video encoding control process provided by an embodiment of this specification;

[0017] Figure 3 A schematic diagram of uploading a video at a publishing end provided in an embodiment of this specification;

[0018] Figure 4 A timing diagram of rate control between a publisher and a subscriber provided in an embodiment of this specification;

[0019] Figure 5 A schematic diagram of the structure of a video encoding control device provided in an embodiment of this specification;

[0020] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0021] The embodiments of the present specification provide a video coding control method, apparatus, device, and storage medium.

[0022] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0023] In a multi-person conference scenario, each publisher can use simulcast technology or SVC technology to publish video streams with multiple resolutions and bit rates, and can also subscribe to streams from multiple video sources. Figure 1 As shown, Figure 1 This is a schematic diagram of the system architecture involved in the embodiments of this specification. In the architecture of this schematic diagram, the room is responsible for signaling coordination and room management; the master is responsible for SFU load balancing and scheduling; the audio and video routing forwarding server (Selective Forwarding Unit, SFU) is responsible for the transfer of media streams, and the SFU uses a cascade structure to transmit video and other data. Everyone participating in the meeting can be both a publisher and a subscriber, and the video streams subscribed by each person may be different. Based on Figure 1 Based on the architecture shown, the embodiments of this specification provide a video encoding control solution to provide a better user experience.

[0024] In the first aspect, Figure 2 As shown, Figure 2 A schematic flow chart of a video encoding control process provided in an embodiment of this specification is applied to a system including N publishers and M subscribers, and includes the following steps:

[0025] S201: Obtain uploaded videos from each publishing end, wherein the uploaded videos include a plurality of different encoding bit rates.

[0026] As mentioned above, every user can upload video streams as a publisher. Figure 1 The publisher abcd shown in the figure publishes video streams to SFU. Each video stream can use simulcast technology or SVC technology, and the same video stream contains multiple different encoding bit rates. When SFU forwards it to the subscriber, it only selects one layer to send.

[0027] For example, when using simulcast technology, the publisher will encode the same frame at different resolutions and bit rates (for example, 360P / 300kbps, 540P / 600kbps, and 720P / 1200kbps), thereby generating multiple video streams with different encoding bit rates. Figure 3 As shown, Figure 3 A schematic diagram of a video upload provided by a publisher in an embodiment of this specification. In this schematic diagram, for the same frame of video, the publisher uses 360P, 720P and 1080P encoding resolutions at the same time, and uploads three video streams with different encoding bit rates. The SFU can select a video stream with a certain encoding bit rate to be downloaded to the subscriber based on the actual situation.

[0028] For another example, when using Scalable Video Coding (SVC) to provide a multi-layer coded stream, the base layer encodes the temporal, spatial and quality streams of the lowest layer; the enhancement layer uses the base layer as a starting point to encode additional information, thereby reconstructing the quality, encoding bit rate and temporal layer of the higher layer during the decoding process. By decoding the base layer and adjacent enhancement layers, the decoder can generate a video stream of a specific enhancement layer (the encoding bit rate of each enhancement layer is increased from the base layer).

[0029] In addition, since each user's needs may be different, although each user can be both a publisher and a subscriber, the numbers M and N are generally different. For example, in a multi-party conference, some users may turn off the upload function, in which case N is less than M.

[0030] S203: Determine experience quality indicators corresponding to different encoding bit rates of the uploaded video.

[0031] Quality of Experience (QoE) is the subjective feeling of end users on the service performance provided by mobile networks. It can express the end users' experience and feelings about services and networks in a nearly quantitative way, and reflect the gap between the quality of current services and networks and user expectations.

[0032] Generally speaking, subscribers want to watch videos with higher bitrates. However, the higher the bitrate, the greater the pressure on the subscriber's downlink bandwidth. Therefore, there is usually a nonlinear positive correlation between the bitrate and the quality of experience index. From experimental data, this relationship can be represented by a modified exponential curve.

[0033] Based on this, the relationship between the encoding bit rate and the experience quality index can be statistically analyzed in advance, and the functional relationship between the encoding bit rate and the experience quality index can be obtained by fitting regression. For example, some indirect indicators that are objective, repeatable, and quantifiable can be used to characterize the experience quality index. These indirect indicators may include, for example, video quality and bit rate, startup delay, freeze times and duration, and bit rate fluctuations.

[0034] For another example, in terms of specific calculation methods, for the experience quality indicators of such interactive network video streams involving multiple parties, evaluation and calculation can be performed using models such as media layer models, packet layer models, and bitstream layer models. When using the media layer model, different methods such as full reference, reduced reference, and no reference can be used for evaluation.

[0035] Based on these different representation methods, the corresponding relationship between the encoding bit rate and the experience quality index can be obtained, and then the functional relationship between the different encoding bit rates and the experience quality index of the uploaded video can be obtained by fitting. For example, the obtained functional relationship can be y=k+a*b^x. Among them, x is the encoding bit rate, y is the QoE value, and k, a and b are the correction coefficients obtained by fitting.

[0036] After obtaining the above functional relationship, the corresponding experience quality index value can be given for the encoding bit rate of any layer of the uploaded video. For example, assuming that for N publishers, each publisher uses 3 different encoding bit rates to upload video streams, then in this process, 3N experience quality indicators will be calculated.

[0037] S205: Obtain the downlink bandwidth of the j-th subscriber, where 1≤j≤M.

[0038] A bbr bandwidth detection algorithm such as Web Real-Time Communications (webrtc) can be used to detect the downlink bandwidth from SFU to the subscriber. That is, SFU sends a detection packet to the subscriber, and the subscriber determines its own downlink bandwidth (i.e., the transmission rate at which the subscriber can receive information from SFC) based on the detection packet, and uploads the downlink bandwidth to SFU.

[0039] S207: Determine a preferred subscription combination for the j-th subscriber according to the experience quality indicator and the downlink bandwidth.

[0040] After determining the downlink bandwidth and the quality of experience index of each layer of video, a preferred subscription combination of any j-th subscriber can be personalized and given to the subscriber. The preferred subscription combination should meet the following characteristics:

[0041] (1) The preferred subscription combination contains N layers of uploaded videos corresponding to N publishers. That is, the preferred subscription combination should select a layer of corresponding uploaded videos from each publisher. When there are N publishers, the preferred subscription combination should contain non-repeated N layers of uploaded videos.

[0042] For example, assuming that there are three publishers, and the video streams uploaded by them are (A11, A12, A13), (A21, A22, A23) and (A31, A32, A33), then a possible preferred subscription combination may be to select one element from each of the three sets mentioned above to form a preferred subscription combination (A11, A21, A32).

[0043] (2) The sum of the encoding bit rates of the N-layer uploaded videos does not exceed the downlink bandwidth.

[0044] Continuing with the previous example, assuming that the downlink bandwidth is 3M, the sum of the encoding bit rates of A11, A21 and A32 mentioned above should not exceed 3M.

[0045] (3) The sum of the experience quality indicators corresponding to the N layers of uploaded videos is the largest. Since the experience quality indicator of each layer of video stream has been determined, at this point, the problem is actually converted into maximizing the experience quality indicator in the preferred subscription combination based on constraints (1) and (2). This process can be implemented by designing different algorithms, for example, a greedy algorithm can be used.

[0046] Through the above-mentioned method, the maximum quality of experience index for the subscriber is obtained under the premise of satisfying the subscription and downlink bandwidth. By pushing the video stream combination corresponding to the preferred subscription combination, since the quality of experience index represents the user experience, the video stream combination corresponding to the selected preferred subscription combination can bring a better user experience to the user.

[0047] By obtaining uploaded videos from each publishing end, wherein the uploaded videos include multiple layers of different encoding bit rates; respectively determining the experience quality indicators corresponding to the uploaded videos at different encoding bit rates; obtaining the downlink bandwidth of the j-th subscriber, wherein 1≤j≤M; determining the preferred subscription combination of the j-th subscriber according to the experience quality indicator and the downlink bandwidth, wherein the preferred subscription combination includes N layers of uploaded videos corresponding to N publishing ends, the sum of the encoding bit rates of the N layers of uploaded videos does not exceed the downlink bandwidth, and the sum of the experience quality indicators corresponding to the N layers of uploaded videos is the largest, thereby achieving the selection of the optimal subscription combination based on the principle of maximizing the experience quality indicator under the condition of satisfying the downlink bandwidth, thereby improving the user experience.

[0048] In one implementation, when determining the preferred subscription combination of the j-th subscriber according to the experience quality indicator and the downlink bandwidth, the following algorithm may be used for calculation:

[0049] Determine the ratio of the experience quality index and the encoding rate corresponding to different encoding rates, sort them and generate a sequence K.

[0050] A maximum ratio is selected from each of the N publishers to generate a candidate subscription combination L, the selected maximum ratio is deleted from the sequence K, and the following cyclic judgment is performed: whether the sum of the encoding bit rates in L is less than the downlink bandwidth of the j-th subscriber is judged, if so, the maximum ratio is selected from the sequence K to replace the candidate ratio, and the selected maximum ratio is deleted from the sequence K, wherein the maximum ratio and the candidate ratio correspond to the same publisher; otherwise, the replacement is terminated, and the L at the time of termination is determined as the preferred subscription combination of the j-th subscriber.

[0051] For example, for a video stream (A11, A12, A13) uploaded by a publisher, assuming that the corresponding encoding bit rates are 300 kbps, 600 kbps and 1200 kbps, respectively, based on the aforementioned functional relationship, the calculated experience quality indicators are 600, 900 and 1200, respectively, then the ratios of the aforementioned experience quality indicator and the encoding bit rate can be determined as 600 / 300, 900 / 600, 1200 / 1200, so that the ratios corresponding to the video streams uploaded by the publisher are the following sequence (2, 1.5, 1), and this method can be used for calculation for each publisher to obtain the ratios corresponding to the video streams uploaded by each publisher, and finally sorted to obtain a sequence K, and K may be in the form of {(2, 1.5, 1), (2.2, 1.7, 1.4), (1.5, 1.3, 1.1)}.

[0052] Furthermore, the largest ratio can be selected from each of the N publishers to generate a candidate subscription combination L. That is, a combined candidate subscription combination (2, 2.2, 1.5) is formed, and the largest ratio is deleted and selected from the sequence K, that is, the form of K at this time becomes {(1.5, 1), (1.7, 1.4), (1.3, 1.1)}.

[0053] Then, based on the candidate subscription combination, it is calculated whether the corresponding encoding bit rate is less than the downlink bandwidth. If it is less, it means that a video with a higher bit rate can be used for replacement. That is, the largest ratio is selected from the sequence K to replace the candidate ratio, and the largest ratio is deleted from the sequence K, wherein the largest ratio and the candidate ratio correspond to the same publishing end.

[0054] For example, if it is determined that the sum of the bit rates of the videos corresponding to the candidate subscription combination (2, 2.2, 1.5) is less than the downlink bandwidth, then the largest ratio 1.7 can be selected from K {(1.5, 1), (1.7, 1.4), (1.3, 1.1)} to replace the candidate ratio 2.2, thereby obtaining an updated candidate subscription combination (2, 1.7, 1.5), and again determining whether the sum of the encoding bit rates at this time is less than the downlink bandwidth.

[0055] If the sum of the encoding bit rates of the videos in L exceeds the downlink bandwidth after 2.2 is replaced by 1.7, the replacement is terminated, and L at the time of termination (ie, combination (2, 2.2, 1.5)) is determined as the preferred subscription combination for the jth subscriber.

[0056] The actual idea of ​​the above algorithm is based on the negative correlation between the encoding bit rate and the ratio. Under the premise of meeting the downlink bandwidth constraint, first use a small bit rate (with the largest ratio) to combine, and then use a large bit rate (with the second largest ratio) to continuously try to replace, so as to obtain the preferred subscription combination. In this way, the preferred subscription combination with the maximum QoE under the condition of meeting the bandwidth constraint can be obtained.

[0057] In one embodiment, the SFU can also determine the preferred coding combination that any publisher should adopt based on the theoretical optimal combination of subscribers and give feedback. The specific method includes: obtaining multiple candidate coding rates for each publisher; determining the M preferred subscription combinations corresponding to the M subscribers under the multiple candidate coding rates; for the i-th publisher, determining the preferred coding combination of the i-th publisher based on the M preferred subscriptions; pushing the preferred coding combination to the i-th publisher so that the i-th publisher can make coding adjustments.

[0058] It should be noted that the multiple encoding rates to be selected at each publishing end are not equal to the actual encoding rate of the publishing end. For example, the encoding rate actually adopted by a publishing end may be 300kbps, 600kbps and 1200kbps. However, the multiple encoding rates to be selected available can include any value from 200kbps to 1800kbps. For example, the encoding rate can be a value selected arbitrarily from the following arithmetic sequence (200, 250, 300, ..., 1800) with a common difference of 50, that is, any value in the arithmetic sequence can be the encoding rate to be selected.

[0059] Then, the M preferred subscription combinations corresponding to each M subscriber end can be calculated in the manner described above, given the candidate encoding bit rates of each publisher end. It should be noted that this process is actually a theoretical calculation process, and each publisher end has not actually started to use those candidate encoding bit rates for video upload, and each subscriber end will not subscribe to videos based on the calculated M preferred subscription combinations at this time. The set of M preferred subscription combinations at this time is the theoretical optimal combination of each subscriber end.

[0060] Furthermore, the preferred coding combination of any publisher can be reversely determined based on the theoretical optimal combination of each subscriber, and the preferred coding combination can be pushed to the i-th publisher so that the i-th publisher can adjust the coding.

[0061] The reason is that for any publisher, once its candidate encoding rate is given, the number of subscribers to each candidate encoding rate of the publisher can be counted in the theoretical optimal combination, and feedback can be given based on the number of votes. Figure 4 As shown, Figure 4 A timing diagram of bit rate control between a publisher and a subscriber provided in an embodiment of this specification.

[0062] In a relatively concise implementation, the feedback result should give at least two encoding bit rates. Then, in the SFU, the number of votes corresponding to the encoding bit rates to be selected of the ith publisher included in the M preferred subscription combinations can be counted; the lower bound of the encoding bit rate is determined according to the percentage of the votes, and the encoding bit rate with the most votes is determined; and a preferred encoding combination including the lower bound of the encoding bit rate and the encoding bit rate with the most votes is generated.

[0063] The percentage of votes refers to the percentage of subscription votes above a candidate encoding rate to the total subscription votes. The lower bound of the encoding rate refers to the minimum encoding rate used in the preferred encoding combination. The percentage of votes and the lower bound of the encoding rate actually determine the minimum encoding rate that a large group of people want to use.

[0064] For example, assuming that for any i-th publisher, it can use five candidate encoding rates m1 to m5 in ascending order, the counting results show that the number of subscriptions for these five candidate encoding rates in the theoretical optimal combination are (5, 15, 40, 50, 30) respectively.

[0065] That is, we can get the percentage of votes above m2 as 96%, the percentage of votes above m3 as 85%, and the percentage of votes above m4 as 57%. If the percentage of votes is set to 95%, then m2 can be used as the lower bound of the encoding rate. At the same time, we know that the encoding rate with the most votes is m4, so the optimal encoding combination generated is (m2, m4).

[0066] After the preferred encoding combination is pushed to the publisher, the publisher changes its video encoding method to (m2, m4), so that the subscriber can also change its current subscription combination according to the new encoding bit rate, thereby achieving the best experience on the M subscriber groups.

[0067] Furthermore, if the encoder capability constraint of the publishing end is taken into account (the encoder capability constrains the types of encoding resolutions that can be used), for example, the encoder of the i-th publishing end can use three types of encoding resolutions. Then the encoding resolution type constraint of the i-th publishing end can also be obtained, and the encoding bit rate that meets the encoding resolution type constraint and has the largest number of votes is added to the preferred encoding combination, wherein the encoding bit rate with the largest number of votes is not lower than the encoding bit rate lower bound.

[0068] For example, continuing with the previous example, for the i-th publishing end, it can use 3 encoding resolutions. m2 and m4 already exist in the preferred encoding combination. The resolution of m3 is different from that of m4 and m2. Then, a coding rate with more votes, namely m3, can be added. At this time, the generated preferred encoding combination is (m2, m3, m4).

[0069] This method can adjust the encoding strategy of the publisher based on the needs of the subscriber, improving the user experience of the subscriber. The final encoding strategy not only meets the bit rate expectations of most subscribers, but also takes into account the minimum requirements of subscribers with lower bandwidth.

[0070] In one embodiment, when counting votes, the SFU can determine the server closest to any i-th publisher, and send the votes corresponding to the selected encoding rate of the i-th publisher to the server closest to the i-th publisher, so that the nearest server can count the votes of the selected encoding rate of the publisher related to itself. Through this distributed control method of voting aggregation, a centralized control node is avoided to perform unified calculations, a large amount of control signaling is reduced, and the resource load of the system is reduced.

[0071] Based on the same idea, one or more embodiments of this specification also provide devices and apparatuses corresponding to the above methods, such as Figure 5 , Figure 6 shown.

[0072] In the second aspect, Figure 5 As shown, Figure 5 A schematic diagram of the structure of a video encoding control device provided in an embodiment of this specification, the device comprising:

[0073] The video acquisition module 501 acquires the uploaded videos from each publishing end, wherein the uploaded videos contain multiple layers of different encoding bit rates;

[0074] The indicator determination module 503 determines the experience quality indicators corresponding to the uploaded video at different encoding bit rates respectively;

[0075] The bandwidth acquisition module 505 acquires the downlink bandwidth of the j-th subscriber, where 1≤j≤M;

[0076] The subscription combination determining module 507 determines the preferred subscription combination of the j-th subscriber according to the experience quality indicator and the downlink bandwidth.

[0077] Optionally, the subscription combination determination module 507 determines the ratio of the experience quality indicator and the coding rate corresponding to different coding rates, sorts them to generate a sequence K, selects the maximum ratio from each of the N publishing terminals, generates a candidate subscription combination L, deletes the selected maximum ratio from the sequence K, and performs the following cyclic judgment: judge whether the sum of the coding rates in L is less than the downlink bandwidth of the j-th subscriber terminal; if so, selects the maximum ratio from the sequence K to replace the candidate ratio, and deletes the selected maximum ratio from the sequence K, wherein the maximum ratio and the candidate ratio correspond to the same publishing terminal; otherwise, terminate the replacement, and determine L at the time of termination as the preferred subscription combination of the j-th subscriber terminal.

[0078] Optionally, the device also includes a coding adjustment module 509, which obtains multiple candidate coding rates for each publishing end; determines M preferred subscription combinations corresponding to the M subscribers under the multiple candidate coding rates; for the i-th publishing end, determines the preferred coding combination of the i-th publishing end according to the M preferred subscriptions; and pushes the preferred coding combination to the i-th publishing end so that the i-th publishing end performs coding adjustment.

[0079] Optionally, the encoding adjustment module 509 counts the number of votes corresponding to the selected encoding bit rate of the i-th publishing end included in the M preferred subscription combinations; determines the lower limit of the encoding bit rate according to the percentage of the votes, and determines the encoding bit rate with the largest number of votes; and generates a preferred encoding combination including the lower limit of the encoding bit rate and the encoding bit rate with the largest number of votes.

[0080] Optionally, the coding adjustment module 509 obtains the coding type constraint of the i-th publishing end, and adds the coding bit rate that meets the coding type constraint and has the largest number of votes to the preferred coding combination, wherein the coding bit rate with the largest number of votes is not lower than the coding bit rate lower limit.

[0081] Optionally, the encoding adjustment module 509 determines the server closest to the i-th publisher, and sends the votes corresponding to the selected encoding bit rate of the i-th publisher to the server closest to the i-th publisher, so that the server closest to the i-th publisher can summarize the votes.

[0082] Optionally, the device also includes a functional relationship determination module 511, which determines the functional relationship between the encoding bit rate and the experience quality index; correspondingly, the index determination module 503 determines the experience quality index corresponding to the uploaded video at different encoding bit rates according to the functional relationship, wherein the different encoding bit rates are negatively correlated with the ratio.

[0083] In the third aspect, Figure 6 As shown, Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification, the device includes:

[0084] at least one processor; and,

[0085] a memory communicatively connected to the at least one processor; wherein,

[0086] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to the first aspect.

[0087] In the fourth aspect, based on the same idea, the embodiment of this specification also provides a non-volatile computer storage medium corresponding to the above method, storing computer executable instructions. When the computer reads the computer executable instructions in the storage medium, the instructions enable one or more processors to execute the method described in the first aspect.

[0088] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0089] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0090] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0091] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0092] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0093] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0094] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0096] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0097] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0098] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0099] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0100] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0101] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0102] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0103] The above description is only one or more embodiments of this specification and is not intended to limit this specification. For those skilled in the art, one or more embodiments of this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included in the scope of the claims of this specification.

Claims

1. A video encoding control method, applied to a system including N publishers and M subscribers, the method include: Obtaining uploaded videos from each publishing end, wherein the uploaded videos include multiple different encoding resolutions and bit rates; Determining the experience quality indicators corresponding to the uploaded video at different encoding bit rates respectively; Get the downlink bandwidth of the jth subscriber, where 1≤j≤M; The preferred subscription combination of the j-th subscriber is determined according to the experience quality indicator and the downlink bandwidth, wherein the preferred subscription combination includes N layers of uploaded videos corresponding to N publishers, the sum of the encoding bit rates of the N layers of uploaded videos does not exceed the downlink bandwidth, and the sum of the experience quality indicators corresponding to the N layers of uploaded videos is the largest.

2. The method according to claim 1, in, Determining the preferred subscription combination of the j-th subscriber according to the experience quality indicator and the downlink bandwidth includes: Determine the ratio of the experience quality index and the encoding rate corresponding to different encoding rates, sort them and generate a sequence K. The largest ratio is selected from each of the N publishers to generate a candidate subscription combination L, and the selected largest ratio is deleted from the sequence K, and the following cyclic judgment is performed: Determine whether the sum of the encoding bit rates in L is less than the downlink bandwidth of the j-th subscriber end; if so, select the largest ratio from the sequence K to replace the candidate ratio, and delete the largest ratio from the sequence K, wherein the largest ratio and the candidate ratio correspond to the same publisher; Otherwise, the replacement is terminated, and L at the time of termination is determined as the preferred subscription combination of the j-th subscriber.

3. The method according to claim 1, in, The method further comprises: Obtain multiple encoding bit rates to be selected from each publishing end; Determine M preferred subscription combinations corresponding to the M subscribers under the multiple encoding bit rates to be selected; For the i-th publisher, determine the preferred coding combination of the i-th publisher according to the M preferred subscriptions; The preferred encoding combination is pushed to the i-th publishing end so that the i-th publishing end performs encoding adjustment.

4. The method according to claim 3, in, Determining the preferred coding combination of the i-th publisher according to the M preferred subscriptions includes: Counting the number of votes corresponding to the to-be-selected encoding bit rates of the i-th publisher included in the M preferred subscription combinations; Determine the lower bound of the encoding rate according to the percentage of the votes, and determine the encoding rate with the most votes; Generate a preferred coding combination including the coding rate lower bound and the coding rate with the largest number of votes.

5. The method according to claim 4, in, Determining the preferred coding combination of the i-th publisher according to the M preferred subscriptions includes: Obtain the encoding type constraint of the i-th publishing end, and add the encoding bit rate that meets the encoding type constraint and has the largest number of votes to the preferred encoding combination, wherein the encoding bit rate with the largest number of votes is not lower than the encoding bit rate lower bound.

6. The method according to claim 4, in, Counting the number of votes corresponding to the to-be-selected encoding bit rates of the i-th publisher included in the M preferred subscription combinations includes: Determine the server closest to the i-th publisher, and send the votes corresponding to the selected encoding bit rate of the i-th publisher to the server closest to the i-th publisher, so that the server closest to the i-th publisher can summarize the votes.

7. The method according to claim 2, in, Before obtaining the uploaded videos from each publishing end, the method further includes: Determine the functional relationship between the encoding bit rate and the quality of experience indicator; Accordingly, the experience quality indicators corresponding to the uploaded video at different encoding bit rates are determined respectively, including: The experience quality indicators corresponding to the uploaded video at different encoding bit rates are determined respectively according to the functional relationship, wherein the different encoding bit rates are negatively correlated with the ratio.

8. A video encoding control device, applied to a system including N publishers and M subscribers, wherein the device include: A video acquisition module is used to acquire uploaded videos from various publishing terminals, wherein the uploaded videos contain multiple layers of different encoding bit rates; An indicator determination module, for respectively determining the experience quality indicators corresponding to the uploaded video at different encoding bit rates; The bandwidth acquisition module obtains the downlink bandwidth of the j-th subscriber, where 1≤j≤M; A subscription combination determination module determines a preferred subscription combination of the j-th subscriber according to the experience quality indicator and the downlink bandwidth, wherein the preferred subscription combination includes N layers of uploaded videos corresponding to N publishers, the sum of the encoding bit rates of the N layers of uploaded videos does not exceed the downlink bandwidth, and the sum of the experience quality indicators corresponding to the N layers of uploaded videos is the largest.

9. An electronic device, include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

Images