Conference video code rate intelligent allocation method, system, device and storage medium

Abstract

The application provides a conference video code rate intelligent allocation method, system, device and storage medium, the method comprises the following steps: based on the current interface layout parameters of the client and the user interaction state, the level of each video channel is analyzed, and the focus distribution result is obtained; based on the average code rate of the video frames transmitted by the main view channel in the preset statistical period, the main view code rate baseline is determined; based on the main view code rate baseline and the field of view proximity of each slave view channel relative to the main view channel, the target video stream level is determined; based on the target video stream level of each slave view channel and the network packet loss rate of the main view channel, the master-slave collaborative control instruction is generated; based on the master-slave collaborative control instruction, the encoding parameter configuration signal for the main view channel and the level selection signal for the slave view channel are issued to the video stream processing unit, and the master-slave switching operation is performed. The application improves the definition and smoothness of the main view picture currently focused by the user in the conference video.

Description

Technical Field

[0001] This invention relates to the field of video conferencing communication technology, and in particular to a method, system, device, and storage medium for intelligent allocation of video conferencing bitrate. Background Technology

[0002] In existing multi-person video conferencing systems, servers typically employ static or semi-dynamic bitrate allocation strategies to ensure a good viewing experience for all participants. A common approach is for the server to calculate an average available bitrate based on the network bandwidth limits and screen resolution requirements of all online users in the current meeting, and then distribute this average bitrate evenly to each participant's terminal that is sending a video stream; alternatively, a higher fixed bitrate may be assigned only to the speaker, while other participants receive a lower fixed bitrate.

[0003] However, uniform allocation or allocation based on fixed identities ignores the dynamic changes in users' actual visual focus. In real-world meeting scenarios, users' attention is highly concentrated on the currently active main view window (i.e., the video feed of a participant currently maximized or pinned to the top), while their visual sensitivity to other participants' video feeds in smaller panes or hidden states is extremely low. Existing methods allocate a fixed bitrate, often higher than the actual perceived demand, to any video stream a user is watching, resulting in a significant waste of bandwidth resources on non-focused feeds. Secondly, when multiple meetings are accessed concurrently, the server's total outbound bandwidth is limited. Because non-focused feeds consume too much unnecessary bitrate resources, the "main view" video stream that users are truly focused on cannot obtain sufficient bitrate support. Especially in network congestion or high-concurrency scenarios, the clarity of key images decreases, resulting in pixelation or stuttering, reducing the clarity and smoothness of the main view currently being viewed by the user in the meeting video. Summary of the Invention

[0004] This invention provides a method, system, device, and storage medium for intelligent allocation of video bitrate in conference sessions, aiming to improve the clarity and smoothness of the main view currently being viewed by the user in the conference video.

[0005] In a first aspect, the present invention provides a method for intelligent allocation of video bitrate in conference sessions, comprising: The hierarchy of each video channel is analyzed based on the current interface layout parameters of the client and the user interaction status to obtain the focus distribution result. Based on the average bitrate of the video frames transmitted in the main view channel within the preset statistical period before the current time in the focus distribution result, the main view bitrate baseline is determined. Based on the main view bitrate baseline and the field-of-view proximity of each secondary view channel relative to the main view channel in the focus distribution results, the target video stream level of each secondary view channel is determined. Based on the target video stream level of each slave view channel and the network packet loss rate of the master view channel, master-slave collaborative control instructions for the video layer are generated. Based on the master-slave collaborative control command, the video stream processing unit is issued encoding parameter configuration signals for the master view channel and hierarchical selection signals for the slave view channel to perform master-slave switching operation.

[0006] In a second aspect, the present invention also provides a conference video bitrate intelligent allocation system for implementing the conference video bitrate intelligent allocation method as described in the first aspect; the conference video bitrate intelligent allocation system includes: The video channel hierarchy analysis module is used to analyze the hierarchy of each video channel based on the current interface layout parameters of the client and the user interaction status, obtain the focus distribution result, and determine the main view bitrate baseline based on the average bitrate of the video frames transmitted in the main view channel within a preset statistical period before the current time in the focus distribution result. The frequency stream hierarchy distribution analysis module is used to determine the target video stream hierarchy of each slave view channel based on the main view bitrate baseline and the field of view proximity of each slave view channel relative to the main view channel in the focus distribution results. The collaborative control instruction generation module is used to generate master-slave collaborative control instructions at the video level based on the target video stream level of each slave view channel and the network packet loss rate of the master view channel. The master-slave collaborative control switching module is used to send encoding parameter configuration signals for the master view channel and hierarchical selection signals for the slave view channel to the video stream processing unit based on the master-slave collaborative control instructions, and to perform master-slave switching operations.

[0007] Thirdly, the present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for reading and executing the computer program, thereby realizing the intelligent allocation method for conference video bitrate as described above.

[0008] Fourthly, the present invention also provides a non-transitory computer-readable storage medium storing a computer program, which, when executed by a processor, implements the intelligent allocation method for conference video bitrate as described above.

[0009] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the intelligent allocation method for conference video bitrate as described above.

[0010] The intelligent video bitrate allocation method provided in this invention analyzes the hierarchical structure of each video channel based on real-time client interface layout parameters and user interaction status. It accurately outputs focus distribution results that align with users' actual visual attention habits, breaking the inherent limitations of existing technologies that allocate bitrate solely based on user bandwidth, resolution, or fixed identity. This method precisely distinguishes between the user-focused main view channel and the less-focused secondary view channels. The main view bitrate baseline is calibrated based on the average bitrate of historical transmitted video frames within a preset statistical period, ensuring stable and reliable basic transmission quality and avoiding quality anomalies caused by blind adjustments. Simultaneously, by combining the visual proximity of each secondary view channel to the main view channel, the user's visual perception priority for different secondary views is quantified, accurately determining the target video stream hierarchy for each secondary view channel. This enables on-demand hierarchical adaptation of secondary view bitrate resources, eliminating the resource waste caused by existing technologies that uniformly allocate high fixed bitrates to all video streams and allow non-focused images to occupy a large amount of ineffective bandwidth. By combining the real-time network packet loss rate of the main view channel to generate master-slave collaborative control commands, the real-time network transmission status of the main view is deeply bound to the hierarchical control of all slave views, realizing dynamic bitrate control linked to network status. In scenarios where network congestion and high concurrency of multiple conferences lead to limited server outbound bandwidth, the resource usage of slave views can be flexibly adjusted according to the transmission pressure of the main view. By issuing corresponding main view encoding parameter configuration signals and slave view hierarchical selection signals and executing master-slave switching operations, the dynamic reallocation of bandwidth resources is completed. A large amount of redundant bandwidth resources are released from low-attention, non-focus slave view channels and prioritized for the core main view channel. This effectively avoids the defects of existing technologies, such as non-focus images crowding out bandwidth, resulting in insufficient bitrate for the key main view images, and the appearance of mosaic and stuttering. It optimizes the utilization rate of limited server bandwidth resources and improves the clarity and smoothness of the main view image currently being viewed by the user in the conference video. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating the intelligent bitrate allocation method for conference videos provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the intelligent video bitrate allocation system provided in an embodiment of the present invention; Figure 3 An embodiment diagram of the electronic device provided in this invention; Figure 4 An embodiment diagram of a computer-readable storage medium provided in accordance with the present invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0014] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0015] Optionally, see Figure 1 , Figure 1 This is a flowchart illustrating the intelligent bitrate allocation method for conference videos provided by the present invention. In this embodiment of the invention, the execution entity of the intelligent bitrate allocation method for conference videos is a bitrate allocation system. Therefore, the intelligent bitrate allocation method for conference videos includes: Step 10: Analyze the hierarchy of each video channel based on the current interface layout parameters of the client and the user interaction status to obtain the focus distribution result. Based on the average bitrate of the video frames transmitted in the main view channel within the preset statistical period before the current time in the focus distribution result, determine the main view bitrate baseline.

[0016] Optionally, the bitrate allocation system obtains the client's current interface layout parameters and user interaction status. The current interface layout parameters refer to the position coordinates, window size, and stacking relationship of each video window on the client's current display interface. User interaction status refers to the user's interactive behaviors on each video window during the most recent operation cycle, including clicking to select, dragging to move, double-clicking to zoom in, and keeping the video on top.

[0017] The bitrate allocation system analyzes the hierarchy of each video channel based on the current interface layout parameters and user interaction status. Specifically, it iterates through all video windows recorded in the current interface layout parameters, identifies the video window with the largest window size and the topmost display position, and determines the video channel corresponding to this video window as the main view channel. The main view channel is the video channel where the user's visual focus is located or has the highest priority. The bitrate allocation system determines the remaining video channels other than the main view channel as secondary view channels. This yields the focus distribution result, which includes both the main view channel and the secondary view channels.

[0018] The bitrate allocation system determines the main view bitrate baseline based on the average bitrate of video frames transmitted by the main view channel within a preset statistical period prior to the current time, as determined by the focus distribution results. The preset statistical period is a pre-defined time window length used to statistically analyze the bitrate of recently transmitted video frames by the main view channel. The bitrate allocation system obtains the bitrate value of each video frame transmitted by the main view channel within the preset statistical period, sums the bitrate values ​​of all frames, divides by the number of frames, and obtains the average bitrate. This average bitrate is then used as the main view bitrate baseline. The main view bitrate baseline represents the reference bitrate of the main view channel under congestion-free conditions.

[0019] In one embodiment, assume there are five participants in a current video conference: User A, User B, User C, User D, and User E. The current interface layout of the client is a 1-large-4-small layout, meaning one main view window occupies most of the left side of the interface, and four secondary view windows are arranged vertically on the right side. User A's video window is the largest and is displayed at the top of the current interface. The video channel corresponding to User A is designated as the main view channel, and the video channels corresponding to Users B, C, D, and E are designated as secondary view channels, respectively. Assuming a preset statistical period of 10 seconds, the bitrate values ​​of the 300 video frames transmitted by the main view channel within 10 seconds are obtained. The bitrate values ​​of the 300 frames are summed and divided by 300 to obtain an average bitrate of 2000 kilobits per second. The bitrate allocation system determines 2000 kilobits per second as the baseline bitrate for the main view.

[0020] Step 20: Based on the main view bitrate baseline and the field-of-view proximity of each secondary view channel relative to the main view channel in the focus distribution results, determine the target video stream level for each secondary view channel.

[0021] Optionally, the bitrate allocation system determines the target video stream level for each secondary view channel based on the field-of-view proximity of each secondary view channel relative to the primary view channel in the primary view bitrate baseline and focus distribution results, as described in steps 201 to 204. Field-of-view proximity refers to the normalized value of the visual distance between the video window corresponding to the secondary view channel and the video window corresponding to the primary view channel on the client display interface. The smaller the value, the closer the secondary view channel is to the primary view channel on the interface, and the easier it is for the user's peripheral vision to focus on the content of the secondary view channel. The target video stream level refers to the video coding quality level determined for the secondary view channel that should be used under the current bandwidth conditions.

[0022] Step 30: Generate master-slave collaborative control instructions for the video layer based on the target video stream level of each slave view channel and the network packet loss rate of the master view channel.

[0023] Optionally, the rate allocation system generates master-slave collaborative control instructions for video layers based on the target video stream layer of each slave view channel and the network packet loss rate of the master view channel, as described in steps 301 to 307. The network packet loss rate refers to the proportion of data packets lost by the master view channel on the current transmission link to the total number of data packets sent. The master-slave collaborative control instructions are control instructions containing encoding parameter adjustment information for the master view channel and video stream layer adjustment information for the slave view channel. These instructions are used to release bandwidth resources by reducing the video stream layer of the slave view channel when network quality deteriorates, and then use the released bandwidth resources to improve the forward error correction redundancy level of the master view channel, thereby enhancing the master view channel's resilience against packet loss.

[0024] Step 40: Based on the master-slave collaborative control command, send the encoding parameter configuration signal for the master view channel and the hierarchy selection signal for the slave view channel to the video stream processing unit to perform the master-slave switching operation.

[0025] Optionally, the bitrate allocation system sends encoding parameter configuration signals for the master view channel and layer selection signals for the slave view channel to the video stream processing unit based on master-slave collaborative control instructions, performing a master-slave switching operation. The video stream processing unit refers to the server-side functional module responsible for encoding, transcoding, and layer switching of the video stream. The encoding parameter configuration signal is a control signal sent by the video stream processing unit to instruct the master view channel to adjust its forward error correction redundancy level. The layer selection signal is a control signal sent by the video stream processing unit to instruct the slave view channel to switch to a specified video stream layer. The master-slave switching operation refers to the process by which the video stream processing unit, based on the received encoding parameter configuration signal and layer selection signal, performs a forward error correction redundancy level increase operation on the master view channel and a video stream layer switching operation on the slave view channel.

[0026] Continuing with the above embodiment, assume that in the generated master-slave collaborative control command, the forward error correction redundancy level of the master view channel is increased from level 1 to level 2, the video stream level of the slave view channel corresponding to user E remains unchanged from the lowest level (already at the lowest level), and the video stream level of the slave view channel corresponding to user D remains unchanged from the lowest level. The encoding parameter configuration signal is sent to the video stream processing unit, carrying the forward error correction redundancy level of the master view channel as level 2. The level selection signal is sent to the video stream processing unit, carrying the video stream level of the slave view channel corresponding to user E as low level, the video stream level of the slave view channel corresponding to user D as low level, the video stream level of the slave view channel corresponding to user C as mid level, and the video stream level of the slave view channel corresponding to user B as mid level. After receiving the encoding parameter configuration signal, the video stream processing unit switches the forward error correction encoding parameters of the master view channel from level 1 to level 2. After receiving the level selection signal, the video stream processing unit switches the output video stream of each slave view channel to the corresponding video stream level.

[0027] This invention analyzes the hierarchical structure of each video channel based on real-time client interface layout parameters and user interaction status, accurately outputting focus distribution results that align with users' actual visual attention habits, and precisely distinguishing between the main view channel, which receives high user attention, and the secondary view channels, which receive low attention. The main view bitrate baseline is calibrated based on the average bitrate of historically transmitted video frames within a preset statistical period, ensuring stable and reliable basic transmission quality and avoiding quality anomalies caused by blind adjustments. Simultaneously, by combining the visual proximity of each secondary view channel to the main view channel, the user's visual perception priority for different secondary views is quantified, accurately determining the target video stream hierarchy for each secondary view channel, and achieving on-demand hierarchical adaptation of secondary view bitrate resources. Master-slave collaborative control instructions are generated based on the real-time network packet loss rate of the main view channel, deeply binding the real-time network transmission status of the main view with the hierarchical control of all secondary views, achieving dynamic bitrate control linked to network status. In scenarios where network congestion and high concurrency of multiple conferences limit server outbound bandwidth, the resource usage of secondary views can be flexibly adjusted according to the transmission pressure of the main view. By issuing corresponding main view encoding parameter configuration signals and slave view level selection signals and performing master-slave switching operations, the bandwidth resources are dynamically reallocated. A large amount of redundant bandwidth resources are released from the low-attention non-focus slave view channels and given priority to the core main view channels. This effectively avoids the defects of existing technologies, such as non-focus screens crowding out bandwidth, resulting in insufficient bitrate of the key focus main view screen, and the appearance of mosaic and stuttering. It optimizes the utilization rate of limited server bandwidth resources and improves the clarity and smoothness of the main view screen that the user is currently focusing on in the conference video.

[0028] Optionally, the processes of steps 201 to 204 include: Step 201: Based on the field proximity of each secondary view channel to the primary view channel and the distance range in the preset distance priority mapping table, the channel priority level of each secondary view channel is obtained.

[0029] Optionally, the preset distance priority mapping table is a pre-stored mapping table that records multiple distance intervals and the corresponding channel priority level for each distance interval. A distance interval is a segmented range of values ​​representing the proximity of the viewport, and each distance interval corresponds to a channel priority level. The channel priority level is a rating indicating the importance of the secondary view channel in bitrate allocation. A smaller channel priority level indicates that the secondary view channel is closer to the primary view channel, making it easier for the user to focus on the secondary view channel's image, and thus, it will be demoted from the video stream hierarchy later when bandwidth is limited.

[0030] For each slave view channel, the rate allocation system compares the field proximity of the slave view channel with each distance interval in the preset distance priority mapping table one by one to determine the distance interval into which the field proximity falls, and determines the channel priority level corresponding to the distance interval as the channel priority level of the slave view channel.

[0031] Continuing with the above embodiment, the preset distance priority mapping table records four distance intervals: the first distance interval is [0, 0.2), corresponding to a channel priority level of 1. The second distance interval is [0.2, 0.4), corresponding to a channel priority level of 2. The third distance interval is [0.4, 0.6), corresponding to a channel priority level of 3. The fourth distance interval is [0.6, 1], corresponding to a channel priority level of 4. The field-of-view proximity of the view channel corresponding to User B is 0.15. 0.15 falls into the first distance interval, so the channel priority level of the view channel corresponding to User B is determined to be 1. The field-of-view proximity of the view channel corresponding to User C is 0.30. 0.30 falls into the second distance interval, so the channel priority level of the view channel corresponding to User C is determined to be 2. The field-of-view proximity of the view channel corresponding to User D is 0.50. 0.50 falls into the third distance interval, so the channel priority level of the view channel corresponding to User D is determined to be 3. The field proximity of the view channel corresponding to user E is 0.70. Since 0.70 falls into the 4th distance interval, the channel priority level of the view channel corresponding to user E is determined to be 4.

[0032] Step 202: Match the main view bitrate baseline with the baseline range in the preset bandwidth status index table to obtain the bandwidth status level.

[0033] Optionally, the preset bandwidth status index table is a pre-stored mapping table that records multiple baseline intervals and the corresponding bandwidth status level for each baseline interval. A baseline interval is a segmented range of the main view bitrate baseline. The bandwidth status level represents the remaining bandwidth resources available to the slave view channel under the current main view occupancy. A larger bandwidth status level value indicates less bandwidth occupied by the main view channel and more abundant remaining bandwidth resources available to the slave view channel. A smaller bandwidth status level value indicates more bandwidth occupied by the main view channel and more strained remaining bandwidth resources available to the slave view channel.

[0034] The bitrate allocation system compares the main view bitrate baseline with each baseline interval in the preset bandwidth status index table one by one to determine the baseline interval into which the main view bitrate baseline falls, and determines the bandwidth status level corresponding to the baseline interval as the current bandwidth status level.

[0035] Continuing with the above embodiment, the preset bandwidth status index table records three baseline intervals: The first baseline interval is greater than or equal to 0 and less than 1000 kilobits per second, corresponding to a bandwidth status level of 3, indicating that the main view channel has low bandwidth usage and the secondary view channel has ample available bandwidth. The second baseline interval is greater than or equal to 1000 kilobits per second and less than 2500 kilobits per second, corresponding to a bandwidth status level of 2, indicating that the main view channel has moderate bandwidth usage and the secondary view channel has moderate available bandwidth. The third baseline interval is greater than or equal to 2500 kilobits per second, corresponding to a bandwidth status level of 1, indicating that the main view channel has high bandwidth usage and the secondary view channel has limited available bandwidth. The main view bitrate baseline is 2000 kilobits per second, and 2000 falls into the second baseline interval, so the current bandwidth status level is determined to be 2.

[0036] Step 203: Based on the first entry matched in the preset level recommendation table according to the bandwidth status, generate a video stream level recommendation configuration for video stream levels with different channel priority levels.

[0037] Optionally, the preset recommendation table is a pre-stored mapping table, indexed by bandwidth status level. Each bandwidth status level corresponds to a first entry, and each first entry records the recommended video stream level corresponding to each channel priority level under the bandwidth status level. The video stream level recommendation configuration refers to the configuration scheme of the video stream level that should be adopted for each channel priority level under the current bandwidth status level, based on the preset recommendation table.

[0038] The bitrate allocation system uses the bandwidth status level as the index key, searches for the first entry corresponding to the bandwidth status level in the preset level recommendation table, reads the recommended video stream level corresponding to each channel priority level from the first entry, and combines the reading results into a video stream level recommendation configuration.

[0039] Continuing with the above embodiment, in the preset level recommendation table, the first entry corresponding to bandwidth status level 2 records the following mapping relationship: the recommended video stream level corresponding to channel priority level 1 is medium level, the recommended video stream level corresponding to channel priority level 2 is medium level, the recommended video stream level corresponding to channel priority level 3 is low level, and the recommended video stream level corresponding to channel priority level 4 is low level. Using bandwidth status level 2 as the index key, the first entry is found in the preset level recommendation table, and the recommended video stream level corresponding to each channel priority level is read to generate the recommended video stream level configuration: channel priority level 1 corresponds to medium level, channel priority level 2 corresponds to medium level, channel priority level 3 corresponds to low level, and channel priority level 4 corresponds to low level.

[0040] Step 204: Determine the target video stream level for each slave view channel based on the recommended video stream level configuration and the channel priority level of each slave view channel.

[0041] Optionally, the bitrate allocation system determines the target video stream level for each slave view channel based on the recommended configuration of the video stream level and the channel priority level of each slave view channel, as described in steps 2041 to 2044.

[0042] This invention implements a tiered, differentiated bitrate allocation for secondary view channels based on user visual attention levels under limited bandwidth resources. This avoids bandwidth waste caused by uniformly distributing bandwidth resources to all secondary view channels. Secondary view channels closer to the main view channel and more easily focused on by users receive a relatively higher video stream level, while those farther away and less easily focused on receive a relatively lower video stream level. This improves bandwidth utilization efficiency while ensuring the overall user viewing experience, thereby enhancing the clarity and smoothness of the main view currently focused on by the user in the conference video.

[0043] Optionally, the process of steps 2041 to 2044 includes: Step 2041: Match the video stream layer recommended configuration based on the priority level of each view channel to obtain the target video stream layer of each view channel, and match the target video stream layer of each view channel with its supported video stream layer list to obtain the layer matching result.

[0044] Optionally, the video stream hierarchy list refers to the set of selectable video encoding quality levels actually supported by the transmitting device of the view channel. The hierarchy matching result refers to the matching conclusion obtained by comparing the target video stream hierarchy of the view channel with the video stream hierarchy list of the view channel, and the matching conclusion indicates whether the target video stream hierarchy exists in the video stream hierarchy list.

[0045] For each slave view channel, the bitrate allocation system uses the channel priority level of the slave view channel as the search condition to find the corresponding recommended video stream level in the recommended video stream level configuration, and determines the recommended video stream level as the target video stream level of the slave view channel.

[0046] The bitrate allocation system compares the target video stream level with the video stream level list from the view channel one by one to determine whether the video stream level list contains the target video stream level and generates a level matching result.

[0047] Continuing with the above embodiment, the channel priority level of the slave view channel corresponding to user B is 1. The middle level corresponding to channel priority level 1 is found in the recommended video stream layer configuration, and the middle level is determined as the target video stream layer of the slave view channel corresponding to user B.

[0048] User B's corresponding view channel supports a video stream hierarchy list of high, medium, and low levels. The medium level is compared with the video stream hierarchy list. The video stream hierarchy list contains a medium level, and the hierarchy matching result indicates that the target video stream hierarchy exists in the video stream hierarchy list.

[0049] User C's view channel has a channel priority level of 2. The recommended video stream layer configuration shows a middle layer corresponding to channel priority level 2. This middle layer is determined as the target video stream layer for User C's view channel. User C's view channel supports two video stream layers: high-level and low-level. Comparing the middle layer with the video stream layer list reveals that it does not exist. The layer matching result indicates that the target video stream layer does not exist in the video stream layer list.

[0050] The channel priority level of the slave view channel corresponding to user D is 3. The recommended video stream layer configuration shows that channel priority level 3 corresponds to a low layer, which is then determined as the target video stream layer for user D's slave view channel. The supported video stream layer list for user D's slave view channel includes high, medium, and low layers. Comparing the low layer with the video stream layer list shows that a low layer exists in the list, indicating that the target video stream layer exists within the list.

[0051] The channel priority level of the slave view channel corresponding to user E is 4. In the recommended video stream layer configuration, a low layer corresponding to channel priority level 4 is found. This low layer is determined as the target video stream layer for user E's slave view channel. The supported video stream layer list for user E's slave view channel includes high-level, mid-level, and low-level layers. Comparing the low layer with the video stream layer list, a low layer is found in the list. The layer matching result indicates that the target video stream layer exists in the video stream layer list.

[0052] Step 2042: If the level matching result indicates that a target video stream level exists in the video stream level list, then the target video stream level is determined as the initial video stream level from the view channel. If the level matching result indicates that a target video stream level does not exist in the video stream level list, then the lowest level video stream level in the video stream level list is determined as the initial video stream level from the view channel.

[0053] Optionally, if the level matching result indicates that the target video stream level exists in the video stream level list, then the target video stream level is determined as the initial video stream level of the view channel. If the level matching result indicates that the target video stream level does not exist in the video stream level list, then the lowest level video stream level in the video stream level list is determined as the initial video stream level of the view channel. The initial video stream level refers to the video stream level that can actually be used by the view channel after the availability check of the video stream level list.

[0054] Continuing with the above embodiments, if the layer matching result of the view channel corresponding to User B indicates that a target video stream layer exists in the video stream layer list, then the middle layer is determined as the initial video stream layer of the view channel corresponding to User B. If the layer matching result of the view channel corresponding to User C indicates that a target video stream layer does not exist in the video stream layer list, then the lowest layer in the video stream layer list of the view channel corresponding to User C is the low layer, and the low layer is determined as the initial video stream layer of the view channel corresponding to User C. If the layer matching result of the view channel corresponding to User D indicates that a target video stream layer exists in the video stream layer list, then the low layer is determined as the initial video stream layer of the view channel corresponding to User D. If the layer matching result of the view channel corresponding to User E indicates that a target video stream layer exists in the video stream layer list, then the low layer is determined as the initial video stream layer of the view channel corresponding to User E.

[0055] Step 2043: Based on sorting the initial video stream levels of each view channel according to the identification order of the view channels, a video stream level configuration is obtained. Using the remaining bandwidth status level as the key value, a search is conducted in the preset level configuration table to see if there is a second entry that matches the video stream level configuration, and the search matching result is obtained.

[0056] Optionally, the identifier order of the view channels refers to the order in which the unique identifier numbers pre-assigned to each view channel are arranged. The video stream hierarchy configuration refers to the combination of the initial video stream hierarchy of all view channels arranged in identifier order. The preset hierarchy configuration table is a pre-stored mapping table, with bandwidth status levels as keys. Each bandwidth status level records at least one second entry, and each second entry corresponds to a set of video stream hierarchy combination schemes that have been verified for bandwidth feasibility. The search matching result refers to the conclusion of whether a second entry matching the current video stream hierarchy configuration is found in the preset hierarchy configuration table.

[0057] The bitrate allocation system arranges the initial video stream hierarchy of each view channel in sequence according to the identification order of the view channels, forming a video stream hierarchy configuration.

[0058] The bitrate allocation system uses the bandwidth status level as the key value to locate all second entries under the bandwidth status level in the preset level configuration table. It then compares the video stream level configuration with each second entry one by one to determine whether there is a second entry that is completely consistent with the video stream level configuration, and obtains the search matching result.

[0059] Continuing with the above embodiment, the view channel identifiers are User B, User C, User D, and User E. The initial video stream hierarchy is arranged according to the identifier order as follows: User B corresponds to the middle level, User C to the low level, User D to the low level, and User E to the low level, resulting in a video stream hierarchy configuration of "Middle Level, Low Level, Low Level, Low Level". Using bandwidth status level 2 as the key, a search is performed in the preset hierarchy configuration table to determine if a second entry completely matches "Middle Level, Low Level, Low Level, Low Level". Assume that the preset hierarchy configuration table records the following second entries under bandwidth status level 2: the first entry is "Middle Level, Middle Level, Low Level, Low Level", the second entry is "Middle Level, Low Level, Low Level, Low Level", and the third entry is "Low Level, Low Level, Low Level, Low Level". The video stream layer configuration "medium layer, low layer, low layer, low layer" was compared with the above three second entries one by one. It was found that it was completely consistent with the second second entry, and the search matching results indicated that the second entry existed.

[0060] Step 2044: Determine the target video stream level for each view channel based on the search matching results.

[0061] Optionally, the bitrate allocation system determines the target video stream level for each slave view channel based on the search matching results. If the search matching results indicate the existence of a second entry, it means that the current video stream level configuration is feasible under bandwidth constraints, and the initial video stream level is confirmed as the target video stream level. If the search matching results indicate the absence of a second entry, it means that the current video stream level configuration exceeds the range that the current bandwidth status can support, and the video stream levels of some slave view channels need to be gradually reduced until a feasible video stream level configuration is found, as described in steps 20441 to 20447.

[0062] In this embodiment of the invention, the initial video stream levels of all slave view channels are combined into a video stream level configuration according to the identification order. Feasibility verification is performed in a preset level configuration table using the bandwidth status level as the key value. The existence of a matching second entry is used to determine whether the current level combination scheme meets the bandwidth constraint. Under the premise of meeting the bandwidth constraint, an actual executable target video stream level is determined for each slave view channel. This avoids the allocation scheme from being unexecutable due to the recommended level exceeding the device capacity or bandwidth carrying capacity. It ensures that the target video stream level conforms to both the actual encoding capability of the device and the current bandwidth resource constraint, thereby improving the smoothness of the main view screen that the user is currently focusing on in the conference video.

[0063] Optionally, the processes of steps 20441 to 20447 include: Step 20441: If the search matching result indicates the existence of a second entry, then the initial video stream level of each view channel is determined as the target video stream level of each view channel.

[0064] Optionally, if the search matching result indicates the existence of a second entry, then the initial video stream level of each view channel is determined as the target video stream level of each view channel. Continuing with the above embodiment, if the search matching result indicates the existence of a second entry, the middle level of the initial video stream level of the view channel corresponding to user B is determined as the target video stream level of the view channel corresponding to user B; the lower level of the initial video stream level of the view channel corresponding to user C is determined as the target video stream level of the view channel corresponding to user C; the lower level of the initial video stream level of the view channel corresponding to user D is determined as the target video stream level of the view channel corresponding to user D; and the lower level of the initial video stream level of the view channel corresponding to user E is determined as the target video stream level of the view channel corresponding to user E.

[0065] Step 20442: If the search matching result indicates that there is no second entry, then take the view channel corresponding to the lowest channel priority level as the first adjusted view channel, and search the list of video stream levels supported by the first adjusted view channel to see if there is a next video stream level lower than the initial video stream level.

[0066] Optionally, if the search result indicates that there is no second entry, the slave view channel corresponding to the lowest channel priority level is selected as the first adjusted view channel. The search then checks the video stream level list supported by the first adjusted view channel to see if there is a next video stream level lower than the initial video stream level. The first adjusted view channel refers to the slave view channel selected by the bitrate allocation system that is the first to undergo video stream level downgrading when the search result indicates that there is no second entry. The next video stream level refers to the video stream level in the video stream level list that is immediately adjacent to the initial video stream level and one level lower than the initial video stream level.

[0067] The bitrate allocation system selects the view channel with the highest channel priority level from all view channels as the first adjusted view channel. Then, it checks the list of video stream levels supported by the first adjusted view channel to see if there is a video stream level that is one level lower than the current initial video stream level.

[0068] In one embodiment, it is assumed that in another scenario, the search matching result indicates that there is no second entry. The channel priority level of the slave view channel corresponding to user E is 4, which is the slave view channel with the highest channel priority level value among all slave view channels. The slave view channel corresponding to user E is determined as the first adjusted view channel. The video stream layer list supported by the slave view channel corresponding to user E is high-level, mid-level, and low-level. The initial video stream layer of the slave view channel corresponding to user E is mid-level. The next video stream layer below mid-level is searched in the video stream layer list. A low-level layer is found, that is, there is a next video stream layer below the initial video stream layer.

[0069] Step 20443: If a next video stream level exists, update the initial video stream level of the first adjusted view channel to the next video stream level. Based on the next video stream level of the first adjusted view channel and the initial video stream levels of the remaining view channels other than the first adjusted view channel, construct a new video stream level configuration.

[0070] Optionally, if a next video stream level exists, the initial video stream level of the first adjusted view channel is updated to the next video stream level. A new video stream level configuration is then constructed based on the next video stream level of the first adjusted view channel and the initial video stream levels of the remaining view channels (excluding the first adjusted view channel). The remaining view channels refer to the other view channels besides the first adjusted view channel.

[0071] The bitrate allocation system replaces the initial video stream level of the first adjusted view channel with the next video stream level, keeps the initial video stream levels of the remaining view channels unchanged, and rearranges the video stream levels of all view channels according to the identification order of the view channels to form a new video stream level configuration.

[0072] Continuing with the above embodiment, the initial video stream level of the view channel corresponding to user E is updated from a middle level to a low level. The initial video stream level of the view channel corresponding to user B remains at a middle level, the initial video stream level of the view channel corresponding to user C remains at a low level, and the initial video stream level of the view channel corresponding to user D remains at a low level. The new video stream level configuration is constructed according to the identifier order as "middle level, low level, low level, low level".

[0073] Step 20444: Based on the new video stream layer configuration, perform a search and match in the preset layer configuration table until the search and match result indicates the existence of a second entry, and determine the video stream layer of each slave view channel in the new video stream layer configuration as the target video stream layer of each slave view channel.

[0074] Optionally, the bitrate allocation system searches and matches the new video stream layer configuration in a preset layer configuration table until the search result indicates the existence of a second entry. The video stream layer of each slave view channel in the new video stream layer configuration is then determined as the target video stream layer for that slave view channel. The bitrate allocation system uses the bandwidth status level as the key value to compare the new video stream layer configuration with each second entry in the preset layer configuration table. If a matching second entry exists, the adjustment stops, and the video stream layer of each slave view channel in the current new video stream layer configuration is determined as the corresponding target video stream layer. If no matching second entry exists, the system continues to perform the next level reduction operation on the first adjusted view channel, repeating the above process until the search result indicates the existence of a second entry.

[0075] Continuing with the above embodiment, using bandwidth status level 2 as the key value, the new video stream level configuration "medium level, low level, low level, low level" is compared with each of the second entries in the preset level configuration table. Assuming that a second entry exists under bandwidth status level 2 in the preset level configuration table as "medium level, low level, low level, low level", the search matching result indicates the existence of this second entry. The target video stream level of the secondary view channel corresponding to user B is determined to be medium level, the target video stream level of the secondary view channel corresponding to user C is determined to be low level, the target video stream level of the secondary view channel corresponding to user D is determined to be low level, and the target video stream level of the secondary view channel corresponding to user E is determined to be low level.

[0076] Step 20445: If there is no next video stream level, or if the first adjusted view channel is the lowest video stream level and the search matching result indicates that there is no second entry, then keep the initial video stream level of the first adjusted view channel unchanged, and use a secondary view channel with a higher channel priority level than the first adjusted view channel as the second adjusted view channel, and perform a single-channel progressive degradation loop process of the first adjusted view channel based on the second adjusted view channel.

[0077] Optionally, if no next video stream level exists, or if the first adjusted view channel is at the lowest video stream level and the search matching result indicates that no second entry exists, then the initial video stream level of the first adjusted view channel remains unchanged. A slave view channel with a higher channel priority than the first adjusted view channel is selected as the second adjusted view channel, and a single-channel progressive degradation loop process is performed based on the second adjusted view channel. The second adjusted view channel refers to the slave view channel selected by the bitrate allocation system to take over the video stream level reduction adjustment when the first adjusted view channel can no longer further reduce its video stream level. The single-channel progressive degradation loop process refers to a loop operation where, for a single slave view channel, starting from its current initial video stream level, the next video stream level is searched progressively downwards. Each time the level is reduced, a new video stream level configuration is constructed and searched in a preset level configuration table until the search matching result indicates that a second entry exists or the channel has been reduced to the lowest video stream level.

[0078] The bitrate allocation system determines whether the first adjusted view channel is already at the lowest video stream level in the video stream level list, or whether there is no next video stream level lower than the current initial video stream level in the video stream level list. If so, the initial video stream level of the first adjusted view channel remains unchanged, and a secondary view channel with a priority level value one less than that of the first adjusted view channel is selected from the remaining secondary view channels as the second adjusted view channel. The same single-channel progressive degradation loop process is then performed on the second adjusted view channel as on the first adjusted view channel.

[0079] In one embodiment, it is assumed that the initial video stream level of the view channel corresponding to user E is already a low level, where low level is the lowest video stream level in the list of video stream levels supported by the view channel corresponding to user E, and the search matching result indicates that there is no second entry. The initial low level of the view channel corresponding to user E is maintained unchanged. The channel priority level of the view channel corresponding to user D is 3, which is one level higher than the channel priority level 4 of the view channel corresponding to user E. The view channel corresponding to user D is determined as the second view channel to be adjusted. A single-channel progressive degradation loop process is performed on the view channel corresponding to user D: the initial video stream level of the view channel corresponding to user D is a medium level. The next video stream level below the medium level is searched in the list of video stream levels supported by the view channel corresponding to user D. If a low level is found, the video stream level of the view channel corresponding to user D is updated from medium to low. A new video stream level configuration is constructed and searched for in the preset level configuration table.

[0080] Step 20446: If the search matching result indicates the existence of a second entry during the single-channel progressive degradation loop, then the video stream level of each slave view channel in the video stream level configuration obtained in the current loop is determined as the target video stream level of each slave view channel.

[0081] Optionally, if the search matching result indicates the existence of a second entry during the single-channel progressive degradation loop, then the video stream level of each slave view channel in the video stream level configuration obtained in the current loop is determined as the target video stream level of each slave view channel.

[0082] Continuing with the above embodiment, a single-channel progressive degradation loop is performed on the view channel corresponding to user D. After reducing the video stream level of the view channel corresponding to user D from the middle level to the low level, a new video stream level configuration is constructed as "middle level, low level, low level, low level". A search is performed in the preset level configuration table using bandwidth status level 2 as the key value. The search result indicates the existence of a second entry. The target video stream level of the view channel corresponding to user B is determined to be the middle level, the target video stream level of the view channel corresponding to user C is determined to be the low level, the target video stream level of the view channel corresponding to user D is determined to be the low level, and the target video stream level of the view channel corresponding to user E is determined to be the low level.

[0083] Step 20447: If the search matching result during the single-channel progressive degradation loop indicates that there is no second entry, and the second adjusted view channel is the slave view channel with the highest channel priority level, then the lowest video stream level of each slave view channel is determined as the target video stream level of each slave view channel.

[0084] Optionally, if the search and matching results during the single-channel progressive degradation loop indicate that there is no second entry, and the second adjusted view channel is the slave view channel with the highest channel priority level, then the lowest video stream level of each slave view channel is determined as the target video stream level of each slave view channel. The slave view channel with the highest channel priority level refers to the slave view channel with the smallest channel priority level value among all slave view channels. During the single-channel progressive degradation loop, if the current second adjusted view channel is already the slave view channel with the smallest channel priority level value among all slave view channels, and the search and matching results still indicate that there is no second entry after progressive degradation, it means that even if all slave view channels are reduced to the lowest video stream level, there is still no matching entry in the preset level configuration table. At this time, the bitrate allocation system determines the lowest level video stream level in the video stream level list of each slave view channel as the target video stream level of the slave view channel, as a fallback allocation scheme when bandwidth is extremely tight.

[0085] In one embodiment, assuming an extremely bandwidth-constrained scenario, after sequentially performing a single-channel progressive degradation loop on the secondary view channels corresponding to users E, D, C, and E, the search matching results still indicate that there is no second entry, and the current second adjusted view channel is the secondary view channel corresponding to user B, whose channel priority level is 1, making it the secondary view channel with the lowest channel priority level among all secondary view channels. The target video stream level of the secondary view channel corresponding to user B is determined to be a low level, as are the target video stream levels of the secondary view channels corresponding to user C, D, and E.

[0086] In this embodiment of the invention, when the initial video stream layer configuration fails the bandwidth feasibility verification of the preset layer configuration table, a gradual adjustment strategy is adopted, starting with the lowest priority view channel and progressively reducing the video stream layer level for each channel. Each time, only one video stream layer is reduced for a view channel, and then the bandwidth feasibility verification is repeated. Once the verification is successful, the adjustment stops, preserving the video stream layer level of each view channel to the greatest extent possible. If a view channel, even after being reduced to the lowest video stream layer, still fails the verification, the system switches to a view channel with a higher priority level to continue the downgrade adjustment. This ensures that the adjustment process proceeds in order of increasing user visual attention, prioritizing the sacrifice of the image quality of view channels least likely to attract user attention. When all view channels fail to pass verification even after being downgraded to the lowest video stream level, the lowest video stream level of all view channels is used as a fallback solution. This ensures that an executable target video stream level allocation result can be output under any bandwidth conditions, avoiding system stagnation due to the inability to determine the allocation scheme. At the same time, the progressive degradation strategy minimizes the impact on the user's viewing experience, thereby improving the smoothness of the main view that the user is currently focusing on in the conference video.

[0087] Optionally, the processes of steps 301 to 307 include: Step 301: If the network packet loss rate is greater than the preset packet loss rate threshold, the slave view channels are sorted according to the channel priority level of each slave view channel to obtain the slave view channel adjustment queue.

[0088] Optionally, if the network packet loss rate of the main view channel exceeds a preset packet loss rate threshold, the secondary view channels are sorted based on their channel priority level to obtain a secondary view channel adjustment queue. The preset packet loss rate threshold is a pre-set threshold used to determine whether the network quality of the main view channel has degraded to a point where forward error correction redundancy level upgrade operation needs to be initiated. The secondary view channel with the lowest channel priority level is located at the beginning of the secondary view channel adjustment queue, and the secondary view channel with the highest channel priority level is located at the end of the secondary view channel adjustment queue. The secondary view channel adjustment queue refers to an ordered sequence of secondary view channels waiting to undergo video stream level downgrade adjustment, arranged in ascending order of channel priority level.

[0089] The rate allocation system obtains the network packet loss rate of the main view channel and compares it with a preset packet loss rate threshold. If the network packet loss rate is greater than the preset packet loss rate threshold, the rate allocation system arranges all secondary view channels in ascending order of channel priority, with the secondary view channel with the lowest priority level at the end of the queue and the secondary view channel with the highest priority level at the beginning of the queue, thus forming a secondary view channel adjustment queue.

[0090] Continuing with the above embodiment, the network packet loss rate of the main view channel is 8%, and the preset packet loss rate threshold is 5%, where 8% is greater than 5%. Users B, C, D, and E are sorted according to their respective channel priority levels: User E's channel priority level is 4, User D's is 3, User C's is 2, and User B's is 1. The slave view channel with the lowest priority level is placed first, meaning User E is at the head of the slave view channel adjustment queue, User D is second, User C is third, and User B is last.

[0091] Step 302: Taking the first view channel in the view channel adjustment queue as the view channel to be adjusted, determine whether there is a next video stream level in the list of video stream levels supported by the view channel to be adjusted that is lower than the target video stream level of the view channel to be adjusted.

[0092] Optionally, the bitrate allocation system uses the first sub-view channel in the sub-view channel adjustment queue as the sub-view channel to be adjusted, and determines whether there exists a video stream level lower than the target video stream level in the list of video stream levels supported by the sub-view channel to be adjusted. The sub-view channel to be adjusted refers to the sub-view channel for which the bitrate allocation system is currently performing a video stream level reduction adjustment operation in the current loop. The bitrate allocation system retrieves the first sub-view channel from the sub-view channel adjustment queue as the sub-view channel to be adjusted, and searches the list of video stream levels supported by the sub-view channel to be adjusted to see if there exists a video stream level one level lower than the current target video stream level.

[0093] Continuing with the above embodiments, the first view channel in the view channel adjustment queue, namely the view channel corresponding to user E, is the view channel to be adjusted.

[0094] User E's target video stream level for the view channel is low-level, and the supported video stream level list for User E's view channel includes high-level, mid-level, and low-level. The search for the next video stream level below the low-level in the video stream level list fails because the low-level is already the lowest level in the list, and there is no next video stream level below it.

[0095] Step 303: If there is a next video stream level lower than the target video stream level, then based on the single-channel reclaimable bandwidth increment released by reducing the video stream level to be adjusted from the target video stream level to the next video stream level, add it to the cumulative reclaimable bandwidth after the previous cycle update to obtain the updated cumulative reclaimable bandwidth, and determine whether the updated cumulative reclaimable bandwidth is greater than or equal to the target additional bandwidth required to upgrade the main view channel from the current forward error correction redundancy level to the next forward error correction redundancy level.

[0096] Optionally, if a lower video stream level exists below the target video stream level, the incremental recoverable bandwidth per channel released by reducing the video stream level of the view channel to be adjusted from the target video stream level to the next video stream level is added to the cumulative recoverable bandwidth after the previous cycle update to obtain the updated cumulative recoverable bandwidth. It is then determined whether the updated cumulative recoverable bandwidth is greater than or equal to the target additional bandwidth required to upgrade the main view channel from the current forward error correction redundancy level to the next forward error correction redundancy level. The incremental recoverable bandwidth per channel refers to the bandwidth difference released by the view channel after reducing the video stream level of a single view channel from the current target video stream level to the next video stream level. The updated cumulative recoverable bandwidth refers to the total bandwidth released by all view channels that have undergone downgrade operations from the start of the cycle to the current round. The target additional bandwidth required refers to the additional bandwidth needed to upgrade the forward error correction redundancy level of the main view channel from the current level to the next level. Forward error correction redundancy level refers to the strength level of error correction redundancy data added to the main view channel when transmitting video data. The higher the forward error correction redundancy level, the stronger the anti-packet loss recovery capability of the main view channel, but the more bandwidth is required.

[0097] The bitrate allocation system calculates the difference between the encoded bitrate of the target video stream layer to be adjusted from the view channel and the encoded bitrate of the next video stream layer, and determines this difference as the single-channel reclaimable bandwidth increment. The bitrate allocation system adds the single-channel reclaimable bandwidth increment to the cumulative reclaimable bandwidth after the previous cycle update to obtain the updated cumulative reclaimable bandwidth.

[0098] The rate allocation system compares the updated cumulative recoverable bandwidth with the target additional bandwidth consumption to determine whether the updated cumulative recoverable bandwidth is greater than or equal to the target additional bandwidth consumption.

[0099] Continuing with the above embodiment, since there is no next video stream level below the target video stream level for the slave view channel corresponding to user E, the process follows step 305. In another scenario, assume the target video stream level for the slave view channel corresponding to user E is a mid-level, with a coding bitrate of 600 kilobits per second and a coding bitrate of 300 kilobits per second for the low-level. Subtracting 300 kilobits per second from 600 kilobits per second yields a single-channel reclaimable bandwidth increment of 300 kilobits per second. Assuming the cumulative reclaimable bandwidth after the previous cycle update is 0 kilobits per second (first cycle), adding 0 kilobits per second to 300 kilobits per second yields a cumulative reclaimable bandwidth of 300 kilobits per second after the update. Assuming the target additional bandwidth required to upgrade the master view channel from the current forward error correction redundancy level 1 to the next forward error correction redundancy level 2 is 500 kilobits per second. Comparing 300 kilobits per second with 500 kilobits per second, 300 kilobits per second is less than 500 kilobits per second, and the cumulative recoverable bandwidth after the update is less than the target additional bandwidth consumed.

[0100] Step 304: If the cumulative recoverable bandwidth after the update is greater than or equal to the target additional consumed bandwidth, then the video stream level of the view channel to be adjusted is updated to the next video stream level, the remaining unprocessed video stream levels of the view channels in the view channel adjustment queue are kept as the target video stream level, and a master-slave collaborative control instruction is generated, which includes the forward error correction redundancy level after the main view channel is improved, the updated video stream level of the view channel to be adjusted, and the target video stream level kept by the remaining unprocessed view channels.

[0101] Optionally, if the cumulative recoverable bandwidth after the update is greater than or equal to the target additional bandwidth consumed, the video stream level of the slave view channel to be adjusted will be updated to the next video stream level. The video stream level of the remaining unprocessed slave view channels in the slave view channel adjustment queue will be maintained as the target video stream level. A master-slave collaborative control instruction will be generated, which includes the forward error correction redundancy level after the master view channel is upgraded, the updated video stream level of the slave view channel to be adjusted, and the target video stream level maintained by the remaining unprocessed slave view channels. The remaining unprocessed slave view channels refer to the slave view channels in the slave view channel adjustment queue that have not yet undergone a downgrade adjustment operation.

[0102] After confirming that the accumulated recoverable bandwidth meets the target additional bandwidth consumption, the rate allocation system updates the video stream level of the slave view channel to be adjusted from the target video stream level to the next video stream level. The remaining unprocessed slave view channel video stream levels in the slave view channel adjustment queue maintain their respective target video stream levels unchanged. The rate allocation system encapsulates the improved forward error correction redundancy level of the master view channel, the updated video stream level of the slave view channel to be adjusted, and the target video stream level maintained by the remaining unprocessed slave view channels into a master-slave collaborative control command.

[0103] In one embodiment, assuming a different scenario, in the view channel adjustment queue, user E is at the first position, user D is at the second position, user C is at the third position, and user B is at the last position. The target video stream level for user E's view channel is a mid-level, and there exists a lower video stream level below the mid-level, i.e., a low-level. The single-channel reclaimable bandwidth increment is 300 kilobits per second, and the updated cumulative reclaimable bandwidth is 300 kilobits per second, which is less than the target additional bandwidth consumption of 500 kilobits per second. The process continues with the view channel corresponding to user D. The target video stream level for user D's view channel is also a mid-level, and there exists a lower video stream level below the mid-level, i.e., a low-level. The single-channel reclaimable bandwidth increment is 300 kilobits per second. Adding 300 kilobits per second to the previous round's updated cumulative reclaimable bandwidth of 300 kilobits per second, the updated cumulative reclaimable bandwidth is 600 kilobits per second. 600 kilobits per second is greater than or equal to the target additional bandwidth consumption of 500 kilobits per second. Update the video stream level of the slave view channel corresponding to user D to a low level. Maintain the video stream levels of the remaining unprocessed slave view channels in the slave view channel adjustment queue (i.e., the slave view channels corresponding to user C and user B) at the target video stream level, i.e., a mid-level. Generate a master-slave collaborative control command, which includes: increasing the forward error correction redundancy level of the master view channel from level 1 to level 2; setting the video stream level of the slave view channel corresponding to user E to a low level; setting the video stream level of the slave view channel corresponding to user D to a low level; setting the video stream level of the slave view channel corresponding to user C to a mid-level; and setting the video stream level of the slave view channel corresponding to user B to a mid-level.

[0104] Step 305: If there is no next video stream level below the target video stream level, or if the cumulative recoverable bandwidth after the update is less than the target additional bandwidth consumed, then mark the view channel to be adjusted as processed and determine whether there is an unprocessed view channel in the view channel adjustment queue.

[0105] Optionally, if there is no next video stream level lower than the target video stream level, or if the cumulative recoverable bandwidth after the update is less than the target additional bandwidth consumption, the view channel to be adjusted is marked as processed, and it is determined whether there are any unprocessed view channels in the view channel adjustment queue. The processed state refers to the flag set for the view channel, indicating that the view channel has been evaluated in the current loop and cannot continue to reduce the video stream level, or the view channel has contributed recoverable bandwidth but the cumulative recoverable bandwidth still does not meet the target additional bandwidth consumption.

[0106] If the bitrate allocation system sets a processed status flag for the view channel to be adjusted if there is no next video stream level below the target video stream level, or if the cumulative recoverable bandwidth after the update is less than the target additional bandwidth consumed, the bitrate allocation system then traverses the view channel adjustment queue to check if there are any view channels for which a processed status flag has not yet been set.

[0107] Continuing with the above embodiment, the target video stream level of the slave view channel corresponding to user E is a low level, and there is no next video stream level below the low level. The slave view channel corresponding to user E is marked as processed. Traversing the slave view channel adjustment queue, the slave view channels corresponding to users D, C, and B are not marked as processed, meaning there are unprocessed slave view channels.

[0108] Step 305: If there are no unprocessed slave view channels and the updated cumulative recoverable bandwidth is less than the target additional bandwidth consumption, then set the video stream level of each slave view channel to the lowest video stream level in its respective supported video stream level list, determine the updated cumulative recoverable bandwidth at the end of the loop as the total recoverable bandwidth, and generate master-slave collaborative control instructions based on the total recoverable bandwidth, including the forward error correction redundancy level of the master view channel after improvement and the lowest video stream level of all slave view channels.

[0109] Optionally, if there are no unprocessed slave view channels and the updated cumulative recoverable bandwidth is less than the target additional bandwidth consumption, then the video stream level of each slave view channel is set to the lowest video stream level in its respective supported video stream level list. The updated cumulative recoverable bandwidth at the end of the loop is determined as the total recoverable bandwidth. Based on the total recoverable bandwidth, a master-slave collaborative control command is generated, including the forward error correction redundancy level after the master view channel is upgraded and the lowest video stream level of all slave view channels. The total recoverable bandwidth refers to the total amount of bandwidth that can be released after all slave view channels are reduced to their respective lowest video stream levels.

[0110] In one embodiment, assuming that in an extreme scenario, after traversing all slave view channels in the slave view channel adjustment queue, all slave view channels have been marked as processed, and the cumulative recoverable bandwidth after the update is 800 kilobits per second, which is still less than the target additional bandwidth consumption of 1000 kilobits per second, the video stream levels of the slave view channels corresponding to users B, C, D, and E are all set to the lowest video stream level (low level) in their respective supported video stream level lists. The total recoverable bandwidth of 800 kilobits per second is determined, and a master-slave collaborative control instruction is generated. The instruction includes the forward error correction redundancy level of the master view channel after its upgrade and the lowest video stream level of all slave view channels.

[0111] Step 307: If there are unprocessed slave view channels, the next slave view channel in the slave view channel adjustment queue is used as the slave view channel to be adjusted to execute the control instruction configuration until the master-slave collaborative control instruction is obtained.

[0112] Optionally, if there are unprocessed slave view channels, the rate allocation system will use the next slave view channel in the slave view channel adjustment queue as the slave view channel to be adjusted to execute the control instruction configuration until the master-slave collaborative control instruction is obtained.

[0113] After confirming that there are unprocessed slave view channels in the slave view channel adjustment queue, the rate allocation system takes the next unprocessed slave view channel as the new slave view channel to be adjusted according to the order of the slave view channel adjustment queue. The judgment and accumulation process of steps 302 to 305 is repeated until the updated cumulative recoverable bandwidth is greater than or equal to the target additional consumed bandwidth, at which point a master-slave collaborative control instruction is generated, or when all slave view channels have been processed, a master-slave collaborative control instruction is generated according to the fallback scheme.

[0114] Continuing with the above embodiment, the slave view channel corresponding to user E has been marked as processed, and there are unprocessed slave view channels in the slave view channel adjustment queue. Taking the next slave view channel in the slave view channel adjustment queue, namely the slave view channel corresponding to user D, as the slave view channel to be adjusted, it is determined whether there is a next video stream level lower than its target video stream level in the list of video stream levels supported by the slave view channel corresponding to user D, and the subsequent bandwidth accumulation and judgment process continues until the master-slave collaborative control instruction is obtained.

[0115] In this embodiment of the invention, when the network packet loss rate of the main view channel exceeds a preset packet loss rate threshold, the video stream level of the secondary view channels is sequentially reduced according to channel priority from low to high. By accumulating the incremental single-channel reclaimable bandwidth released by each secondary view channel in rounds, the degradation operation is immediately stopped and a master-slave collaborative control command is generated when the accumulated reclaimable bandwidth meets the target additional bandwidth required for the main view channel to improve its forward error correction redundancy level. This achieves the improvement of the main view channel's packet loss resistance at the lowest possible cost to the secondary view channel's image quality. Simultaneously, secondary view channels without a lower video stream level are automatically skipped and marked as processed, and the next secondary view channel is processed, ensuring the continuity and integrity of the adjustment process. When all secondary view channels have been reduced to the lowest video stream level but still cannot meet the target additional bandwidth requirement, a fallback master-slave collaborative control command is generated with the total reclaimable bandwidth as a constraint. This ensures that an executable control scheme can be output under any network conditions. Overall, this achieves master-slave collaborative bandwidth reallocation that prioritizes the transmission reliability of the main view channel when network quality deteriorates, while minimizing the impact on the image quality of the secondary view channels.

[0116] Furthermore, the intelligent video bitrate allocation system described below and the intelligent video bitrate allocation method described above can be referred to in correspondence.

[0117] Optionally, refer to Figure 2 , Figure 2 This is a schematic diagram of the intelligent video bitrate allocation system for conferences provided by the present invention. The intelligent video bitrate allocation system for conferences includes: The video channel hierarchy analysis module 210 is used to analyze the hierarchy of each video channel based on the current interface layout parameters of the client and the user interaction status, obtain the focus distribution result, and determine the main view bitrate baseline based on the average bitrate of the video frames transmitted in the main view channel within a preset statistical period before the current time in the focus distribution result.

[0118] The frequency stream hierarchy distribution analysis module 220 is used to determine the target video stream hierarchy of each slave view channel based on the main view bit rate baseline and the field of view proximity of each slave view channel relative to the main view channel in the focus distribution results.

[0119] The collaborative control instruction generation module 230 is used to generate master-slave collaborative control instructions at the video layer based on the target video stream layer of each slave view channel and the network packet loss rate of the master view channel.

[0120] The master-slave collaborative control switching module 240 is used to send encoding parameter configuration signals for the master view channel and hierarchical selection signals for the slave view channel to the video stream processing unit based on the master-slave collaborative control instructions, and to perform master-slave switching operations.

[0121] This invention analyzes the hierarchical structure of each video channel based on real-time client interface layout parameters and user interaction status, accurately outputting focus distribution results that align with users' actual visual attention habits, and precisely distinguishing between the main view channel, which receives high user attention, and the secondary view channels, which receive low attention. The main view bitrate baseline is calibrated based on the average bitrate of historically transmitted video frames within a preset statistical period, ensuring stable and reliable basic transmission quality and avoiding quality anomalies caused by blind adjustments. Simultaneously, by combining the visual proximity of each secondary view channel to the main view channel, the user's visual perception priority for different secondary views is quantified, accurately determining the target video stream hierarchy for each secondary view channel, and achieving on-demand hierarchical adaptation of secondary view bitrate resources. Master-slave collaborative control instructions are generated based on the real-time network packet loss rate of the main view channel, deeply binding the real-time network transmission status of the main view with the hierarchical control of all secondary views, achieving dynamic bitrate control linked to network status. In scenarios where network congestion and high concurrency of multiple conferences limit server outbound bandwidth, the resource usage of secondary views can be flexibly adjusted according to the transmission pressure of the main view. By issuing corresponding main view encoding parameter configuration signals and slave view level selection signals and performing master-slave switching operations, the bandwidth resources are dynamically reallocated. A large amount of redundant bandwidth resources are released from the low-attention non-focus slave view channels and given priority to the core main view channels. This effectively avoids the defects of existing technologies, such as non-focus screens crowding out bandwidth, resulting in insufficient bitrate of the key focus main view screen, and the appearance of mosaic and stuttering. It optimizes the utilization rate of limited server bandwidth resources and improves the clarity and smoothness of the main view screen that the user is currently focusing on in the conference video.

[0122] Please see Figure 3 , Figure 3 An embodiment diagram of an electronic device provided in accordance with the present invention. For example... Figure 3 As shown, an embodiment of the present invention provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, it implements the processes of steps 10 to 40.

[0123] Please see Figure 4 , Figure 4 An embodiment diagram of a computer-readable storage medium provided in accordance with an embodiment of the present invention is shown. Figure 4 As shown, this embodiment provides a computer-readable storage medium 400, on which a computer program 311 is stored. When the computer program 311 is executed by a processor, it implements the processes of steps 10 to 40.

[0124] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the intelligent video conferencing bitrate allocation method provided by the above methods, which includes steps 10 to 40.

[0125] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0126] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent allocation of video conferencing bitrate, characterized in that, include: The hierarchy of each video channel is analyzed based on the current interface layout parameters of the client and the user interaction status to obtain the focus distribution result. Based on the average bitrate of the video frames transmitted in the main view channel within the preset statistical period before the current time in the focus distribution result, the main view bitrate baseline is determined. Based on the main view bitrate baseline and the field-of-view proximity of each secondary view channel relative to the main view channel in the focus distribution results, the target video stream level of each secondary view channel is determined. Based on the target video stream level of each slave view channel and the network packet loss rate of the master view channel, master-slave collaborative control instructions for the video layer are generated. Based on the master-slave collaborative control command, the video stream processing unit is issued encoding parameter configuration signals for the master view channel and hierarchical selection signals for the slave view channel to perform master-slave switching operation.

2. The intelligent video bitrate allocation method according to claim 1, characterized in that, The steps involved in generating master-slave collaborative control instructions at the video level include: If the network packet loss rate of the main view channel is greater than the preset packet loss rate threshold, the secondary view channels are sorted according to the channel priority level of each secondary view channel to obtain the secondary view channel adjustment queue. Taking the first subview channel in the subview channel adjustment queue as the subview channel to be adjusted, determine whether there is a next video stream level lower than the target video stream level of the subview channel to be adjusted in the video stream level list supported by the subview channel to be adjusted; If there is a next video stream level lower than the target video stream level, the incremental single-channel reclaimable bandwidth released by reducing the video stream level to be adjusted from the target video stream level to the next video stream level is added to the cumulative reclaimable bandwidth after the previous cycle update to obtain the updated cumulative reclaimable bandwidth. It is then determined whether the updated cumulative reclaimable bandwidth is greater than or equal to the target additional bandwidth required to upgrade the main view channel from the current forward error correction redundancy level to the next forward error correction redundancy level. If the cumulative recoverable bandwidth after the update is greater than or equal to the target additional consumed bandwidth, the video stream level of the view channel to be adjusted will be updated to the next video stream level. The remaining unprocessed video stream levels of the view channels in the view channel adjustment queue will be kept as the target video stream level. A master-slave collaborative control instruction will be generated, which includes the forward error correction redundancy level after the master view channel is upgraded, the updated video stream level of the view channel to be adjusted, and the target video stream level kept by the remaining unprocessed view channels.

3. The intelligent video bitrate allocation method according to claim 2, characterized in that, The intelligent allocation method for conference video bitrate also includes: If there is no next video stream level below the target video stream level, or if the cumulative recoverable bandwidth after the update is less than the target additional bandwidth consumed, then the view channel to be adjusted is marked as processed, and it is determined whether there is an unprocessed view channel in the view channel adjustment queue. If there are no unprocessed slave view channels and the updated cumulative recoverable bandwidth is less than the target additional bandwidth consumption, then the video stream level of each slave view channel is set to the lowest video stream level in its respective supported video stream level list. The updated cumulative recoverable bandwidth at the end of the loop is determined as the total recoverable bandwidth. Based on the total recoverable bandwidth, a master-slave collaborative control instruction is generated, which includes the forward error correction redundancy level after the master view channel is improved and the lowest video stream level of all slave view channels. If there are any unprocessed slave view channels, the next slave view channel in the slave view channel adjustment queue will be used as the slave view channel to be adjusted to execute the control command configuration until the master-slave collaborative control command is obtained.

4. The intelligent video bitrate allocation method according to claim 1, characterized in that, The determination of the target video stream level for each slave view channel based on the main view bitrate baseline and the field-of-view proximity of each slave view channel relative to the main view channel in the focus distribution results includes: The channel priority level of each view channel is obtained by matching the field proximity of each view channel relative to the main view channel with the distance range in the preset distance priority mapping table. The bandwidth status level is obtained by matching the main view bitrate baseline with the baseline range in the preset bandwidth status index table; the bandwidth status level represents the remaining bandwidth resource level available for use by the slave view channel under the current main view occupancy. Based on the first entry matched in the preset hierarchical recommendation table according to the bandwidth status level, a recommended configuration for video stream hierarchy is generated for video stream hierarchy with different channel priority levels. Based on the recommended video stream hierarchy configuration and the channel priority level of each slave view channel, the target video stream hierarchy for each slave view channel is determined.

5. The intelligent video bitrate allocation method according to claim 4, characterized in that, The step of determining the target video stream level for each slave view channel based on the recommended configuration of the video stream hierarchy and the channel priority level of each slave view channel includes: Based on the priority level of each view channel, the target video stream level of each view channel is obtained by matching in the recommended configuration of the video stream level. Then, based on the target video stream level of each view channel and its supported video stream level list, the level matching result is obtained. If the level matching result indicates that the target video stream level exists in the video stream level list, then the target video stream level is determined as the initial video stream level from the view channel; if the level matching result indicates that the target video stream level does not exist in the video stream level list, then the lowest level video stream level in the video stream level list is determined as the initial video stream level from the view channel. Based on sorting the initial video stream levels of each view channel according to the identification order of the view channel, a video stream level configuration is obtained. Then, using the remaining bandwidth status level as the key value, a search is conducted in the preset level configuration table to see if there is a second entry that matches the video stream level configuration, and the search matching result is obtained. The target video stream level for each view channel is determined based on the search matching results.

6. The intelligent video bitrate allocation method according to claim 5, characterized in that, Determining the target video stream level for each view channel based on the search matching results includes: If the search results indicate the existence of a second entry, then the initial video stream level of each view channel is determined as the target video stream level of each view channel; If the search results indicate that there is no second entry, then the view channel corresponding to the lowest channel priority level is taken as the first adjusted view channel, and the list of video stream levels supported by the first adjusted view channel is searched to see if there is a next video stream level lower than the initial video stream level. If a next video stream level exists, the initial video stream level of the first adjusted view channel is updated to the next video stream level. A new video stream level configuration is constructed based on the next video stream level of the first adjusted view channel and the initial video stream levels of the remaining view channels other than the first adjusted view channel. Based on the new video stream hierarchy configuration, a search and matching process is performed in the preset hierarchy configuration table until the search and matching result indicates the existence of a second entry. The video stream hierarchy of each slave view channel in the new video stream hierarchy configuration is then determined as the target video stream hierarchy for each slave view channel.

7. The intelligent video bitrate allocation method according to claim 6, characterized in that, The intelligent allocation method for conference video bitrate also includes: If there is no next video stream level, or if the first adjusted view channel is the lowest video stream level and the search matching result indicates that there is no second entry, then the initial video stream level of the first adjusted view channel remains unchanged, and the second adjusted view channel is a secondary view channel with a channel priority level one level higher than that of the first adjusted view channel. Based on the second adjusted view channel, the single-channel step-down loop process of the first adjusted view channel is executed. If the search and matching results indicate the existence of a second entry during the single-channel progressive degradation loop, then the video stream level of each slave view channel in the video stream level configuration obtained in the current loop will be determined as the target video stream level of each slave view channel. If the search and matching results during the single-channel progressive degradation loop indicate that there is no second entry, and the second adjusted view channel is the slave view channel with the highest channel priority level, then the lowest video stream level of each slave view channel is determined as the target video stream level of each slave view channel.

8. A video conferencing bitrate intelligent allocation system, characterized in that, The system is used to implement the intelligent allocation method for conference video bitrate as described in any one of claims 1 to 7; the system includes: The video channel hierarchy analysis module is used to analyze the hierarchy of each video channel based on the current interface layout parameters of the client and the user interaction status, obtain the focus distribution result, and determine the main view bitrate baseline based on the average bitrate of the video frames transmitted in the main view channel within a preset statistical period before the current time in the focus distribution result. The frequency stream hierarchy distribution analysis module is used to determine the target video stream hierarchy of each slave view channel based on the main view bitrate baseline and the field of view proximity of each slave view channel relative to the main view channel in the focus distribution results. The collaborative control instruction generation module is used to generate master-slave collaborative control instructions at the video level based on the target video stream level of each slave view channel and the network packet loss rate of the master view channel. The master-slave collaborative control switching module is used to send encoding parameter configuration signals for the master view channel and hierarchical selection signals for the slave view channel to the video stream processing unit based on the master-slave collaborative control instructions, and to perform master-slave switching operations.

9. An electronic device, comprising: A memory for storing a computer program; a processor for reading and executing the computer program, characterized in that, when the processor executes the computer program, it implements the intelligent allocation method for conference video bitrate as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the intelligent allocation method for conference video bitrate as described in any one of claims 1 to 7.

Images