A method and system for adjusting audio bit rate of audio and video transmission

Abstract

The embodiment of the present application discloses a method and system for adjusting the audio bit rate of audio and video transmission. The technical solution provided by the embodiment of the present application obtains the congestion information of the current audio and video transmission, determines based on the congestion information that the current audio encoding bit rate is in a bit rate gear adjustment state, obtains the target bit rate of the audio and video transmission statistics of the current acquisition period, and determines the bit rate gear of the current audio encoding bit rate. Based on the target bit rate and the bit rate gear of the current audio encoding bit rate, the bit rate gear of the current audio encoding bit rate is adjusted, and the audio encoding of the current audio and video transmission is performed based on the audio bit rate value corresponding to the adjusted bit rate gear. The above-mentioned technical means are adopted to adjust the audio bit rate by combining the network congestion situation and the target bit rate of the audio and video transmission, and the audio bit rate is adaptively adjusted according to the real-time network situation, so as to adaptively provide a reasonable audio bit rate service, optimize the audio and video call effect, and improve the audio and video call experience.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of audio and video coding technology, and more particularly to a method and system for adjusting audio bit rate of audio and video transmission. Background Art

[0002] At present, with the development of streaming media technology and the popularization of smart terminal devices, various data transmission scenarios based on real-time audio and video are becoming an important way of communication between people. In order to continuously improve the audio and video quality experience of users in audio and video calls, it is necessary to control the network congestion caused by packet loss or delay in the network. The current congestion control scheme is mainly achieved by dynamically adjusting the video bit rate and fixedly configuring the audio bit rate, and optimizing the user's video viewing experience through adaptive adjustment of the video bit rate.

[0003] However, traditional congestion control solutions only adjust the video bitrate, while the audio bitrate cannot change dynamically with network changes, and can only provide fixed audio bitrate services. It cannot provide higher-definition bitrate services in high-quality network scenarios, and can only provide smooth bitrate services in poor network scenarios. Its audio encoding transmission effect is poor, affecting users' audio and video call experience. Summary of the invention

[0004] The embodiments of the present application provide a method and system for adjusting the audio bit rate of audio and video transmission, which can adaptively adjust the audio bit rate according to the real-time network conditions, optimize the audio bit rate service, and solve the technical problem of how to adaptively adjust the audio bit rate in the audio and video transmission scenario.

[0005] In a first aspect, an embodiment of the present application provides an audio bit rate adjustment method for audio and video transmission, comprising:

[0006] Obtain the congestion information of the current audio and video transmission, and determine whether the current audio encoding bit rate is in a bit rate gear adjustment state based on the congestion information;

[0007] When it is determined that the current audio encoding bit rate is in a bit rate gear adjustment state, the target bit rate of audio and video transmission in the current acquisition period is obtained, and the bit rate gear of the current audio encoding bit rate is determined. The bit rate gear is pre-configured with corresponding audio bit rate values ​​according to different setting gears;

[0008] Based on the target bit rate and the bit rate level of the current audio encoding bit rate, the bit rate level of the current audio encoding bit rate is adjusted, and audio encoding of the current audio and video transmission is performed based on the audio bit rate value corresponding to the adjusted bit rate level.

[0009] In a second aspect, an embodiment of the present application provides an audio bit rate adjustment system for audio and video transmission, including:

[0010] A state judgment module is configured to obtain congestion information of the current audio and video transmission, and judge whether the current audio encoding bit rate is in a bit rate gear adjustment state based on the congestion information;

[0011] The acquisition module is configured to obtain the target bit rate of audio and video transmission counted in the current acquisition period and determine the bit rate gear of the current audio encoding bit rate when determining that the current audio encoding bit rate is in the bit rate gear adjustment state, and the bit rate gear is pre-configured with the corresponding audio bit rate value according to different set gears;

[0012] The bit rate adjustment module is configured to adjust the bit rate level of the current audio encoding bit rate based on the target bit rate and the bit rate level of the current audio encoding bit rate, and perform audio encoding for the current audio and video transmission based on the audio bit rate value corresponding to the adjusted bit rate level.

[0013] In a third aspect, an embodiment of the present application provides an audio bit rate adjustment device for audio and video transmission, including:

[0014] memory and one or more processors;

[0015] The memory is configured to store one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the audio bit rate adjustment method for audio and video transmission as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are configured to execute the audio bit rate adjustment method for audio and video transmission as described in the first aspect.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the method for adjusting the audio bit rate of audio and video transmission as described in the first aspect.

[0019] The embodiment of the present application obtains the congestion information of the current audio and video transmission, and determines whether the current audio encoding bit rate is in a bit rate gear adjustment state based on the congestion information. Then, when it is determined that the current audio encoding bit rate is in a bit rate gear adjustment state, the target bit rate of the audio and video transmission statistically calculated in the current acquisition period is obtained, and the bit rate gear of the current audio encoding bit rate is determined, and the bit rate gear is pre-configured with a corresponding audio bit rate value according to different set gears. Afterwards, based on the target bit rate and the bit rate gear of the current audio encoding bit rate, the bit rate gear of the current audio encoding bit rate is adjusted, and the audio encoding of the current audio and video transmission is performed based on the audio bit rate value corresponding to the adjusted bit rate gear. The above-mentioned technical means are adopted to adjust the audio bit rate by combining the network congestion situation and the target bit rate of the audio and video transmission, and the audio bit rate is adaptively adjusted according to the real-time network situation, so as to adaptively provide a reasonable audio bit rate service, optimize the audio and video call effect, and improve the audio and video call experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of an audio bit rate adjustment method for audio and video transmission provided in an embodiment of the present application;

[0021] Figure 2 This is a flow chart of the bit rate gear up adjustment in the embodiment of the present application;

[0022] Figure 3 This is a flow chart of bit rate gear downshift adjustment in the embodiment of the present application;

[0023] Figure 4 is a schematic diagram of bit rate adjustment in the upshift state in an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of bit rate adjustment in a downshift state in an embodiment of the present application;

[0025] Figure 6 It is a structural schematic diagram of an audio bit rate adjustment system for audio and video transmission provided in an embodiment of the present application;

[0026] Figure 7 It is a structural schematic diagram of an audio bit rate adjustment device for audio and video transmission provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0028] The present application provides an audio bit rate adjustment method for audio and video transmission, which aims to adjust the audio bit rate in combination with the network congestion situation and the target bit rate of the audio and video transmission, so as to adaptively adjust the audio bit rate according to the real-time network situation. For the bit rate adjustment scheme in the traditional audio and video transmission scenario, it generally adjusts the video bit rate according to the network congestion situation, and the audio bit rate cannot change dynamically with the change of the network, and can only provide a fixed audio bit rate service. Therefore, it is impossible to provide a higher-definition audio bit rate service in a high-quality network scenario, and to provide a smooth audio bit rate service in a poor network scenario, and the user's audio and video call experience is relatively poor. Based on this, an audio bit rate adjustment method for audio and video transmission according to an embodiment of the present application is provided to solve the technical problem of how to perform adaptive audio bit rate adjustment in existing audio and video transmission scenarios.

[0029] Example:

[0030] Figure 1 A flow chart of an audio bit rate adjustment method for audio and video transmission provided in an embodiment of the present application is given. The audio bit rate adjustment method for audio and video transmission provided in this embodiment can be performed by an audio bit rate adjustment device for audio and video transmission. The audio bit rate adjustment device for audio and video transmission can be implemented by software and / or hardware. The audio bit rate adjustment device for audio and video transmission can be composed of two or more physical entities, or can be composed of one physical entity. Generally speaking, the audio bit rate adjustment device for audio and video transmission can be a processing device such as an audio and video encoder, a server, or a computer.

[0031] The following description is made by taking the audio bit rate adjustment device for audio and video transmission as an example to implement the audio bit rate adjustment method for audio and video transmission. Figure 1 The audio bit rate adjustment method for audio and video transmission specifically includes:

[0032] S110, obtaining congestion information of current audio and video transmission, and determining whether the current audio encoding bit rate is in a bit rate gear adjustment state based on the congestion information.

[0033] S120. When it is determined that the current audio encoding bit rate is in a bit rate gear adjustment state, obtain the target bit rate of the audio and video transmission statistics of the current acquisition period, and determine the bit rate gear of the current audio encoding bit rate. The bit rate gear is pre-configured with corresponding audio bit rate values ​​according to different set gears.

[0034] S130, based on the target bit rate and the bit rate level of the current audio encoding bit rate, adjust the bit rate level of the current audio encoding bit rate, and perform audio encoding for the current audio and video transmission based on the audio bit rate value corresponding to the adjusted bit rate level.

[0035] When adjusting the audio bit rate in the audio and video transmission scenario, the embodiment of the present application uses network congestion information to determine the actual state of the audio and video transmission network, and then adjusts the audio bit rate according to the network state of the audio and video transmission network. In addition, during the audio bit rate adjustment process, the audio bit rate level is adaptively adjusted by obtaining the target bit rate of the audio and video transmission, and a reasonable audio bit rate service is adaptively provided to improve the audio and video call experience.

[0036] In the audio and video transmission scenario, the audio bit rate adjustment device of the audio and video transmission encodes the audio and video generated in real time to send the encoded audio and video stream to the corresponding receiving end. In addition, during the audio encoding process, the network congestion information of the current audio and video transmission is obtained to determine whether to adjust the audio encoding bit rate according to the network congestion information.

[0037] The network congestion information can be determined by the data sending end according to the data feedback information of the receiving end, or it can be obtained by obtaining the network congestion information calculated by the bandwidth detection module. The bandwidth detection module determines the one-way delay of the current audio and video data transmission and compares it with the set network congestion threshold. If the one-way delay is greater than the network congestion threshold, it is determined that the current audio and video transmission network is in a congested state. Otherwise, the current network state is normal. The network congestion threshold is determined in advance based on the actual network state for subsequent network congestion state detection. Exemplarily, the bandwidth detection module generates network congestion information every 20 milliseconds. The network congestion information is used to indicate whether the network is in a congested state, such as a mark of 1 for congestion, and a mark of 0 for non-congestion.

[0038] Furthermore, based on the acquired congestion information, by determining whether the current network is in a congested state, it is determined whether the current audio encoding bit rate is in a bit rate gear adjustment state. It is understandable that, generally speaking, if the current network is in a congested state, it may be necessary to appropriately lower the encoding bit rate to avoid stuttering and unsmooth audio and video calls. On the contrary, if the network state is determined to be better based on the congestion information, the bit rate can be appropriately increased. Based on this principle, the corresponding bit rate gear adjustment state of the current audio encoding bit rate can be determined based on the congestion information.

[0039] Specifically, when it is determined that the current audio encoding bit rate is in a bit rate gear adjustment state, the bit rate gear adjustment of the current audio encoding bit rate can be performed. By setting a plurality of different setting gears, wherein different setting gear configurations correspond to different audio bit rate values. Under different setting gear settings, the audio encoding bit rate is different. Exemplarily, a plurality of setting gears are set from high to low, and the bit rate value corresponding to the setting gear of the high bit rate is greater than the bit rate value corresponding to the setting gear of the low bit rate. In the bit rate gear adjustment state, the audio encoding bit rate can be changed by switching different bit rate gears, thereby realizing adaptive adjustment of the bit rate gear. Since different setting gears are configured with corresponding audio encoding bit rates, adaptive adjustment of the audio bit rate can be realized in this way.

[0040] When adjusting the bit rate gear, the target bit rate of audio and video transmission in the current acquisition cycle is obtained and combined with the bit rate gear of the current audio encoding bit rate to adjust the bit rate gear of the current audio encoding bit rate. Since in most scenarios, the video traffic and bit rate in audio and video calls are much higher than the audio traffic and bit rate, the video traffic accounts for the vast majority of the estimated target bit rate and congestion situation and plays a major role. Therefore, the embodiment of the present application is based on the congestion information and estimated target bit rate of both audio and video during audio and video transmission, and prioritizes the dynamic adjustment of the audio gear switch to achieve audio encoding bit rate adjustment.

[0041] Optionally, the estimated bit rate can be determined based on the set starting bit rate, real-time packet loss and network delay. During audio and video transmission, in each target bit rate estimation period, the target bit rate of the estimation period is calculated using the set target bit rate estimation model based on the set starting bit rate, the packet loss and network delay of the estimation period. The estimated target bit rate is used as the starting bit rate of the next estimation period, and so on. The target bit rate of audio and video data transmission is periodically estimated for subsequent audio encoding bit rate and video encoding bit rate adjustment.

[0042] Based on the embodiments of the present application, the audio quality (bit rate) is dynamically adjusted based on the network environment in a real-time audio and video transmission scenario. This can provide better sound quality when the network scenario is good, and provide a smoother audio experience when the network scenario is poor. This can better improve the audio and video call environment, thereby enhancing the user's overall audio and video call experience.

[0043] Below, the audio adjustment scheme of the embodiment of the present application is described in detail in combination with the network status.

[0044] Among them, the bit rate gear adjustment state of the embodiment of the present application includes an up-shift state; correspondingly, when judging whether the current audio encoding bit rate is in the bit rate gear adjustment state based on the congestion information, it is determined based on the congestion information that the audio and video transmission network has not been congested within the preset interval time, and it is determined that the current audio encoding bit rate is in the up-shift state.

[0045] It is understandable that if there is no congestion within the preset interval, it means that the current network status of audio and video transmission is better, and the audio encoding bit rate can be adaptively improved to provide a higher-definition audio bit rate service. If it is determined based on the congestion information that the audio and video transmission network is not congested within the preset interval, it is determined that the current audio encoding bit rate is in an upshift state, and the current audio encoding bit rate is ready to be adjusted up. The preset interval is set according to the actual bit rate adjustment requirements, and there is no fixed limit here.

[0046] Every set upshift period (such as 5S), it is determined whether the current audio and video transmission is in the upshift state according to the congestion information. If not, the original bit rate level is maintained for audio encoding. If yes, the preparation state for bit rate upshift adjustment is entered.

[0047] Optionally, before the bit rate gear is upgraded, the embodiment of the present application also determines whether the current audio and video transmission meets the set upgrade conditions. When it is determined that the current audio and video transmission meets the set upgrade conditions, the target bit rate of the audio and video transmission statistically calculated in the current acquisition period is obtained; the set upgrade conditions include: during the audio and video transmission process in the set time period, the audio is in a non-muted state; and the current bit rate gear has not reached the highest gear; and the time interval from the last audio bit rate upgrade exceeds a set interval threshold.

[0048] It is understandable that in order to avoid frequent bit rate switching and to avoid upgrading the bit rate at inappropriate times, the embodiment of the present application sets corresponding setting upgrading conditions according to the situation of inappropriate bit rate upgrading. Only when the setting upgrading conditions are met, the target bit rate of the audio and video transmission in the current acquisition cycle statistics is obtained, and the audio bit rate gear is upgraded. Otherwise, if it is determined that the current audio and video transmission does not meet the setting upgrading conditions, the original audio encoding gear is maintained to perform the audio encoding operation.

[0049] After that, the target bit rate of audio and video transmission calculated in the current acquisition cycle is combined with the bit rate level of the current audio encoding bit rate to adjust the bit rate level. Figure 2 The bit rate level up adjustment process includes:

[0050] S1301, determining a pre-configured corresponding upshift threshold according to the bit rate level of the current audio encoding bit rate;

[0051] S1302: Compare the minimum value of the target bit rate with the upshift threshold, and adjust the bit rate level of the current audio encoding bit rate according to the comparison result.

[0052] During a set target bit rate acquisition cycle, multiple target bit rate estimation cycles will be performed, multiple target bit rate estimation operations will be performed (such as estimating the target bit rate once every 1 second), and the corresponding multiple target bit rates will be determined. It can be understood that the target bit rate is the encoding bit rate value expected to be achieved during the current audio and video transmission. When the minimum value of the target bit rate exceeds the upshift threshold configured corresponding to the current bit rate gear, it means that the current bit rate gear cannot meet the encoding bit rate value expected to be achieved during the current audio and video transmission, and an upshift operation is required. Otherwise, there is no need to upshift, and the original bit rate gear is maintained for audio encoding.

[0053] Specifically, when the minimum value of the target bit rate is greater than the upshift threshold, the bit rate gear of the current audio encoding bit rate is upshifted by a set gear. It can be understood that, since multiple set gears are pre-set from high to low according to the audio bit rate value, in the upshift adjustment process, the bit rate gear of the audio encoding is adjusted upward by a set unit so that the bit rate gear of the audio encoding meets the encoding bit rate value (i.e., the target bit rate) expected to be achieved during the current audio and video transmission, thereby realizing the adaptive upshift adjustment of the audio encoding bit rate.

[0054] On the other hand, the bit rate gear adjustment state of the embodiment of the present application includes a downshift state; correspondingly, when judging whether the current audio encoding bit rate is in the bit rate gear adjustment state based on the congestion information, it is determined based on the congestion information that a congestion occurs in the audio and video transmission network, and it is determined that the current audio encoding bit rate is in a downshift state.

[0055] It is understandable that if the audio and video transmission network is determined to be congested based on the congestion information, it means that the current audio and video transmission network status is poor, and the audio encoding bit rate needs to be adaptively reduced to avoid the situation where the audio and video transmission is stuck due to excessive bit rate. If the audio and video transmission network is determined to be congested based on the congestion information, the current audio encoding bit rate is determined to be in a downshift state, and the current audio encoding bit rate gear is prepared to be downshifted.

[0056] Every set downshift period (such as 5S), the system will determine whether the current audio and video transmission is in a downshift state based on the congestion information. If not, the original bit rate level is maintained for audio encoding. If yes, the system will enter a ready state for bit rate downshift adjustment.

[0057] Optionally, before downshifting the bit rate gear, the embodiment of the present application also determines whether the current audio and video transmission meets the set downshift conditions. When it is determined that the current audio and video transmission meets the set downshift conditions, the target bit rate of the audio and video transmission statistics of the current acquisition period is obtained; the set downshift conditions include: during the audio and video transmission process of the set time period, the audio is in a non-muted state; and the current bit rate gear has not reached the lowest gear; and the time interval from the last audio bit rate downshift exceeds the set interval threshold.

[0058] It is understandable that in order to avoid frequent bit rate switching and downshifting at inappropriate times, the embodiment of the present application sets corresponding downshift conditions according to the situation of inappropriate downshifting of the bit rate. Only when the set downshift conditions are met, the target bit rate of the audio and video transmission statistics of the current acquisition period is obtained, and the downshift adjustment of the audio bit rate is performed. Otherwise, if it is determined that the current audio and video transmission does not meet the set downshift conditions, the original audio encoding gear is maintained to perform the audio encoding operation.

[0059] After that, the target bit rate of audio and video transmission calculated in the current acquisition cycle is used to adjust the bit rate down in combination with the bit rate level of the current audio encoding bit rate. Figure 3 , the bit rate gear downshift adjustment process includes:

[0060] S1303, determining a pre-configured downshift threshold value according to the bit rate level of the current audio encoding bit rate;

[0061] S1304: Compare the maximum value of the target bit rate with the downshift threshold, and adjust the bit rate level of the current audio encoding bit rate according to the comparison result.

[0062] It is understandable that the target bit rate is the encoding bit rate value expected to be achieved during the current audio and video transmission. When the maximum value of the target bit rate is still less than the downshift threshold configured for the current bit rate gear, it means that the current bit rate gear cannot meet the encoding bit rate value expected to be achieved during the current audio and video transmission, and a downshift operation is required. Otherwise, there is no need to downshift, and the original bit rate gear is maintained for audio encoding.

[0063] Specifically, when the maximum value of the target bit rate is less than the downshift threshold, the bit rate gear of the current audio encoding bit rate is downshifted by a set gear. It can be understood that, since multiple set gears are set in advance from high to low according to the audio bit rate value, in the downshift adjustment process, the bit rate gear of the audio encoding is adjusted downward by a set unit so that the bit rate gear of the audio encoding meets the encoding bit rate value (i.e., the target bit rate) expected to be achieved during the current audio and video transmission, thereby realizing the adaptive downshift adjustment of the audio encoding bit rate.

[0064] Illustratively, in the entire audio bit rate process, the embodiment of the present application will evaluate the periodically updated congestion information and the target bit rate value according to the actual scenario, and then flexibly select the bit rate gear to be raised or lowered.

[0065] Optionally, taking 16k bit rate as the standard (initial) bit rate for example to dynamically adjust the bit rate level, the bit rate level is divided into three setting levels: standard level (16k), ordinary level (24k) and high definition level (32k).

[0066] When adjusting the bit rate, the bit rate is first in a steady state, and the audio encoding bit rate is fixed at a certain bit rate. After obtaining the congestion information and the target bit rate, the information is updated and counted, and then the upshift and downshift cycles are respectively used to make upshift and downshift judgments.

[0067] Reference Figure 4 When adjusting the bit rate gear up, first determine whether to enter the upshift state based on the congestion information. After entering the upshift state, determine whether the bit rate gear is upshifted based on the set upshift conditions. If any of the set upshift conditions is not met, the bit rate gear will not be upshifted, and the bit rate gear state returns to a steady state. Then continue to update the statistical congestion information and target bit rate until the next upshift cycle is judged to come.

[0068] If it is determined that the upshift condition is met, the target bit rate minimum value min is obtained, and the current bit rate gear is determined to be a standard gear or a normal gear. It can be understood that since the highest gear is a high-definition gear (32k), the upshift condition is not met at the highest gear, so the current bit rate gear must be a standard gear or a normal gear.

[0069] Further, the upshifting threshold preset for the current bitrate gear is compared with the target bitrate minimum value min to determine how to upshift:

[0070] If the current bitrate gear is the standard gear, and the target bitrate minimum value min is greater than the standard gear upshift threshold, the bitrate gear is upgraded to the normal gear, and the audio encoding is performed at the audio bitrate value 24K corresponding to the normal gear. Otherwise, the bitrate gear adjustment state returns to the steady state, and the audio encoding bitrate value 16K corresponding to the standard gear is maintained for audio bitrate encoding.

[0071] If the current bitrate gear is the normal gear, and the target bitrate minimum value min is greater than the normal gear upshift threshold, the bitrate gear is upgraded to the HD gear, and the audio encoding is performed at the audio bitrate value 32K corresponding to the HD gear. Otherwise, the bitrate gear adjustment state returns to the steady state, and the audio encoding bitrate value 24K corresponding to the normal gear is maintained for audio bitrate encoding.

[0072] On the other hand, referring to Figure 5 When adjusting the bit rate downshift, first determine whether to enter the downshift state based on the congestion information. After entering the downshift state, determine whether the bit rate gear should be downshifted based on the set downshift conditions. If any of the set downshift conditions is not met, the bit rate gear will not be downshifted, and the bit rate gear state returns to a steady state. Then continue to update the statistical congestion information and target bit rate until the next downshift cycle is judged to come.

[0073] If it is determined that the downshift condition is met, the maximum value of the target bit rate max is obtained, and the current bit rate gear is determined to be a high-definition gear or a normal gear. It can be understood that since the lowest gear is the standard gear (16k), the downshift condition is not met at the lowest gear, so the current bit rate gear must be a high-definition gear or a normal gear.

[0074] Further, the downshifting threshold preset at the current bitrate gear is compared with the target bitrate maximum value max to determine how to downshift:

[0075] If the current bitrate gear is the normal gear, and the target bitrate maximum value max is greater than the downshift threshold of the standard gear, the bitrate gear is downshifted to the standard gear, and the audio encoding is performed at the audio bitrate value 16K corresponding to the standard gear. Otherwise, the bitrate gear adjustment state returns to the steady state, and the audio encoding bitrate value 24K corresponding to the normal gear is maintained for audio bitrate encoding.

[0076] If the current bitrate gear is the HD gear, and the target bitrate maximum value max is greater than the downshift threshold of the HD gear, the bitrate gear is downshifted to the normal gear, and the audio encoding is performed at the audio bitrate value 24K corresponding to the normal gear. Otherwise, the bitrate gear adjustment state returns to the steady state, and the audio encoding bitrate value 32K corresponding to the HD gear is maintained for audio bitrate encoding.

[0077] In audio and video transmission scenarios, the audio bit rate is dynamically adjusted according to network congestion information and target bit rate to provide smooth audio quality services as much as possible in congested scenarios, and provide audio high-definition services as much as possible in scenarios with good network conditions, thereby improving the overall user audio and video call experience.

[0078] Optionally, after completing the audio bit rate adjustment, the embodiment of the present application further determines the target video bit rate by subtracting the target bit rate from the current corresponding audio bit rate value, and performs video encoding for the current audio and video transmission based on the target video bit rate.

[0079] By incorporating the audio bitrate determined in the above audio encoding process into the considerations for adjusting the video encoding bitrate, the encoding bitrate of the video can be dynamically affected. When the audio bitrate is updated according to the up / downshift, the target bitrate minus the updated audio bitrate is the target bitrate for the video, and the video target bitrate is used as input to dynamically adjust the input parameters of the video bitrate. At the same time, the bitrate is synchronized to the sending bitrate setting, so that audio takes priority over the sending bitrate. In this way, dynamic bitrate adjustment of both audio and video in audio and video transmission can be achieved, further optimizing the bitrate adjustment effect and improving the user's audio and video call experience.

[0080] In the above, by obtaining the congestion information of the current audio and video transmission, it is determined whether the current audio encoding bit rate is in the bit rate gear adjustment state based on the congestion information. Then, when it is determined that the current audio encoding bit rate is in the bit rate gear adjustment state, the target bit rate of the audio and video transmission statistically calculated in the current acquisition period is obtained, and the bit rate gear of the current audio encoding bit rate is determined, and the bit rate gear is pre-configured with the corresponding audio bit rate value according to different set gears. Afterwards, based on the target bit rate and the bit rate gear of the current audio encoding bit rate, the bit rate gear of the current audio encoding bit rate is adjusted, and the audio encoding of the current audio and video transmission is performed based on the audio bit rate value corresponding to the adjusted bit rate gear. The above technical means are adopted to adjust the audio bit rate by combining the network congestion situation and the target bit rate of the audio and video transmission, and the audio bit rate is adaptively adjusted according to the real-time network situation, so as to adaptively provide reasonable audio bit rate services, optimize the audio and video call effect, and improve the audio and video call experience.

[0081] Based on the above embodiments, Figure 6 This is a schematic diagram of the structure of an audio bit rate adjustment system for audio and video transmission provided by this application. Figure 6 The audio bit rate adjustment system for audio and video transmission provided in this embodiment specifically includes: a state judgment module 21, an acquisition module 22 and a bit rate adjustment module 23.

[0082] The state determination module 21 is configured to obtain the congestion information of the current audio and video transmission, and determine whether the current audio encoding bit rate is in a bit rate gear adjustment state based on the congestion information;

[0083] The acquisition module 22 is configured to obtain the target bit rate of the audio and video transmission counted in the current acquisition period and determine the bit rate gear of the current audio encoding bit rate when determining that the current audio encoding bit rate is in the bit rate gear adjustment state, and the bit rate gear is pre-configured with the corresponding audio bit rate value according to different set gears;

[0084] The bit rate adjustment module 23 is configured to adjust the bit rate level of the current audio encoding bit rate based on the target bit rate and the bit rate level of the current audio encoding bit rate, and perform audio encoding for the current audio and video transmission based on the audio bit rate value corresponding to the adjusted bit rate level.

[0085] Specifically, the bit rate gear adjustment state includes an up-shift state; the state judgment module 21 is configured to determine based on the congestion information that the audio and video transmission network is not congested within a preset interval time, and determine that the current audio encoding bit rate is in an up-shift state.

[0086] Specifically, the acquisition module 22 is configured to obtain the target bit rate of the audio and video transmission counted in the current acquisition period when it is determined that the current audio and video transmission meets the set upshift conditions; the set upshift conditions include: during the audio and video transmission process in the set time period, the audio is in a non-muted state; and the current bit rate gear has not reached the highest gear; and the time interval from the last audio bit rate upshift exceeds the set interval threshold.

[0087] Specifically, the bit rate adjustment module 23 is configured to determine a pre-configured upshift threshold value according to the bit rate gear of the current audio encoding bit rate; compare the minimum value of the target bit rate with the upshift threshold value, and adjust the bit rate gear of the current audio encoding bit rate according to the comparison result. Wherein, when the minimum value of the target bit rate is greater than the upshift threshold value, the bit rate gear of the current audio encoding bit rate is upshifted by a set gear.

[0088] Specifically, the bit rate gear adjustment state includes a downshift state; the state judgment module 21 is configured to determine that a congestion state occurs in the audio and video transmission network based on the congestion information, and determine that the current audio encoding bit rate is in a downshift state.

[0089] Specifically, the acquisition module 22 is configured to obtain the target bit rate of the audio and video transmission counted in the current acquisition period when it is determined that the current audio and video transmission meets the set downshift conditions; the set downshift conditions include: during the audio and video transmission process in the set time period, the audio is in a non-muted state; and the current bit rate gear has not reached the lowest gear; and the time interval from the last audio bit rate downshift exceeds the set interval threshold.

[0090] Specifically, the bit rate adjustment module 23 is configured to determine a pre-configured downshift threshold value according to the bit rate gear of the current audio encoding bit rate; compare the maximum value of the target bit rate with the downshift threshold value, and adjust the bit rate gear of the current audio encoding bit rate according to the comparison result. Wherein, when the maximum value of the target bit rate is less than the downshift threshold value, the bit rate gear of the current audio encoding bit rate is downshifted by a set gear.

[0091] The audio bit rate adjustment system for audio and video transmission is also configured to determine a target video bit rate by subtracting a target bit rate from a current corresponding audio bit rate value, and perform video encoding for the current audio and video transmission based on the target video bit rate.

[0092] In the above, by obtaining the congestion information of the current audio and video transmission, it is determined whether the current audio encoding bit rate is in the bit rate gear adjustment state based on the congestion information. Then, when it is determined that the current audio encoding bit rate is in the bit rate gear adjustment state, the target bit rate of the audio and video transmission statistically calculated in the current acquisition period is obtained, and the bit rate gear of the current audio encoding bit rate is determined, and the bit rate gear is pre-configured with the corresponding audio bit rate value according to different set gears. Afterwards, based on the target bit rate and the bit rate gear of the current audio encoding bit rate, the bit rate gear of the current audio encoding bit rate is adjusted, and the audio encoding of the current audio and video transmission is performed based on the audio bit rate value corresponding to the adjusted bit rate gear. The above technical means are adopted to adjust the audio bit rate by combining the network congestion situation and the target bit rate of the audio and video transmission, and the audio bit rate is adaptively adjusted according to the real-time network situation, so as to adaptively provide reasonable audio bit rate services, optimize the audio and video call effect, and improve the audio and video call experience.

[0093] The audio bit rate adjustment system for audio and video transmission provided in the embodiment of the present application can be configured to execute the audio bit rate adjustment method for audio and video transmission provided in the above embodiment, and has corresponding functions and beneficial effects.

[0094] Based on the above practical example, the present application embodiment also provides an audio bit rate adjustment device for audio and video transmission, referring to Figure 7, the audio bit rate adjustment device for audio and video transmission includes: a processor 31, a memory 32, a communication module 33, an input device 34 and an output device 35. The memory 32, as a computer-readable storage medium, can be configured to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the audio bit rate adjustment method for audio and video transmission described in any embodiment of the present application (for example, a state judgment module, an acquisition module and a bit rate adjustment module in the audio bit rate adjustment system for audio and video transmission). The communication module 33 is configured to perform data transmission. The processor 31 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory, that is, to implement the above-mentioned audio bit rate adjustment method for audio and video transmission. The input device 34 can be configured to receive input digital or character information, and generate key signal input related to the user settings and function control of the device. The output device 35 may include a display device such as a display screen. The audio bit rate adjustment device for audio and video transmission provided above can be configured to execute the audio bit rate adjustment method for audio and video transmission provided in the above-mentioned embodiment, and has corresponding functions and beneficial effects.

[0095] On the basis of the above embodiments, the embodiments of the present application further provide a storage medium containing computer executable instructions, wherein the computer executable instructions are configured to execute a method for adjusting the audio bit rate of audio and video transmission when executed by a computer processor, and the storage medium may be any of various types of memory devices or storage devices. Of course, the storage medium containing computer executable instructions provided in the embodiments of the present application, whose computer executable instructions are not limited to the method for adjusting the audio bit rate of audio and video transmission as described above, may also execute related operations in the method for adjusting the audio bit rate of audio and video transmission provided in any embodiment of the present application.

[0096] On the basis of the above embodiments, the embodiments of the present application also provide a computer program product. The technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium, including a number of instructions for enabling a computer device, a mobile terminal or a processor therein to execute all or part of the steps of the audio bit rate adjustment method for audio and video transmission described in each embodiment of the present application.

Claims

1. A method for adjusting the audio bit rate of audio and video transmission, characterized in that: include: Obtaining congestion information of current audio and video transmission, and determining whether the current audio encoding bit rate is in a bit rate gear adjustment state based on the congestion information; When it is determined that the current audio encoding bit rate is in the bit rate gear adjustment state, the target bit rate of the audio and video transmission statistics of the current acquisition period is obtained, and the bit rate gear of the current audio encoding bit rate is determined, and the bit rate gear is pre-configured with the corresponding audio bit rate value according to different setting gears; Based on the target bit rate and the bit rate gear of the current audio encoding bit rate, adjust the bit rate gear of the current audio encoding bit rate, and perform audio encoding for the current audio and video transmission based on the audio bit rate value corresponding to the adjusted bit rate gear; The bit rate gear adjustment state includes an upshift state; The determining whether the current audio encoding bit rate is in a bit rate gear adjustment state based on the congestion information includes: Determining based on the congestion information that the audio and video transmission network is not congested within a preset interval, and determining that the current audio encoding bit rate is in an upshift state; The adjusting the bit rate level of the current audio encoding bit rate based on the target bit rate and the bit rate level of the current audio encoding bit rate includes: Determining a pre-configured corresponding upshift threshold according to the bit rate level of the current audio encoding bit rate; Comparing the minimum value of the target bit rate with the upshift threshold, and adjusting the bit rate gear of the current audio encoding bit rate according to the comparison result, wherein when the minimum value of the target bit rate is greater than the upshift threshold, the bit rate gear of the current audio encoding bit rate is upshifted by a set gear; By performing multiple target bit rate estimation cycles within a set target bit rate acquisition cycle, multiple target bit rate estimation operations are performed to determine the corresponding multiple target bit rates, and the minimum value of the target bit rate is determined from the determined multiple target bit rates.

2. The method for adjusting the audio bit rate of audio and video transmission according to claim 1, characterized in that: The step of obtaining the target bit rate of audio and video transmission based on statistics of the current acquisition period includes: When it is determined that the current audio and video transmission meets the set upshift condition, the target bit rate of the audio and video transmission in the current acquisition period is obtained; The setting of the upshift condition includes: During the audio and video transmission process of the set period, the audio is not muted; and, The current bit rate level does not reach the highest level; and, The time interval since the last audio bitrate upscaling exceeds the set interval threshold.

3. The method for adjusting the audio bit rate of audio and video transmission according to claim 1, characterized in that: The bit rate gear adjustment state includes a downshift state; The determining whether the current audio encoding bit rate is in a bit rate gear adjustment state based on the congestion information includes: Based on the congestion information, it is determined that the audio and video transmission network is in a congestion state, and it is determined that the current audio encoding bit rate is in a downshift state.

4. The method for adjusting the audio bit rate of audio and video transmission according to claim 3, characterized in that: The step of obtaining the target bit rate of audio and video transmission based on statistics of the current acquisition period includes: When it is determined that the current audio and video transmission meets the set downshift condition, the target bit rate of the audio and video transmission in the current acquisition period is obtained; The setting downshift condition includes: During the audio and video transmission process of the set period, the audio is not muted; and, The current bit rate level does not reach the lowest level; and, The time interval since the last audio bitrate downshift exceeds the set interval threshold.

5. The method for adjusting the audio bit rate of audio and video transmission according to claim 4, characterized in that: The adjusting the bit rate level of the current audio encoding bit rate based on the target bit rate and the bit rate level of the current audio encoding bit rate includes: Determining a pre-configured downshift threshold value according to the bit rate level of the current audio encoding bit rate; Comparing the maximum value of the target bit rate with the downshift threshold, and adjusting the bit rate gear of the current audio encoding bit rate according to the comparison result, wherein, when the maximum value of the target bit rate is less than the downshift threshold, the bit rate gear of the current audio encoding bit rate is downshifted by a set gear; By performing multiple target bit rate estimation cycles within a set target bit rate acquisition cycle, multiple target bit rate estimation operations are performed to determine the corresponding multiple target bit rates, and the maximum value of the target bit rate is determined from the determined multiple target bit rates.

6. The method for adjusting the audio bit rate of audio and video transmission according to claim 1, characterized in that: After performing audio encoding of the current audio and video transmission based on the audio bit rate value corresponding to the adjusted bit rate gear, the method further includes: The target video bit rate is determined by subtracting the target bit rate from the currently corresponding audio bit rate value, and video encoding of the current audio and video transmission is performed based on the target video bit rate.

7. An audio bit rate adjustment system for audio and video transmission, characterized in that: include: A state judgment module is configured to obtain congestion information of the current audio and video transmission, and judge whether the current audio encoding bit rate is in a bit rate gear adjustment state based on the congestion information; An acquisition module is configured to acquire a target bit rate of audio and video transmission counted in a current acquisition period and determine the bit rate gear of the current audio encoding bit rate when determining that the current audio encoding bit rate is in the bit rate gear adjustment state, wherein the bit rate gear is pre-configured with a corresponding audio bit rate value according to different set gears; A bit rate adjustment module is configured to adjust the bit rate gear of the current audio encoding bit rate based on the target bit rate and the bit rate gear of the current audio encoding bit rate, and perform audio encoding of the current audio and video transmission based on the audio bit rate value corresponding to the adjusted bit rate gear; The bit rate gear adjustment state includes an upshift state; The determining whether the current audio encoding bit rate is in a bit rate gear adjustment state based on the congestion information includes: Determining based on the congestion information that the audio and video transmission network is not congested within a preset interval, and determining that the current audio encoding bit rate is in an upshift state; The adjusting the bit rate level of the current audio encoding bit rate based on the target bit rate and the bit rate level of the current audio encoding bit rate includes: Determining a pre-configured corresponding upshift threshold according to the bit rate level of the current audio encoding bit rate; The minimum value of the target bit rate is compared with the upshift threshold, and the bit rate gear of the current audio encoding bit rate is adjusted according to the comparison result, wherein, when the minimum value of the target bit rate is greater than the upshift threshold, the bit rate gear of the current audio encoding bit rate is upshifted by a set gear; by performing multiple target bit rate estimation cycles within the acquisition cycle of the set target bit rate, performing multiple target bit rate estimation operations, determining the corresponding multiple target bit rates, and determining the minimum value of the target bit rate from the determined multiple target bit rates.

8. An audio bit rate adjustment device for audio and video transmission, characterized in that: include: memory and one or more processors; The memory is configured to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the audio bit rate adjustment method for audio and video transmission as described in any one of claims 1-6.

9. A storage medium containing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are configured to execute the audio bit rate adjustment method for audio and video transmission as described in any one of claims 1-6.

10. A computer program product, characterized in that The computer program product includes instructions, and when the instructions are executed on a computer or a processor, the computer or the processor executes the audio bit rate adjustment method for audio and video transmission as described in any one of claims 1-6.

Images