Method for IP network transmission of audio data

Abstract

The application relates to the technical field of digital information transmission, in particular to an IP network transmission method of audio data, which comprises the following steps: determining an audio packet transmission sequence and an audio packet receiving sequence by using the information variation in an audio packet; determining the transmission interval and the receiving interval between adjacent audio packets under the same serial number in the audio packet transmission sequence and the audio packet receiving sequence respectively; determining the transmission sensitivity degree of a target audio packet by using the transmission interval and the receiving interval, and determining an abnormal audio packet by using the transmission sensitivity degree; determining the transmission convergence degree of a transmission topology network where a target node is located by using the transmission sensitivity degree of the abnormal audio packet in the target node; determining the target adjusting parameter of a jitter buffer by using the transmission convergence degree of the transmission topology network, and correcting the capacity of the jitter buffer by using the target adjusting parameter. Through the technical scheme, the jitter delay phenomenon in the audio packet transmission process is reduced, the audio quality is ensured, and the playing delay is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital information transmission, in particular to an IP network transmission method of audio data. BACKGROUND

[0002] With the continuous development of broadcasting and audio technology, people's demand for real-time information is growing, and the wide application of live broadcast in various commercial, public welfare and other scenarios makes it difficult for traditional broadcast center audio transmission to meet the growing demand for high-quality audio transmission. Therefore, it is of great significance to study how to realize high-quality, low-cost, real-time audio network transmission equipment that can adapt to local area network, wide area network and wireless network environment.

[0003] The rapid progress of current network communication technology, the real-time audio and video transmission application with IP (Internet Protocol Network) network as the transmission medium is increasingly widespread. Because IP network uses best effort delivery mechanism to transmit data packets, the sending end sends audio packets at approximately equal intervals, but due to the size of each packet and the time difference in queuing and processing during transmission, when network congestion, transmission errors and packet processing delays occur, etc. Uncertain factors, when transmitting voice packets in a packet network, it is easy to cause packet loss, delay and jitter, etc. Problems, resulting in the incoherence or even interruption of the audio information at the receiving end.

[0004] In order to reduce the delay jitter in the process of audio data transmission, the existing method constructs a jitter buffer, temporarily stores the received audio packets at the receiving end, waits for the arrival of other data packets, and uses the waiting time of data in the buffer to eliminate the time interval difference between packet data packets. However, due to the fixed size of the jitter buffer, when the delay jitter caused by network lag exceeds the default threshold, the buffer is empty, which will still cause audio interruption. SUMMARY

[0005] In order to solve the technical problem that the jitter buffer capacity constructed by the existing audio data transmission scheme is fixed and easy to cause audio interruption when the network delay is stalled, the purpose of the present application is to provide an IP network transmission method of audio data, and the technical scheme adopted is as follows:

[0006] The present application provides an IP network transmission method of audio data, the method comprising:

[0007] Determine the audio packet transmission sequence composed of each audio packet sequence using the information change amount in the audio packet, and determine the audio packet reception sequence using the audio packet reception timestamp;

[0008] Determine the transmission interval and reception interval between adjacent audio packets at the same sequence number in the audio packet transmission sequence and the audio packet reception sequence, respectively;

[0009] determining the transmission sensitivity of the target audio packet by using the transmission interval and the receiving interval of the target audio packet, and determining the abnormal audio packet in the target audio packet by using the transmission sensitivity;

[0010] determining the transmission convergence degree of the transmission topology network in which the target node is located by using the transmission sensitivity of the abnormal audio packet in the target node;

[0011] determining the target adjustment parameter of the jitter buffer by using the transmission convergence degree of the transmission topology network, and correcting the capacity of the jitter buffer by using the target adjustment parameter.

[0012] Further, the determination of the audio packet transmission sequence composed of the audio packets sorted by the information change amount in each audio packet comprises:

[0013] determining the data importance degree of the audio packet by using the information change amount in the audio packet;

[0014] obtaining the audio packet transmission sequence by performing priority sorting on the audio packets by using the data importance degree.

[0015] Further, the determination of the data importance degree of the audio packet by using the information change amount in the audio packet comprises:

[0016] performing empirical mode decomposition on the audio packet to obtain the intrinsic mode function component of the audio packet, and taking the variance of the intrinsic mode function component as the information change amount in the audio packet;

[0017] calculating the data importance degree of the audio packet by using the information change amount in the audio packet, and the maximum information change amount and the minimum information change amount in all audio packets.

[0018] Further, the determination of the transmission sensitivity of the target audio packet by using the transmission interval and the receiving interval of the target audio packet comprises:

[0019] taking the difference between the transmission interval and the receiving interval of the target audio packet as the transmission time difference of the target audio packet;

[0020] determining the transmission sensitivity of the target audio packet by using the transmission time difference of the target audio packet and the average of the transmission time differences of all audio packets.

[0021] Further, the determination of the transmission sensitivity of the target audio packet by using the transmission time difference of the target audio packet and the average of the transmission time differences of all audio packets comprises:

[0022] obtaining the content correlation between the target audio packets in the same sequence number in the audio packet transmission sequence and the audio packet receiving sequence by using the Pearson correlation coefficient;

[0023] The transmission sensitivity degree of the target audio packet is calculated by using the transmission time difference of the target audio packet, the mean of transmission time difference of all audio packets, and the content correlation.

[0024] Further, the transmission convergence degree of the transmission topology network where the target node is located is determined by using the transmission sensitivity degree of the abnormal audio packet in the target node.

[0025] The transmission load abnormality degree of the target node is determined by using the transmission sensitivity degree of the abnormal audio packet in the target node and the number of abnormal audio packets.

[0026] The transmission convergence degree of the transmission topology network is determined by using the transmission load abnormality degree of the target node and the total number of nodes in the transmission topology network where the target node is located.

[0027] Further, the transmission load abnormality degree of the target node is determined by using the transmission sensitivity degree of the abnormal audio packet in the target node and the number of abnormal audio packets.

[0028] The total number of all abnormal audio packets in the transmission topology network where the target node is located and the mean of transmission sensitivity degree of the abnormal audio packet in the target node are determined.

[0029] The transmission load abnormality degree of the target node is calculated by using the total number of all abnormal audio packets, the mean of transmission sensitivity degree, and the number of abnormal audio packets.

[0030] Further, the transmission convergence degree of the transmission topology network is determined by using the transmission load abnormality degree of the target node and the total number of nodes in the transmission topology network where the target node is located.

[0031] The transmission load abnormality degree difference between the target node and the adjacent node in the network transmission direction is determined.

[0032] The transmission convergence degree of the transmission topology network is calculated by using the transmission load abnormality degree of the target node, the total number of nodes in the transmission topology network where the target node is located, and the transmission load abnormality degree difference.

[0033] Further, the target adjustment parameter of the jitter buffer is determined by using the transmission convergence degree of the transmission topology network.

[0034] The abnormal transmission interval and the abnormal reception interval between the abnormal audio packet and its adjacent normal audio packet under the same serial number in the audio packet transmission sequence and the audio packet reception sequence are respectively determined.

[0035] The target adjustment parameter of the jitter buffer is calculated by using the transmission convergence degree of the transmission topology network, the abnormal transmission interval, and the abnormal reception interval.

[0036] Further, the target adjustment parameter is used to correct the jitter buffer capacity, comprising:

[0037] The target adjustment parameter and the initial jitter buffer capacity are used to calculate the corrected jitter buffer capacity.

[0038] The present application has the following advantages:

[0039] The present application is based on audio data decoding analysis and audio data transmission, determines the key audio in the transmission process to obtain the audio packet transmission sequence, ensures the key audio packet transmission priority, further compares the audio packet receiving sequence of the receiving end with the audio packet transmission sequence of the sending end, analyzes the difference between the audio packet receiving sequence and the audio packet transmission sequence, screens out the transmission abnormal audio data, further determines the transmission convergence degree of the transmission topology network by analyzing the transmission path of the abnormal audio data, and adaptively constructs the adjustment parameter according to the transmission convergence degree, and then uses the adjustment parameter to dynamically adjust the jitter buffer capacity size, and provides more buffering time for other less important audio data packets, reduces the jitter delay phenomenon in the audio packet transmission process, ensures the audio quality while reducing the playing delay, and reduces or avoids the interruption phenomenon of the audio in the transmission process. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art and the advantages thereof, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0041] Figure 1 A step flowchart of an IP network transmission method of audio data provided by an embodiment of the present application;

[0042] Figure 2 A detailed flowchart of step S1 in the IP network transmission method of audio data provided by an embodiment of the present application;

[0043] Figure 3 A detailed flowchart of step S3 in the IP network transmission method of audio data provided by another embodiment of the present application;

[0044] Figure 4 A detailed flowchart of step S4 in the IP network transmission method of audio data provided by an embodiment of the present application;

[0045] Figure 5A detailed flow chart of step S5 in the IP network transmission method of audio data provided by one embodiment of the present application;

[0046] Figure 6 A structural schematic diagram of the hardware running environment of the IP network transmission device of audio data involved in the embodiment scheme of the present application;

[0047] Figure 7 A transmission topology network node schematic diagram in the IP network transmission method of audio data involved in the embodiment scheme of the present application. DETAILED DESCRIPTION

[0048] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined purposes, the specific embodiments, structures, features and effects of the IP network transmission method of audio data according to the present application are described in detail as follows in combination with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0050] For the specific scene targeted by each embodiment of the present application, the following can be provided:

[0051] When transmitting the audio signals collected in real time in the field in a one-way manner at a long distance, the IP network is used to transmit the audio data of news relay, and the sending end divides the audio data into multiple audio packets, each of which is transmitted independently. However, due to the problems such as network environment delay, these audio packets experience different transmission paths during transmission, resulting in fluctuations in the time of audio packets arriving at the receiving end, and even packet loss phenomenon.

[0052] In view of the above specific scene and the problems existing in the prior art, the technical concept of the present application mainly includes:

[0053] The fixed-size jitter buffer cannot be dynamically adjusted according to network conditions and the order of received audio packets, if the network jitter is large, the transmission of the current audio packet can be more chaotic, when the buffer is read empty, audio interruption or loss will still be caused; if the network jitter is small, the fixed-size jitter buffer can increase unnecessary audio delay. Therefore, the embodiment in the application analyzes the audio data, designs a transmission sequence according to the transmission criticality of different audio packets, ensures that the critical audio packets are transmitted preferentially, analyzes the difference between the receiving sequence and the transmission sequence at the receiving end, obtains the transmission sensitivity of the audio packets in the transmission process, constructs an adjustment parameter, further combines the utilization rate of the network bandwidth, dynamically adjusts the size of the jitter buffer, and guarantees the audio quality while reducing the playing delay.

[0054] The specific scheme of the IP network transmission method for audio data provided by the application will be specifically described below in combination with the drawings.

[0055] Embodiment one:

[0056] For the IP network transmission method for audio data provided by the application, refer to Figure 1 , which shows the step flowchart of the IP network transmission method for audio data provided by an embodiment of the application.

[0057] The IP network transmission method for audio data comprises:

[0058] Step S1, determining an audio packet transmission sequence composed of audio packets sorted by the information change amount in each audio packet, and determining an audio packet receiving sequence by the receiving time stamp of the audio packet;

[0059] In this embodiment, the collection and preprocessing of the audio data can be realized by the following implementation manner:

[0060] The original audio data of the sending end in the live broadcast process is collected by an audio device, and the sound channel format is selected as 5.1 surround sound. The collected audio data is denoised; including:

[0061] Noise sampling: record a pure noise audio as a reference value of the environmental noise floor;

[0062] Denoising processing: the noise characteristics of the environmental noise floor segment and the original audio data are identified by using the IZotope RX audio repair tool, the collected original audio data is denoised by difference subtraction, the audio file after denoising is obtained, the audio file after denoising is saved as a to-be-transmitted audio file, and then the to-be-transmitted audio file is transmitted to the receiving end through the IP network.

[0063] In addition, other existing technical means such as audio data collection and denoising preprocessing can also be used, which will not be described in detail here.

[0064] Specifically, referring to Figure 2 , the step S1 of determining the audio packet transmission sequence composed of the audio packets sorted by the information variation in the audio packets comprises:

[0065] The step S11 of determining the data importance degree of the audio packet by using the information variation in the audio packet comprises:

[0066] The step S11 specifically comprises:

[0067] Performing empirical mode decomposition on the audio packet to obtain an intrinsic mode function component of the audio packet, and taking the variance of the intrinsic mode function component as the information variation in the audio packet.

[0068] The data importance degree of the audio packet is calculated by using the information variation in the audio packet and the maximum information variation and the minimum information variation in all audio packets.

[0069] The step S12 of performing priority sorting on the audio packets by using the respective data importance degrees to obtain the audio packet transmission sequence.

[0070] In this embodiment, the audio data contains information of multiple noisy sources, such as multi-person conversation, sentence pause, etc. When the audio data is transmitted by using an IP network, more computing resources are needed to process the redundant audio data, especially in the case of multi-person conversation or sentence intonation, the transmission of the overall audio will significantly increase the pressure of the transmission network, and therefore the audio file to be transmitted needs to be segmented for processing, and then the transmission priority is sorted in combination with the information variation complexity of different audio packets.

[0071] The voice tone change points of different speakers in the audio data are detected by using a pre-trained speaker recognition model, and the sentence pause points in the collected audio are detected by using a voice activity model. The union of the voice tone change points and the sentence pause points is taken as a segmentation boundary, and the audio data is segmented into multiple small audio packets.

[0072] The segmented audio packets are sampled into digital signals by analog-to-digital conversion (ADC), and the sampling results are normalized between [-1, 1]. The corresponding digital signals of the audio packets are processed by empirical mode decomposition (EMD) to obtain a group of intrinsic mode function (IMF) components and a residual term of the audio signal.

[0073] The IMF component represents the local oscillation characteristics of the audio packet at different frequency scales, such as information of high-frequency consonant explosion and low-frequency vowel resonance. The variance of the IMF component in each audio packet is calculated as the information variation of the corresponding audio packet. .

[0074] In normal conversation, the vibration frequency of the sound is usually relatively stable, and the key data in the audio usually includes the starting point of the sentence, the turning point and other positions. The audio at these positions contains a large amount of high-frequency information and a large amount of information change. The part of the audio information is relatively stable, such as the silent section, the pause section between different people's conversations, or the important degree of the smooth speed stage is smaller. Therefore, the data importance degree of each audio packet is measured by (maximum minimum value) normalization:

[0075] ;

[0076] Among them, represents the data importance degree of the tth audio packet (represents any audio packet, and can also be recorded as a target audio packet, the same below); represents the tth audio packet (represents any audio packet, and can also be recorded as a target audio packet, the same below); represents the information change amount contained in the EMD decomposition of the tth audio packet; The function is the maximum value function, which corresponds to the maximum information change amount; The function is the minimum value function, which corresponds to the minimum information change amount; The function is the minimum value function, which corresponds to the minimum information change amount; represents the collection time stamp of the tth audio packet.

[0077] When the information change amount in the audio packet is larger, The calculation result is larger, which indicates that the fluctuation of the frequency component in the tth audio packet is more violent, and the possibility of emotional excitement, speed mutation and other situations is larger. Combined with the complexity of data change, the transmission priority of the audio packet originally in time sequence is reconstructed.

[0078] According to the above implementation process, the data importance degree of each audio packet is calculated, and the audio packet transmission sequence is formed according to the priority of the data importance degree , wherein each data importance degree .

[0079] According to the audio packet transmission sequence obtained by the above process, the data transmission is carried out by using the IP network transmission layer, the arrival time of each audio packet is recorded by the receiving end, and the receiving time stamp of the audio packet is added. According to the receiving time stamp, each received audio packet is sorted and integrated into the audio packet receiving sequence of the receiving end .

[0080] Step S2, respectively determine the transmission interval and the receiving interval between the adjacent audio packets under the same sequence number in the audio packet transmission sequence and the audio packet receiving sequence;

[0081] Step S3, determining the transmission sensitivity of the target audio packet by using the transmission interval and the receiving interval of the target audio packet, and determining the abnormal audio packet in the target audio packet by using the transmission sensitivity;

[0082] In the embodiment, the audio packet transmission sequence formed by using the IP network is used for audio packet transmission. When some uncertain factors exist, such as network jam, transmission node congestion, and the like, the transmission paths of different audio packets to the receiving end are obviously different, which causes the out-of-order reception of the audio packets at the receiving end, and further causes the time delay, packet loss, and the like in the audio playing process. The phenomenon is obviously related to the size of the jitter buffer. In order to eliminate the jitter and reduce the time delay length and the number of packet losses as much as possible, the embodiment combines the difference degree of the audio files between the receiving end and the transmission end, constructs an adaptive adjustment parameter, and dynamically adjusts the size of the jitter buffer.

[0083] Specifically, referring to Figure 3 , the step S3, determining the transmission sensitivity of the target audio packet by using the transmission interval and the receiving interval of the target audio packet, includes:

[0084] Step S31, taking the difference between the transmission interval and the receiving interval of the target audio packet as the transmission time difference of the target audio packet.

[0085] Step S32, determining the transmission sensitivity of the target audio packet by using the transmission time difference of the target audio packet and the average of the transmission time differences of all the audio packets.

[0086] The step S32 specifically includes:

[0087] obtaining the content correlation between the target audio packets in the same sequence number in the audio packet transmission sequence and the audio packet receiving sequence by using the Pearson correlation coefficient;

[0088] calculating the transmission sensitivity of the target audio packet by using the transmission time difference of the target audio packet, the average of the transmission time differences of all the audio packets, and the content correlation.

[0089] In the embodiment, by comparing the difference between the audio packet receiving sequence and the audio packet transmission sequence, it is checked whether there is missing audio data or disordered situation, the stall and transmission path change usually occur in the performance bottleneck, network delay and other areas of the IP network, the difference between the audio packet receiving sequence and the audio packet transmission sequence is analyzed, and the audio packet arrival time change sensitivity is calculated by using the difference between the receiving time difference and the sending time difference. When the receiving interval of the tth audio packet is obviously larger than other receiving intervals in the sequence, and the correlation between the audio packet and the corresponding position audio packet in the transmission sequence is smaller, it is indicated that there may be transmission disorder or missing phenomenon in the transmission process of the audio packet, and the transmission sensitivity of the audio packet needs to be distinguished:

[0090] The audio packet transmission sequence is obtained by using the Pearson correlation coefficient and the audio packet receiving sequence , wherein the content correlation between the audio packets with the same sequence number in the audio packet transmission sequence and the audio packet receiving sequence ; ;

[0091] The interval between the tth audio packet and the previous audio packet receiving time stamp at the receiving end is recorded as the receiving interval of the tth audio packet; similarly, the transmission interval of the tth audio packet at the corresponding sending end is obtained; the difference between the receiving interval of the tth audio packet in the audio packet receiving sequence and the transmission interval of the tth audio packet in the audio packet transmission sequence is taken as the transmission time difference of the tth audio packet ;

[0092] The tth audio packet is analyzed, and the transmission sensitivity of the audio packet t is calculated :

[0093] ;

[0094] , wherein represents the average value of all audio packet transmission time differences; represents the transmission time difference of the tth audio packet; represents the content correlation between the target audio packets with the same sequence number t in the audio packet transmission sequence and the audio packet receiving sequence.

[0095] The smaller the difference between the transmission time difference of the tth audio packet and the average value of the transmission time difference, the greater the content correlation of the corresponding position audio packet, it is indicated that the corresponding degree between the audio packet receiving sequence and the audio packet transmission sequence is higher, and the transmission sensitivity of the tth audio packet is lower. It should be noted that, in order to ensure that the calculation result is meaningful, when performing fractional operation, the adjusting factor greater than 0 is added to the denominator to prevent the denominator from being 0, the value of the adjusting factor is set to 0.1 by the implementer according to the actual situation.

[0096] The above calculation results are normalized to screen out audio packets with a transmission sensitivity greater than or equal to 0.5 (a preset sensitivity threshold, which can be adjusted specifically) in the audio packets, and determine that such audio packets have an abnormality in the transmission process, i.e., defined as (transmission) abnormal audio packets.

[0097] Step S4, using the transmission sensitivity of the abnormal audio packets in the target node, determines the transmission convergence degree of the transmission topology network where the target node is located.

[0098] Specifically, please refer to Figure 4 , the step S4 includes:

[0099] Step S41, using the transmission sensitivity of the abnormal audio packets in the target node and the number of abnormal audio packets, determines the transmission load abnormality degree of the target node.

[0100] The step S41 specifically includes:

[0101] Determine the total number of all abnormal audio packets in the transmission topology network where the target node is located and the average transmission sensitivity of the abnormal audio packets in the target node.

[0102] Using the total number of all abnormal audio packets, the average transmission sensitivity, and the number of abnormal audio packets, the transmission load abnormality degree of the target node is calculated.

[0103] Step S42, using the transmission load abnormality degree of the target node and the total number of nodes in the transmission topology network where the target node is located, determines the transmission convergence degree of the transmission topology network.

[0104] The step S42 specifically includes:

[0105] Determine the transmission load abnormality degree difference between the target node and the adjacent node along the network transmission direction.

[0106] Using the transmission load abnormality degree of the target node, the total number of nodes in the transmission topology network where the target node is located, and the transmission load abnormality degree difference, the transmission convergence degree of the transmission topology network is calculated.

[0107] In this embodiment, each node i (representing any node, which can be recorded as a target node) through which the abnormal audio packet passes in the transmission process is recorded.

[0108] ​Since the IP network needs to transmit as many audio packets from the sending end to the receiving end as possible without maintaining the audio connection state, a plurality of audio packets are allocated to a plurality of transmission paths for processing, so that the sequence of the audio packets received by the receiving end is obviously disordered. When a large number of transmission abnormal audio packets are repeatedly transmitted through the same path node, it is indicated that the current network convergence degree is high, and the network transmission process is unstable.

[0109] For the transmission abnormal audio packets screened out, the transmission path of the abnormal audio packets in the IP network is drawn into a transmission topology network node diagram. Please refer to Figure 7 , Figure 7 The transmission topology network node diagram is related to the IP network transmission method of the audio data involved in the embodiment of the present application. The number of transmission abnormal audio packets passing through the ith transmission node is counted ;

[0110] In a high-load network environment, the number of abnormal audio packets passing through different nodes is different. The load degree of each node is also different. When the number of abnormal audio packets passing through the ith node is more, the possibility of transmission load abnormality of the node is greater, and the transmission load abnormality degree of the ith node is calculated :

[0111] ;

[0112] Wherein represents the total number of abnormal audio packets screened out, that is, the total number of all abnormal audio packets in the transmission topology network of the target node; represents the number of abnormal audio packets of the ith node; represents the transmission sensitivity of any abnormal audio packet t on the ith node; represents the mean value of the transmission sensitivity of all abnormal audio packets on the ith node.

[0113] In the IP transmission topology network, the more the number of nodes with transmission load abnormality (as shown by the continuous nodes represented by the larger circle in Figure 7 ), the higher the continuity between such abnormal nodes, and multiple audio packets may be forced to pass through the same transmission path, forming congestion. Therefore, the higher the convergence degree of the transmission path in the current IP network, and the more obvious the damage to the transmission sequence, the greater the degree of disorder of the audio packet sequence received by the receiving end. The transmission convergence degree of the transmission topology network is calculated :

[0114] ;

[0115] Wherein, G represents the total number of transmission nodes contained in the transmission topology network, that is, the total number of nodes in the transmission topology network where the target node i is located; represents the transmission load abnormality degree of the i-th node; represents the transmission load abnormality degree of the i+1-th node adjacent to the i-th node connected in the network transmission direction; represents the transmission load abnormality degree difference, representing the continuity of the abnormality degree between the adjacent two nodes, The greater the ratio of is, the higher the abnormality degree between the adjacent two nodes, the higher the convergence degree of the transmission network obtained by traversing and calculating a plurality of transmission nodes in the topology network, and the higher the degree of transmission abnormality that may occur in the audio packet transmission process. The transmission convergence degree obtained by calculation is normalized by the norm function.

[0116] Step S5, determining the target adjustment parameter of the jitter buffer using the transmission convergence degree of the transmission topology network, and correcting the jitter buffer capacity using the target adjustment parameter.

[0117] Specifically, please refer to Figure 5 , the step S5, determining the target adjustment parameter of the jitter buffer using the transmission convergence degree of the transmission topology network, comprises:

[0118] Step S51, respectively determining the abnormal transmission interval and the abnormal reception interval between the abnormal audio packet and its adjacent normal audio packet under the same serial number in the audio packet transmission sequence and the audio packet reception sequence;

[0119] Step S52, calculating the target adjustment parameter of the jitter buffer using the transmission convergence degree of the transmission topology network, the abnormal transmission interval and the abnormal reception interval.

[0120] In this embodiment, in order to reduce the jitter and delay in the audio playback process of the receiving end and ensure the stability of the audio playback of the receiving end, according to the transmission difference between the audio packets of the receiving end and the sending end in the real-time transmission process, combined with the transmission convergence degree of the transmission network, the target adjustment parameter of the jitter buffer :

[0121] ;

[0122] Wherein, represents the abnormal reception interval between the t-th abnormal audio packet and the previous normal audio packet in the audio packet reception sequence of the receiving end; represents the abnormal transmission interval between the t-th abnormal audio packet and the previous normal audio packet in the audio packet transmission sequence of the sending end. represents the total number of abnormal audio packets screened out, as above; norm ( ) represents the normalization processing to the calculation result.

[0123] The ratio of the two values reflects the degree of network jitter in the audio transmission process. When the ratio is large, the corresponding transmission convergence degree is also large, so the capacity of the jitter buffer needs to be increased to accommodate more data packets, avoiding the jitter buffer from being read empty during playback at the receiving end, causing the audio to pause or break. When the jitter degree between the transmission interval and the receiving interval of the key audio packet (i.e., an audio packet with a high importance level, for which a corresponding judgment threshold can be set) is small, and the corresponding transmission convergence degree is small, the size of the jitter buffer needs to be reduced to reduce the latency of audio data and improve the real-time performance of audio playback.

[0124] The step S5 uses the target adjustment parameter to correct the jitter buffer capacity, which specifically includes:

[0125] The target adjustment parameter and the initial jitter buffer capacity are used to calculate the corrected jitter buffer capacity.

[0126] In this embodiment, during the live transmission of audio data, when there are a large number of abnormal audio packets due to network delay, the large number of abnormal audio packets will occupy a large amount of memory in the jitter buffer, and the sudden increase in the utilization rate of the jitter buffer will also increase the risk of audio packet loss, further causing the direct information to be discontinuous. Therefore, the size of the jitter buffer needs to be corrected adaptively according to the size of the jitter buffer utilization rate and the adjustment parameter, so as to receive sufficient audio packets. Thus we have:

[0127] ;

[0128] where K is the initial size of the jitter buffer, i.e., the initial capacity of the jitter buffer; is the target adjustment parameter of the jitter buffer; is the corrected capacity of the jitter buffer. The initial size of the jitter buffer in this embodiment can be set to cache 20 transmission audio packets.

[0129] The receiving end aligns and arranges the received audio data according to the transmission sequence of the audio packets, determines whether there is a packet loss phenomenon in the received audio data, decodes and plays the audio data when there is no packet loss phenomenon, and continuously monitors the network state and jitter condition.

[0130] When the packet loss phenomenon still exists, the influence degree of the packet loss position on the reconstruction of the audio information is judged, the importance of the packet loss audio data is recalculated according to the feedback information, the size of the jitter buffer and the sorting strategy of the data packet are optimized, and the system is ensured to run in the best state at all times, so that a high-quality audio transmission experience is provided.

[0131] The application is based on audio data decoding analysis and audio data transmission, determines the key audio in the transmission process, obtains the audio packet transmission sequence, ensures that the key audio packet is transmitted preferentially, further compares the audio packet receiving sequence of the receiving end with the audio packet transmission sequence of the sending end, analyzes the difference between the audio packet receiving sequence and the audio packet transmission sequence, screens out the abnormal audio data in transmission, further determines the transmission convergence degree of the transmission topology network by analyzing the transmission path of the abnormal audio data, adaptively constructs the adjustment parameter according to the transmission convergence degree, and then dynamically adjusts the jitter buffer capacity size by using the adjustment parameter, and provides more buffering time for other less important audio data packets, reduces the jitter delay phenomenon in the audio packet transmission process, ensures the audio quality while reducing the playing delay, and reduces or avoids the interruption phenomenon of the audio in the transmission process.

[0132] Embodiment two:

[0133] The embodiment of the application also provides an IP network transmission device for audio data. The device can be a data processing device such as a computer or a server, or a combination of multiple devices.

[0134] As shown in Figure 6 , the hardware running environment structure diagram of the IP network transmission device for audio data involved in the embodiment of the application is shown in Figure 6 .

[0135] As shown in Figure 6 , the IP network transmission device for audio data can include a processor 1001 such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display (Display), an input unit such as a control panel, and an optional user interface 1003 can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a WIFI interface). The memory 1005 can be a high-speed RAM memory, or a stable memory (non-volatile memory) such as a magnetic disk memory. The memory 1005 can optionally be a storage device independent of the aforementioned processor 1001. The memory 1005, as a computer storage medium, can include an IP network transmission program for audio data.

[0136] Those skilled in the art can understand that the hardware structure shown in the foregoing embodiments is not a limitation to the device, and the device can include more or less components than the drawings, or combine certain components, or arrange different components. Figure 6 Those skilled in the art can understand that the hardware structure shown in the foregoing embodiments is not a limitation to the device, and the device can include more or less components than the drawings, or combine certain components, or arrange different components.

[0137] With reference to the foregoing Figure 6 , Figure 6 The memory 1005 in the foregoing embodiments can include an operating device, a user interface module, a network communication module, and an IP network transmission program of audio data.

[0138] In the foregoing embodiments, the network communication module is mainly used for connecting a server and can communicate data with the server; and the processor 1001 can call the IP network transmission program of audio data stored in the memory 1005 and execute the steps in the foregoing embodiments. Figure 6 In the foregoing embodiments, the network communication module is mainly used for connecting a server and can communicate data with the server; and the processor 1001 can call the IP network transmission program of audio data stored in the memory 1005 and execute the steps in the foregoing embodiments.

[0139] Based on the hardware structure of the IP network transmission device of audio data, the embodiments of the IP network transmission method of audio data are implemented.

[0140] In addition, the present application also provides a computer readable storage medium. The computer readable storage medium of the present application stores an IP network transmission program of audio data, wherein the IP network transmission program of audio data is executed by a processor to implement the steps of the IP network transmission method of audio data as described above.

[0141] The method implemented when the IP network transmission program of audio data is executed can refer to the embodiments of the IP network transmission method of audio data of the present application, which will not be described here.

[0142] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are also possible or can be advantageous.

[0143] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment mainly describes the differences from other embodiments.

[0144] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, apparatus, or computer program products. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer-readable program code thereon for use by or in connection with an instruction execution system. For the purposes of this description, a "computer-usable or computer-usable storage medium" can be any apparatus that can contain or store the program for use by or in connection with the instruction execution system, apparatus, or device.

[0145] The above description is merely the preferred embodiments of the present application, and is not intended to limit the protection scope of the present application. Any equivalent structure / method transformation made according to the disclosure content of the present application, or direct / indirect application in other related technical fields, shall be included in the protection scope of the present application.

Claims

1. A method for IP network transmission of audio data, characterized in that, The method comprises: determining an audio packet transmission sequence composed of each audio packet sequence by using the information change amount in the audio packet, and determining an audio packet receiving sequence by using the receiving time stamp of the audio packet; determining the transmission interval and the receiving interval between adjacent audio packets at the same sequence number in the audio packet transmission sequence and the audio packet receiving sequence respectively; determining the transmission sensitivity of the target audio packet by using the transmission interval and the receiving interval of the target audio packet, and determining the abnormal audio packet in the target audio packet by using the transmission sensitivity; determining the transmission convergence degree of the transmission topology network in which the target node is located by using the transmission sensitivity of the abnormal audio packet in the target node; determining the target adjustment parameter of the jitter buffer by using the transmission convergence degree of the transmission topology network, and correcting the capacity of the jitter buffer by using the target adjustment parameter.

2. The IP network transmission method of audio data according to claim 1, characterized in that, The method comprises: determining the data importance degree of the audio packet by using the information change amount in the audio packet; obtaining the audio packet transmission sequence by using the priority sequence of the audio packet obtained by using each data importance degree.

3. The IP network transmission method of audio data according to claim 2, wherein, The method comprises: obtaining the intrinsic mode function component of the audio packet by using the empirical mode decomposition of the audio packet, and taking the variance of the intrinsic mode function component as the information change amount in the audio packet; calculating the data importance degree of the audio packet by using the information change amount in the audio packet, and the maximum information change amount and the minimum information change amount in all audio packets.

4. The IP network transmission method of audio data according to claim 1, wherein, The method comprises: taking the difference between the transmission interval and the receiving interval of the target audio packet as the transmission time difference of the target audio packet; determining the transmission sensitivity of the target audio packet by using the transmission time difference of the target audio packet and the average value of the transmission time difference of all audio packets.

5. The IP network transmission method of audio data according to claim 4, characterized in that, The method comprises: obtaining the content correlation between the target audio packets at the same sequence number in the audio packet transmission sequence and the audio packet receiving sequence by using the Pearson correlation coefficient; calculating the transmission sensitivity of the target audio packet by using the transmission time difference of the target audio packet, the average value of the transmission time difference of all audio packets, and the content correlation.

6. The IP network transmission method of audio data according to claim 1, wherein, The method comprises: determining the transmission load abnormal degree of the target node by using the transmission sensitivity of the abnormal audio packet in the target node and the number of abnormal audio packets; determining the transmission convergence degree of the transmission topology network by using the transmission load abnormal degree of the target node and the total number of nodes in the transmission topology network in which the target node is located.

7. The IP network transmission method of audio data according to claim 6, characterized in that, The method comprises: determining the average value of the transmission sensitivity of the abnormal audio packet in the target node and the total number of all abnormal audio packets in the transmission topology network in which the target node is located; The transmission load abnormality degree of the target node is calculated by using the total number of the abnormal audio packets, the average of the transmission sensitivity and the number of the abnormal audio packets.

8. The IP network transmission method of audio data according to claim 6, wherein, The transmission convergence degree of the transmission topology network is determined by using the transmission load abnormality degree of the target node and the total number of nodes in the transmission topology network where the target node is located, and the transmission convergence degree of the transmission topology network is determined by using the transmission load abnormality degree of the target node and the total number of nodes in the transmission topology network where the target node is located, and the transmission convergence degree of the transmission topology network is determined by using the transmission load abnormality degree of the target node and the total number of nodes in the transmission topology network where the target node is located. The difference of the transmission load abnormality degree between the target node and the adjacent node in the network transmission direction is determined. The transmission convergence degree of the transmission topology network is calculated by using the transmission load abnormality degree of the target node, the total number of nodes in the transmission topology network where the target node is located and the difference of the transmission load abnormality degree.

9. The IP network transmission method of audio data according to claim 1, wherein, The target adjustment parameter of the jitter buffer is determined by using the transmission convergence degree of the transmission topology network, and the target adjustment parameter of the jitter buffer is determined by using the transmission convergence degree of the transmission topology network, and the target adjustment parameter of the jitter buffer is determined by using the transmission convergence degree of the transmission topology network. The abnormal transmission interval and the abnormal receiving interval between the abnormal audio packet and the adjacent normal audio packet under the same serial number in the audio packet transmission sequence and the audio packet receiving sequence are respectively determined. The target adjustment parameter of the jitter buffer is calculated by using the transmission convergence degree of the transmission topology network, the abnormal transmission interval and the abnormal receiving interval.

10. The IP network transmission method of audio data according to claim 1, wherein, The capacity of the jitter buffer is corrected by using the target adjustment parameter, and the corrected capacity of the jitter buffer is calculated by using the target adjustment parameter and the initial capacity of the jitter buffer. ​

Images