Audio transmission control method and system

Abstract

The invention relates to an audio transmission control method and system, and relates to the field of audio transmission control, and the method comprises the steps: continuously monitoring the information of a received audio data packet, the information comprising a serial number or a received signal strength indication value; analyzing the received audio data packet, extracting voice feature information in an audio frame and caching the voice feature information; according to the serial number or the received signal strength indication value, determining whether there is an audio data packet loss condition, and if yes, suppressing a retransmission request for the lost audio data packet; if not, continuously monitoring the information of the received audio data packet; under the condition that the audio data packets are lost, reading the voice feature information cached before the audio data packets are lost, and generating filling audios with matched duration in combination with the quantity of the lost audio data packets; and putting the filling audio into the position where the audio data packet is lost to form a continuous audio stream and outputting the continuous audio stream. According to the invention, the problem that the audio quality is reduced due to an improper retransmission mechanism in the prior art is solved.

Description

Technical Field

[0001] This application relates to the field of audio transmission control, and more specifically, to an audio transmission control method and system. Background Technology

[0002] In modern communication technology, stable and clear transmission of audio data streams is crucial for many real-time applications, especially in professional internal communication systems. Traditional audio transmission control methods typically employ sequence number-based packet retransmission mechanisms to restore the integrity of the audio stream when data packets are lost. However, in certain complex wireless communication environments, such as the interiors of large event venues, the presence of numerous reinforced concrete load-bearing walls, metal support structures, and glass curtain walls, along with the constant movement of large crowds, creates complex multipath effects on radio signal propagation, leading to a sharp decline in signal strength. Similarly, in family gatherings, microwave ovens emit electromagnetic waves while heating food, young people play games, elders video chat with distant relatives, and children watch cartoons on tablets—everyone is connected to the same communication signal, and every device is sending and receiving data, making the wireless channel extremely congested and drastically increasing the probability of data packet collisions and loss. Although this signal fading process is usually short-lived, its duration is often sufficient to cover the transmission time of several consecutive audio data packets, resulting in the simultaneous loss of multiple consecutive data packets, forming a "clustered loss" phenomenon.

[0003] Faced with this clustered packet loss phenomenon, the receiver of the audio communication system responds according to its built-in control method based on its preset logic. Since this logic is designed to handle single or random packet loss, when it detects a "jump" in the sequence number of a data packet (e.g., sequence numbers 3, 4, 5, and 6 are all lost), it considers this to be four independent packet loss events. Therefore, the receiver generates and sends an independent retransmission request for each packet deemed lost. This results in multiple retransmission requests being sent back to the sender in a very short period. Upon receiving this series of dense retransmission requests, the sender immediately attempts to retransmit all the requested retransmission packets at once. This coincides with intense competition for limited channel resources from the new real-time audio data packets being transmitted, potentially causing some new real-time data packets to be dropped due to this "internal congestion," resulting in secondary packet loss. Ultimately, the user at the receiver receives poor-quality "repaired" audio.

[0004] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0005] This application discloses an audio transmission control method and system, which aims to solve the problem that traditional packet loss retransmission mechanisms may lead to secondary packet loss and audio quality degradation in complex wireless communication environments.

[0006] The technical solution of this application is as follows: In a first aspect, this application discloses an audio transmission control method, comprising the following steps: Continuously monitor information about received audio data packets, including sequence numbers or received signal strength indicators; Analyze the received audio data packets, extract the speech feature information in the audio frames and cache it; determine whether there is any audio data packet loss based on the sequence number or received signal strength indicator value; if so, suppress retransmission requests for lost audio data packets; if not, continue to monitor the information of received audio data packets. In the event of audio data packet loss, read the voice feature information cached before the audio data packet loss, and combine it with the number of audio data packets lost to generate duration-matched filler audio; Fill in the missing audio data packets to create a continuous audio stream and output it.

[0007] This technical solution effectively addresses the problem of audio data packet loss in complex wireless environments. By suppressing retransmission requests and generating filler audio, it avoids secondary packet loss and channel congestion, significantly improving the continuity and intelligibility of audio transmission, and solving the problem of audio quality degradation caused by improper retransmission mechanisms in existing technologies.

[0008] Furthermore, in one embodiment, the information of the audio data packet is a sequence number, and determining whether an audio data packet is lost based on the sequence number includes the following steps: Cases of discontinuous sequence numbers are treated as lost audio data packets and counted accordingly. When the number of lost audio data packets exceeds a preset threshold, it is determined that there is a cluster of lost audio data packets.

[0009] This technical solution can accurately identify clustered loss of audio data packets, providing accurate judgment criteria for subsequent retransmission suppression and filler audio generation, avoiding misjudgment of sporadic packet loss, and improving the robustness of the system.

[0010] More specifically, in some implementations, the steps of treating discontinuous sequence numbers as lost audio data packets and counting them include: After receiving an audio data packet with sequence number N, if the sequence number of the next audio data packet is ≥ N + K, the audio data packet is considered lost. N is a natural number greater than or equal to 1, and K is a natural number greater than 1. An internal continuous packet loss counting mechanism is activated to record the number of consecutively lost audio data packets; the internal continuous packet loss counting mechanism includes: The number of lost audio data packets is incremented each time a loss is detected. If consecutive audio data packets with consecutive sequence numbers are received before the number of lost packets reaches a preset threshold, the audio data packets are considered to have been lost in a non-clustered manner, and the accumulated number of lost packets is reset to zero; or If the sequence number of the next audio data packet is N+M and M is greater than the preset threshold before the number of lost data packets reaches the preset threshold, it is considered a new audio data packet loss situation, and the accumulated number of lost data packets is reset to zero and recounted.

[0011] This technical solution enables more precise identification of clustered packet loss, avoiding overreaction to brief signal interruptions and irrelevant packet loss, resulting in more accurate packet loss judgment and thus optimizing the triggering timing of retransmission suppression and filler audio.

[0012] Based on the above, this application further proposes that, in the audio data packet transmission control method, the speech feature information includes pitch feature information, timbre feature information, and an energy envelope representing the instantaneous loudness change of the speech signal. The step of extracting the pitch feature information includes: Calculate the autocorrelation function of the audio frame; Find the maximum peak value of the autocorrelation function at a non-zero delay; The speech period is obtained from the maximum peak value, and pitch feature information is obtained from the speech period.

[0013] This technical solution can accurately extract the pitch features of speech, providing key speech parameters for the subsequent generation of natural and fluent fill audio, and ensuring the pitch consistency between the fill audio and the original speech.

[0014] Preferably, in the audio transmission control method, the step of suppressing retransmission requests for lost audio data packets includes: Set a control flag in the communication protocol stack to prevent audio data, including the receiver, from generating and sending independent retransmission request messages.

[0015] This technical solution can fundamentally avoid retransmission storms caused by clustered packet loss, effectively prevent channel congestion and secondary packet loss, and thus significantly improve the stability of audio transmission in harsh wireless environments.

[0016] In some preferred embodiments, the step of extracting and caching speech feature information from audio frames includes: Extract pitch feature information from the current audio frame; Determine whether the timbre features in the current audio frame are distorted; if so, extract clean timbre feature information from the clean timbre buffer, which pre-stores timbre feature information corresponding to audio frames that have not been contaminated by transient electromagnetic interference within a certain period in the past; if not, extract timbre feature information from the current audio frame. Pitch and timbre features are cached.

[0017] This technical solution ensures the purity and stability of timbre features when generating fill audio. Even if the current audio frame is contaminated, it can utilize cached clean timbre information to generate fill audio with better sound quality and a more natural sound.

[0018] Based on this, the steps for generating filler audio include: Based on the extracted pitch feature information, a periodic excitation signal is generated; Phonic feature information is used as coefficients of a digital filter to construct a speech synthesis filter; The excitation signal is passed through a speech synthesis filter to synthesize a fill audio that retains the original timbre and continues the pitch trend before the loss.

[0019] This technical solution can synthesize filler audio that closely matches the original speech based on the extracted speech feature information, enabling the audio stream to transition smoothly after packet loss, greatly improving the continuity and comfort of the user's auditory experience.

[0020] Based on the above, this application further proposes that the step of storing clean tone feature information in the clean tone buffer includes: Perform spectral flatness analysis on the received audio frames to obtain spectral flatness values; The signal-to-noise ratio (SNR) is estimated by performing an SNR calculation on the audio frames. Monitor the instantaneous zero-crossing rate of audio frames; Based on the spectral flatness value, signal-to-noise ratio, and instantaneous zero-crossing rate, determine whether the audio frame is contaminated by noise; When the audio frame is not contaminated by noise, extract the timbre features of the audio frame and store the timbre features in a clean timbre buffer.

[0021] This technical solution can intelligently identify and cache high-quality clean timbre features, ensuring the timbre purity of the fill audio, avoiding the use of noise-contaminated timbre in synthesis, and further improving the quality of the fill audio.

[0022] As an optional solution, in the audio transmission control method, the information of the audio data packet is the received signal strength indication value. The steps for determining whether there is audio data packet loss based on the received signal strength indication value include: Preset signal jitter warning threshold; Calculate the standard deviation of the most recent Y received signal strength indicators; if the standard deviation exceeds the signal jitter warning threshold, it is determined that there is audio data packet fragmentation and loss. This technical solution can predict and judge potential packet loss by monitoring signal strength jitter, and is particularly suitable for packet loss caused by signal fading, providing the system with another dimension for packet loss judgment and enhancing the system's adaptability.

[0023] Secondly, this application also discloses an audio transmission control system, which applies any of the above-mentioned audio transmission control methods, including: The information monitoring module is used to continuously monitor the information of the received audio data packets, including the sequence number or the received signal strength indicator value. The speech feature extraction module is used to analyze the received audio data packets, extract speech feature information from the audio frames, and cache it. The packet loss detection module is used to determine whether there is any audio data packet loss based on the sequence number or the received signal strength indicator value. The retransmission suppression module is used to suppress retransmission requests for lost audio data packets; The filler audio generation module is used to read the voice feature information cached before the audio data packets were lost, and generate filler audio with matching duration based on the number of audio data packets lost. The audio stream integration module is used to insert filler audio at the locations where audio data packets are missing, forming a continuous audio stream for output.

[0024] This application provides a system for implementing the above-mentioned audio transmission control method through this technical solution. The modular design enables the functions to work together, which can efficiently realize intelligent control of audio transmission, effectively solve the audio transmission quality problem in complex wireless environments, and improve the overall performance and reliability of the system.

[0025] Compared with existing technologies, this invention has significant advantages and beneficial effects. Specifically, as can be seen from the above technical solution, the audio transmission control method and system disclosed in this application continuously monitors the received audio data packet information and determines whether audio data packets are lost based on the sequence number or received signal strength indicator value. If lost, it suppresses retransmission requests for the lost audio data packets and simultaneously generates duration-matched filler audio using cached voice feature information, places it at the lost position, forms a continuous audio stream, and outputs it. This method effectively solves the problem in existing technologies where traditional retransmission-based packet loss recovery mechanisms may lead to secondary packet loss and audio quality degradation in complex wireless communication environments. Specifically, when clustered packet loss occurs, this application avoids competition between retransmitted data streams and new real-time data streams on limited channel resources by suppressing retransmission requests, thereby preventing channel congestion and secondary packet loss. Meanwhile, by intelligently generating filler audio that matches the original speech features, this application can smoothly compensate for lost audio segments, so that the audio stream heard by the user maintains a high degree of continuity and intelligibility, significantly improving the user experience and overcoming the problems of poor audio quality and low intelligibility after "repair" in the prior art.

[0026] To more clearly illustrate the structural features, technical means, and specific objectives and functions achieved by the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Attached Figure Description

[0027] Figure 1 This application provides a flowchart illustrating an audio transmission control method.

[0028] Figure 2 This is a schematic diagram of the framework of an audio transmission control system provided in this application. Detailed Implementation

[0029] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0030] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] In complex wireless communication environments, such as inside large event venues, traditional audio transmission control methods often trigger a series of negative effects when dealing with clustered audio data packet loss. Specifically, when signal fading leads to the loss of consecutive audio data packets, the receiver sends a retransmission request for each lost packet. This not only exacerbates channel congestion but may also cause secondary packet loss, ultimately resulting in users experiencing intermittent, stuttering, and extremely intelligible audio. The quality of this "repaired" audio may even be worse than completely abandoning retransmission and simply accepting brief silence.

[0032] Please refer to the following for details. Figure 1 This application proposes an audio transmission control method, including the following steps: S100: Continuously monitors information of received audio data packets, including sequence number or received signal strength indication value; S200: Analyze the received audio data packets, extract speech feature information from the audio frames, and cache it; S300: Determine whether there is audio data packet loss based on the serial number or received signal strength indication value; if yes, proceed to step S400; if no, proceed to step S100. S400: Suppress retransmission requests for lost audio data packets; S500: In the event of audio data packet loss, read the voice feature information cached before the audio data packet loss, and combine it with the number of audio data packets lost to generate duration-matched filler audio. S600: The filling audio is placed at the location where the audio data packet is lost, forming a continuous audio stream and outputting it.

[0033] This application aims to effectively avoid the drawbacks of traditional retransmission mechanisms in specific scenarios through an intelligent packet loss handling mechanism, thereby significantly improving the stability of audio transmission and user experience.

[0034] To better understand the audio transmission control method proposed in this application, the key terms and implementation environment involved will be explained in detail below.

[0035] An "audio data packet" refers to a discrete data unit segmented from a continuous audio signal during digital audio transmission. Each data packet typically contains a portion of audio data and related control information, such as a sequence number and timestamp. These data packets are transmitted over the network and reassembled at the receiving end to reconstruct the original audio stream. The "sequence number" is a unique identifier for each audio data packet, indicating its order within the entire audio stream. Monitoring the continuity of sequence numbers helps determine if data packets are lost or out of order. The "Received Signal Strength Indicator (RSSI)" measures the strength of a wireless signal, usually represented by a negative value; the closer the value is to zero, the stronger the signal. Changes in RSSI can reflect the quality of the wireless channel, such as signal fading or interference. An "audio frame" is the actual audio data unit contained within an audio data packet, typically corresponding to a short segment of continuous speech or sound. The audio frame is the basic unit for speech feature extraction. "Speech feature information" refers to parameters extracted from audio frames that characterize speech properties, such as pitch, timbre, and energy envelope. This feature information is crucial for speech synthesis and the generation of filler audio.

[0036] The audio transmission control method of this application can be implemented in various wireless communication environments, such as Wi-Fi networks, Bluetooth communication, and cellular networks. It is particularly suitable for scenarios with high real-time audio requirements and complex and variable wireless channel environments, such as professional internal communication systems, real-time voice conferencing systems, and wireless intercom systems. In these scenarios, audio data packet loss is a common problem due to factors such as signal fading, interference, or mobility, and the method of this application can effectively address these challenges.

[0037] The audio transmission control method proposed in this application achieves intelligent handling of audio data packet loss through a series of collaborative steps, thereby improving the quality of audio transmission and user experience.

[0038] First, the method continuously monitors information from received audio data packets. This information may include a sequence number or a received signal strength indicator (RSI). For example, the header information of each incoming audio data packet can be obtained in real time through the driver of a network interface card (NIC) or wireless module, from which the sequence number or RSI can be parsed. Alternatively, the received data packets can be encapsulated at the application layer, and this information can be added or extracted during the encapsulation process.

[0039] Secondly, this method analyzes the received audio data packets, extracts speech feature information from the audio frames, and caches it. Specifically, each audio data packet can be decapsulated to obtain the audio frame data. Then, digital signal processing techniques are used to analyze the audio frames and extract their speech feature information. For example, algorithms such as Linear Predictive Coding (LPC) analysis, Mel-frequency cepstral coefficient (MFCC) extraction, and fundamental frequency (F0) detection can be used to obtain features such as pitch, timbre, and energy envelope. These extracted speech feature information are stored in a buffer for later retrieval when needed. The buffer can be implemented using data structures such as a circular buffer or a first-in-first-out (FIFO) queue to ensure that the stored speech feature information is from the most recent period.

[0040] Next, based on the sequence number or received signal strength indicator value, it is determined whether audio data packets are lost. If loss is detected, retransmission requests for the lost audio data packets are suppressed; otherwise, the information of received audio data packets continues to be monitored. For example, when the information is a sequence number, it can be determined by checking whether the sequence numbers of the received data packets are consecutive. If the sequence numbers jump, for example, after receiving a data packet with sequence number N, the next received data packet with sequence number N+2, it can be determined that the data packet with sequence number N+1 is lost. When the information is a received signal strength indicator value, a signal jitter warning threshold can be preset, and the standard deviation of the most recent M received signal strength indicator values ​​can be calculated. If the standard deviation exceeds the preset threshold, it can be determined that there is audio data packet fragmentation loss. After determining that audio data packets are lost, retransmission requests can be suppressed in various ways. For example, a control flag can be set in the communication protocol stack, which is used to prevent the audio data packet receiver from generating and sending independent retransmission request messages. Another approach is to intercept retransmission requests at the application layer and decide whether to send them based on a preset strategy.

[0041] In the event of audio data packet loss, this method reads the cached speech feature information from before the audio data packets were lost and, based on the number of lost audio data packets, generates filler audio with a matching duration. For example, if three audio data packets are lost, each corresponding to 20 milliseconds of audio, then 60 milliseconds of filler audio needs to be generated. The generation of filler audio can be based on cached speech feature information, such as pitch, timbre, and energy envelope. Speech synthesis techniques, such as waveform concatenation, parametric synthesis, or neural network synthesis, can be used to generate natural-sounding filler audio based on this feature information. For example, a periodic excitation signal can be generated based on the extracted pitch feature information, and the timbre feature information can be used as coefficients of a digital filter to construct a speech synthesis filter. The excitation signal is then passed through the speech synthesis filter to synthesize filler audio that retains the original timbre and continues the pitch trend before the data loss.

[0042] Finally, the method inserts the filler audio at the location where the audio data packet was lost, forming a continuous audio stream for output. For example, an audio stream integration module can be set up before the audio decoder or player. When a data packet loss is detected and filler audio is generated, this module inserts the filler audio into the original audio stream at the location corresponding to the lost data packet. In this way, even with data packet loss, the output audio stream maintains continuity, avoiding silences or stutters caused by data packet loss.

[0043] The audio transmission control method proposed in this application effectively solves the problem caused by the clustering or fragmentation loss of audio data packets in complex wireless communication environments through innovative improvements to the traditional retransmission mechanism. The core innovation of this application lies in the fact that when audio data packet loss is detected, especially when it is clustered, it no longer blindly sends retransmission requests, but instead intelligently generates duration-matched filler audio by suppressing retransmission requests and combining pre-cached voice feature information.

[0044] This application extracts and caches speech feature information from audio frames in real time upon receiving audio data packets. These features, such as pitch, timbre, and energy envelope, are crucial for synthesizing high-quality filler audio. When packet loss occurs, the system can utilize this cached speech feature information, combined with the number of lost data packets, to generate filler audio that matches the style and duration of the original audio. This filler audio can be smoothly integrated into the position of the lost data packets, maintaining the continuity of the output audio stream and greatly improving the user experience.

[0045] Compared with existing technologies, this application has the following advantages: First, it avoids the vicious cycle of "the more you retransmit, the worse it gets" caused by retransmission mechanisms under poor channel conditions, effectively reducing channel load. Second, through intelligent audio padding, it can provide a continuous and natural-sounding audio stream even in the event of data packet loss, significantly improving audio intelligibility and user satisfaction. This method ensures real-time audio transmission while also maintaining audio quality, providing a more reliable solution for demanding applications such as professional internal communications.

[0046] This application further proposes an audio transmission control method, which specifically limits the information of audio data packets to sequence numbers and accurately determines the loss of audio data packets based on the continuity of sequence numbers. In particular, it can identify and count the clustered loss of audio data packets, thereby providing a more refined basis for subsequent retransmission suppression and filler audio generation.

[0047] Specifically, in the above audio transmission control method, the information of the audio data packets is set as a sequence number. The step of determining whether there is audio data packet loss based on the sequence number includes: treating discontinuous sequence numbers as audio data packet loss and counting them; when the number of lost audio data packets is greater than a preset threshold, it is determined that there is a cluster loss of audio data packets.

[0048] The information in the audio data packets is limited to sequence numbers, meaning the system primarily relies on the sequential identification of data packets to detect loss. A discontinuous sequence number means that the received audio data packets are not strictly arranged in ascending order. This discontinuity is considered an indication of audio data packet loss, and the number of lost packets is counted. For example, if audio data packets with sequence numbers 1, 2, and 3 are received, and then audio data packet with sequence number 6 is received directly, then audio data packets with sequence numbers 4 and 5 are considered lost, with a loss count of 2.

[0049] Furthermore, to distinguish between sporadic, isolated packet loss and more severe, continuous clustered packet loss, this application introduces a "preset threshold." When the number of lost audio data packets exceeds this preset threshold, the system determines that there is currently a clustered loss of audio data packets. This preset threshold can be flexibly configured according to the actual application scenario, the stability of the network environment, and the requirements for audio quality. For example, in voice call scenarios with high real-time requirements, this threshold can be set relatively small to quickly respond to and handle continuous packet loss; while in music streaming scenarios with higher audio quality requirements, this threshold can be set relatively large to avoid excessively triggering the determination of clustered packet loss.

[0050] This application's solution specifically defines audio data packet information as sequence numbers and introduces counting of discontinuous sequence numbers and judgment of preset thresholds, thereby enabling more accurate identification and differentiation of different types of audio data packet loss. Specifically, when the number of lost audio data packets does not reach the preset threshold, the system can treat it as sporadic, isolated packet loss, and may adopt a relatively mild padding strategy or not trigger retransmission suppression. However, once the number of lost audio data packets exceeds the preset threshold, the system can accurately identify it as a severe clustered loss. This differentiation allows the system to take more precise and effective follow-up processing measures based on the severity and type of packet loss. For example, for clustered loss, retransmission requests can be more actively suppressed to avoid exacerbating network congestion; simultaneously, when generating padding audio, more complex algorithms can be used based on the characteristics of clustered loss to ensure a smooth transition of the audio stream, thereby effectively improving the robustness of audio transmission and user experience. Through the above technical solution, this application can achieve refined identification of audio data packet loss, especially effectively distinguishing between sporadic packet loss and clustered packet loss. This refined identification provides a more accurate basis for subsequent retransmission suppression and filler audio generation, avoiding a one-size-fits-all approach to all packet loss scenarios. As a result, the system can more decisively suppress retransmissions when packets are lost in clusters, thereby reducing network load and improving transmission efficiency. Simultaneously, by identifying clustered packet loss, the system can guide the filler audio generation strategy, ensuring that when faced with a large number of consecutive packet losses, the generated filler audio can better connect with the preceding and following audio, minimizing auditory discomfort and significantly improving the continuity of audio transmission and user experience.

[0051] In some preferred embodiments, a specific example is given below. Assume the preset threshold is set to 3. After the system receives audio data packets with sequence numbers 1, 2, and 3, the next received audio data packet has a sequence number of 5. At this time, audio data packet with sequence number 4 is considered lost, with a loss count of 1. Since 1 is less than the preset threshold 3, the system determines that the current situation is a non-clustered loss. As another example, after the system receives audio data packets with sequence numbers 10, 11, and 12, the next received audio data packet has a sequence number of 16. At this time, audio data packets with sequence numbers 13, 14, and 15 are considered lost, with a loss count of 3. Since 3 equals the preset threshold 3, the system determines that there is a clustered loss of audio data packets. In this way, the system can accurately identify whether the actual number of lost packets is sporadic or more severe clustered packet loss by comparing it with the preset threshold, thus providing crucial information for subsequent decision-making.

[0052] Traditional audio transmission control methods, while able to identify missing audio data packets based on sequence number discontinuity, have limitations in accurately distinguishing between sporadic, non-clustered single packet losses and persistent, clustered packet losses, as well as in effectively managing packet loss counts to avoid false positives. Failure to meticulously identify and handle different types of packet loss may lead to the system suppressing retransmission requests unnecessarily or failing to respond promptly when suppression is truly necessary, thus impacting audio transmission stability and user experience.

[0053] In response, this application further proposes to optimize the above-mentioned step of treating the discontinuous sequence number as audio data packet loss and counting it, by introducing a more refined packet loss counting mechanism to more accurately identify and handle different types of audio data packet loss.

[0054] Specifically, the steps for treating non-contiguous sequence numbers as audio data packet loss and counting them include: after receiving an audio data packet with sequence number N, if the sequence number of the next audio data packet is ≥ N+K, it is considered an audio data packet loss, where N is a natural number greater than or equal to 1 and K is a natural number greater than 1; an internal continuous packet loss counting mechanism is activated to record the number of consecutively lost audio data packets; the internal continuous packet loss counting mechanism includes: accumulating the number of lost audio data packets each time a loss is detected; if an audio data packet with consecutive sequence numbers is received before the number of lost packets reaches a preset threshold, it is considered that the audio data packets are lost in a non-clustered manner, and the accumulated number of lost packets is cleared to zero; or if, before the number of lost packets reaches the preset threshold, the sequence number of the next audio data packet is N+M, and M is greater than the preset threshold, it is considered a new audio data packet loss, and the accumulated number of lost packets is cleared to zero and recounted.

[0055] Here, sequence number N represents the sequence number of the currently received audio data packet, while K is an integer greater than 1, used to define the sequence number jump interval for determining data packet loss. A data packet is considered lost when the sequence number of the next received data packet is greater than or equal to K from the sequence number N of the current data packet. For example, if K is set to 2, after receiving a data packet with sequence number 100, if the next received data packet has a sequence number of 102 or greater, then the data packet with sequence number 101 is considered lost. This mechanism aims to identify obvious sequence number discontinuities as a preliminary basis for determining data packet loss.

[0056] Furthermore, to distinguish between single, occasional packet loss and continuous, clustered packet loss, this application introduces an internal continuous packet loss counting mechanism. This mechanism records the number of audio data packets lost. Whenever a discontinuous sequence number is detected, it is considered that packet loss has occurred, and the number of packets lost in that instance is counted.

[0057] Building upon this, the counting mechanism includes two reset or recount logics: First, if the system receives consecutive audio data packets with sequential numbers before the accumulated packet loss reaches a preset threshold, this indicates that the previous packet loss was sporadic and non-continuous, not constituting a large-scale clustered loss. In this case, to avoid unnecessary retransmission suppression of non-clustered losses, the accumulated packet loss count is reset to zero. Second, if the sequence number of the next received audio data packet is N+M, where M is a natural number greater than 1, before the accumulated packet loss reaches the preset threshold, this indicates a very large sequence number jump after the current consecutive packet loss sequence. This could signify the start of a new, independent clustered packet loss event, or a very serious network outage. In this case, to accurately identify the new packet loss pattern, the accumulated packet loss count is reset to zero and the counting restarts.

[0058] This dynamic counting and resetting logic enables the system to effectively distinguish between sporadic packet loss and persistent clustered packet loss, providing a more accurate basis for subsequent retransmission suppression decisions. It avoids overreacting to non-clustered losses, thereby reducing unnecessary network resource consumption and potential latency. Simultaneously, for genuine clustered losses, it can suppress them promptly and effectively, and ensure audio stream continuity by padding audio, significantly improving the user's audio experience in environments with fluctuating network conditions.

[0059] In some preferred embodiments, a specific example is given below. Assume the sequence number jump threshold K is 2, and the threshold for determining the number of clustered data loss is 3. For example, when the audio data packets received by the system have the following sequence numbers: 100, 101, 103, 104, 106, 107. First, data packets with sequence numbers 100 and 101 are received, and the sequence numbers are consecutive. Next, data packet with sequence number 103 is received. Compared to the previous data packet 101, the sequence number jump is 103-101=2, satisfying the condition 103≥101+K (2), therefore, data packet with sequence number 102 is determined to be lost. At this time, the internal consecutive packet loss counting mechanism is activated, recording the number of losses as 1. Subsequently, data packet with sequence number 104 is received. Compared to the previous data packet 103, the sequence number is consecutive. Since the currently accumulated number of losses (1) has not reached the preset threshold (3), and data packets with consecutive sequence numbers are received, the accumulated number of losses is cleared to zero. This indicates that the loss of sequence number 102 is considered a non-clustered loss.

[0060] For example, when the audio data packets received by the system have the sequence numbers 200, 201, 205, 206, 207, and 208 respectively. First, data packets with sequence numbers 200 and 201 are received, and the sequence numbers are consecutive. Next, data packet with sequence number 205 is received. Since 205 ≥ 201 + K (2), it is determined that audio data packets with sequence numbers 202, 203, and 204 are lost. The internal consecutive packet loss counting mechanism is activated, and the cumulative number of lost packets is 3. At this time, the number of lost packets (3) reaches the preset threshold (3), and the system determines that there is a clustered loss of audio data packets. Finally, data packet with sequence number 206 is received. Compared with the previous data packet 205, the sequence number is consecutive. Since it has been determined to be a clustered loss, subsequent processing will be based on this judgment.

[0061] Through the above examples, it can be clearly seen how the internal continuous packet loss counting mechanism of this application dynamically judges and records packet loss events based on the continuity and jump of the sequence number, and accurately distinguishes between non-clustered loss and clustered loss according to the preset threshold and clearing conditions, thereby providing a more accurate decision basis for subsequent audio transmission control.

[0062] Specifically, in the above-mentioned audio transmission control method, the step of extracting speech feature information can be further refined. The speech feature information includes pitch feature information, timbre feature information, and an energy envelope representing the instantaneous loudness change of the speech signal. The step of extracting pitch feature information includes: Calculate the autocorrelation function of the audio frame; Find the maximum peak value of the autocorrelation function at a non-zero delay; The speech period is obtained based on the maximum peak value, and the pitch feature information is obtained based on the speech period.

[0063] Among them, pitch feature information mainly reflects the fundamental frequency variation of speech, which is closely related to the intonation and emotion of speech; timbre feature information describes the spectral characteristics of speech, determining the unique timbre of the vocal body; and energy envelope reflects the instantaneous loudness variation of the speech signal, which is an important manifestation of speech rhythm and stress. The extraction steps of the pitch feature information are as follows: First, calculate the autocorrelation function of the audio frame. The autocorrelation function is a signal processing tool used to measure the similarity of a signal to itself at different delays; for periodic signals, it will show a peak at the periodic delay. Second, find the maximum peak of the autocorrelation function at a non-zero delay. The delay time corresponding to this maximum peak is usually related to the fundamental frequency period of the speech. Third, obtain the speech period based on the maximum peak. The speech period, or fundamental frequency period, is the time required for a complete cycle of vocal cord vibration in the speech signal. Finally, obtain the pitch feature information based on the speech period. Pitch is usually a perceptual attribute of the fundamental frequency and can be calculated by the reciprocal of the fundamental frequency period.

[0064] The proposed solution provides high-quality input for subsequent filler audio generation by meticulously defining and extracting speech feature information, particularly the precise acquisition of pitch feature information. By calculating the autocorrelation function of audio frames and finding the maximum peak at non-zero delay, the periodic structure of the speech can be accurately identified, thus obtaining the speech period. Based on this speech period, pitch feature information can be precisely determined. This method ensures that the extracted pitch feature information accurately reflects the fundamental frequency variation trend of the original speech, laying the foundation for subsequent synthesis of filler audio with natural pitch variations. Simultaneously, by combining timbre feature information and energy envelope, the generated filler audio not only maintains continuity with the original audio in pitch but also highly matches the variations in timbre and loudness, effectively compensating for the auditory interruption caused by the loss of audio data packets.

[0065] Through the above technical solution, this application can more accurately extract key features of speech, especially pitch features, which is crucial for generating high-quality filler audio when audio data packets are lost. Accurate pitch extraction helps synthesize filler content that is highly consistent with the original speech in intonation, significantly improving the naturalness and auditory coherence of the filler audio. Therefore, even when unstable network transmission leads to audio data packet loss, users can experience a smoother audio stream with less distortion, effectively improving communication quality and user experience.

[0066] Specifically, the steps to suppress retransmission requests for lost audio data packets may include the following: setting a control flag in the communication protocol stack, which prevents the receiver from generating and sending independent retransmission request messages for audio data. The communication protocol stack can be understood as a software layer structure that handles network communication, such as a Transmission Control Protocol / Internet Protocol (TCP / IP) stack or a Real-Time Transport Protocol (RTP) stack. The control flag is a binary or multi-valued variable within the protocol stack; when its state is set to a specific value, it can indicate or trigger specific behaviors of the protocol stack. In this context, the control flag is configured to prevent the receiver from automatically generating and sending retransmission messages typically used to request lost data packets, such as retransmission requests in an Automatic Repeat Request (ARQ) mechanism, when audio data packet loss occurs.

[0067] This application's solution sets a specific control flag in the communication protocol stack, enabling the system to proactively intervene and prevent the receiving end from initiating retransmission requests when it detects lost audio data packets. Traditionally, data transmission protocols typically trigger retransmission mechanisms to ensure data integrity when data packet loss is detected. However, for real-time audio streams, retransmission can introduce significant latency, causing audio playback stuttering or inconsistency, severely impacting the user experience. By setting this control flag, the system can avoid unnecessary retransmission operations, especially when the number of lost audio data packets is small or can be effectively recovered by padding audio, thus maintaining the real-time performance and smoothness of the audio stream. This technical solution effectively avoids the additional latency and network overhead introduced by retransmission mechanisms, particularly in wireless communication environments or scenarios prone to data packet loss such as network congestion. This helps maintain the real-time performance of the audio stream, reduces audio stuttering or interruptions caused by retransmission waiting, and significantly improves the user's auditory experience during audio transmission. Furthermore, suppressing retransmission requests can also reduce network load and optimize system resource utilization.

[0068] In some embodiments described above, a scheme is proposed to analyze received audio data packets, extract speech feature information from audio frames, and cache it. However, in practical applications, received audio data packets may be subject to transient electromagnetic interference or noise contamination, causing distortion of the extracted timbre feature information. If this distorted timbre feature information is directly cached and used to generate filler audio, the quality of the synthesized audio may degrade, affecting the user experience.

[0069] In this regard, this application further proposes that the step of extracting and caching speech feature information from audio frames includes: Extract pitch feature information from the current audio frame; Determine whether the timbre features in the current audio frame are distorted; if so, extract clean timbre feature information from the clean timbre buffer, which pre-stores timbre feature information corresponding to audio frames that have not been contaminated by transient electromagnetic interference within a certain period in the past; if not, extract timbre feature information from the current audio frame. The pitch feature information and the timbre feature information are cached.

[0070] Specifically, when receiving an audio data packet and analyzing its audio frames, the pitch feature information of the current audio frame is first extracted. Pitch feature information typically reflects the fundamental frequency or periodicity of speech, and can be obtained through signal processing analysis of the audio frame. Simultaneously, the timbre features in the current audio frame need to be assessed to determine if they are distorted. Timbre features are an important component of the speech signal, determining the unique quality of the sound. Timbre distortion can be caused by noise, interference, or transmission errors. Determining whether timbre features are distorted can be achieved through various signal processing techniques; for example, analyzing the spectral characteristics, signal-to-noise ratio, or instantaneous zero-crossing rate of the audio frame.

[0071] If the assessment result indicates that the timbre features in the current audio frame are distorted, the timbre features of the current frame will not be used directly. Instead, clean timbre feature information will be extracted from a clean timbre buffer. This clean timbre buffer is a storage area that pre-stores high-quality timbre features derived from audio frames that have not been contaminated by transient electromagnetic interference within a certain past period. In this way, even if the timbre of the current frame is contaminated, it ensures that the timbre features used for subsequent filling are pure. Conversely, if the assessment result indicates that the timbre features in the current audio frame are not distorted, the timbre feature information will be directly extracted from the current audio frame. Finally, the extracted pitch feature information and the filtered or replaced timbre feature information are cached for use in generating fill audio in the event of audio data packet loss.

[0072] This application's solution effectively solves the problem of timbre feature distortion caused by transient interference during audio data packet transmission, thus affecting the quality of filler audio. This is achieved by introducing a mechanism to judge timbre feature distortion and a clean timbre buffer. When the received audio frame is timbre-distorted, the system can intelligently retrieve high-quality timbre features from the pre-stored clean timbre buffer, instead of using the tainted timbre of the current frame. This ensures that even in poor network conditions or with electromagnetic interference, the timbre features used for speech synthesis can maintain their original purity.

[0073] Through the above technical solution, this application can ensure that the speech feature information, especially the timbre feature information, used to generate filler audio is of high quality and uncontaminated when audio data packets are lost. This significantly improves the naturalness and listening experience of the filler audio, avoids auditory discomfort caused by timbre distortion, and thus can still provide a smooth, high-quality audio stream even when audio data packets are lost, greatly improving the user experience.

[0074] In some preferred embodiments, a specific example is given below. Suppose that during audio transmission, a sudden electromagnetic interference causes severe contamination of the timbre features of a certain audio frame (e.g., frame X) in the received audio data packet. When performing the step of extracting and caching speech feature information, the system first extracts the pitch feature information of frame X. Then, the timbre features of frame X are assessed, and distortion is found. At this point, the system does not directly use the distorted timbre features of frame X, but instead extracts clean, uncontaminated timbre feature information that matches the current speech content from a pre-established clean timbre buffer. For example, the clean timbre buffer may store the average timbre features of high-quality audio frames from the previous few seconds. Finally, the pitch feature information of frame X is cached together with the clean timbre feature information extracted from the clean timbre buffer. When subsequent audio data packets are lost and filler audio needs to be generated, these high-quality pitch and timbre features will be used for synthesis, thereby ensuring that even in interference environments, the generated filler audio can maintain a natural timbre similar to the original speech, avoiding the abruptness caused by timbre distortion.

[0075] In some embodiments described above, the steps of extracting and caching speech feature information are proposed. However, in actual implementation, simply extracting and caching speech feature information is insufficient to guarantee the generation of high-quality, natural-sounding fill audio when audio data packets are lost. A key challenge lies in how to effectively synthesize fill audio that can seamlessly connect with preceding and following audio streams while maintaining the original timbre and pitch trend using this cached speech feature information. If the synthesis method is inappropriate, even with accurate speech feature information, the generated fill audio may sound unnatural and abrupt, thus affecting the overall audio experience.

[0076] To this end, this application further proposes a step for generating filled audio, including: generating a periodic excitation signal based on the extracted pitch feature information; constructing a speech synthesis filter by using the timbre feature information as coefficients of a digital filter; and synthesizing a speech with the original timbre and extended timbre by passing the excitation signal through the speech synthesis filter. Continue filling audio with the lost pitch trend.

[0077] Specifically, when generating filler audio, a periodic excitation signal is first generated based on the pitch feature information extracted and cached from the audio data packet before it was lost. The pitch feature information reflects the fundamental frequency or periodicity of the speech. For spoken speech, this information can be used to generate a periodic pulse sequence simulating vocal cord vibration; for silent speech, it can generate a random noise signal. This excitation signal, acting as the "sound source" for speech synthesis, determines the pitch and some prosody of the synthesized speech. The extracted timbre feature information is used as coefficients for a digital filter to construct the speech synthesis filter. Timbre feature information typically characterizes the spectral envelope of speech, such as Linear Predictive Coding (LPC) coefficients and Mel-frequency cepstral coefficients (MFCC). By converting these timbre feature information into coefficients for the digital filter, the resonance characteristics of the human vocal tract can be simulated, thereby shaping speech with a specific timbre. This speech synthesis filter can be understood as filtering the excitation signal to give it a specific spectral shape, thus reproducing the timbre of the original speech.

[0078] In practical applications, the generated excitation signal is processed through the constructed speech synthesis filter to synthesize the filler audio. This synthesis process aims to ensure that the generated filler audio not only has the same original timbre as the audio data packet before the loss, but also smoothly continues the pitch trend of the speech before the loss. This means that the synthesized audio can be audibly seamlessly connected with the context, avoiding the abruptness or unnaturalness caused by packet loss.

[0079] This application's solution effectively addresses the problem of generating high-quality, natural filler audio when audio data packets are lost by employing a source-filter model-based speech synthesis method. Specifically, pitch feature information is used to generate an excitation signal representing the sound source, which carries the periodicity information of the speech. Simultaneously, timbre feature information is converted into coefficients of a digital filter that simulates the resonance characteristics of the human vocal tract, thus determining the spectral envelope of the speech. It is precisely because the excitation signal representing the sound source is processed by a speech synthesis filter representing the vocal tract characteristics that the synthesized filler audio can accurately reproduce the pitch and timbre of the original speech, thereby achieving a smooth transition and continuity with the audio before the loss. This synthesis method, which separates the sound source and vocal tract characteristics, allows the filler audio to maintain the original speech features while flexibly generating according to the duration of the loss, ensuring the coherence of the audio stream.

[0080] Through the above technical solution, this application can generate high-quality, natural-sounding filler audio, significantly improving the user experience in the event of audio data packet loss. This solution accurately utilizes pitch and timbre feature information for speech synthesis, ensuring that the filler audio not only continues the pitch trend before the loss but, more importantly, maintains a high degree of consistency with the original speech in timbre. This effectively avoids the sound quality degradation, distortion, or unnaturalness that may result from traditional packet loss concealment methods, allowing users to still obtain a smooth and clear audio experience even in environments with unstable networks or severe packet loss, thereby greatly improving the robustness of audio transmission and user satisfaction.

[0081] In some preferred embodiments, a specific example is given below. Suppose that during a VoIP call, consecutive audio data packets are lost due to network jitter. Before the loss occurs, the system has extracted and cached the pitch and timbre feature information of the previous frame of audio using the method described above. For example, the pitch feature information might be a fundamental frequency value (e.g., 150 Hz), while the timbre feature information might be a set of linear predictive coding (LPC) coefficients used to describe the spectral envelope of the speech frame.

[0082] When audio data packet loss is detected, the filler audio generation module reads the cached pitch and timbre feature information. First, based on the 150 Hz pitch feature information, a pulse sequence with a period of 1 / 150th of a second is generated as an excitation signal to simulate vocal cord vibration. Next, a digital filter is constructed using cached LPC coefficients, which simulates the speaker's current vocal tract shape. Finally, the generated pulse excitation signal is filtered through this digital filter. As a result, the synthesized filler audio will have the same timbre as the original speech, and its pitch will smoothly continue the 150 Hz trend, achieving a seamless auditory transition and effectively masking the interruption caused by data packet loss, making the audio stream interruption almost imperceptible to the user.

[0083] In some embodiments described above, a scheme for extracting and caching speech feature information is proposed, which involves extracting clean timbre feature information from a clean timbre buffer. The clean timbre buffer pre-stores timbre feature information corresponding to audio frames that have not been contaminated by transient electromagnetic interference within a certain past period. However, in actual implementation, accurately and reliably determining whether the received audio frames are truly free from noise contamination and ensuring that only high-quality timbre features are stored in the clean timbre buffer is crucial to the quality of subsequent audio filling synthesis. If this problem is not solved, contaminated timbre features may be mixed into the buffer, thus affecting the naturalness and listening experience of the filled audio. Therefore, this application further proposes a step for storing clean timbre feature information in the clean timbre buffer, aiming to ensure the purity of the cached timbre features through multi-dimensional analysis.

[0084] The steps for storing clean tone feature information in the clean tone buffer include: Perform spectral flatness analysis on the received audio frames to obtain spectral flatness values; The signal-to-noise ratio (SNR) of the audio frames is estimated to obtain the SNR. Monitor the instantaneous zero-crossing rate of the audio frame; Based on the spectral flatness value, the signal-to-noise ratio, and the instantaneous zero-crossing rate, it is determined whether the audio frame is contaminated by noise; When the audio frame is not contaminated by noise, the timbre features of the audio frame are extracted and stored in a clean timbre buffer.

[0085] Specifically, spectral flatness analysis involves calculating the spectrum of an audio frame to measure the flatness of its energy distribution. Typically, the spectrum of a speech signal has distinct peaks and valleys, while the spectrum of a noise signal is relatively flat. By calculating the spectral flatness value, it's possible to preliminarily determine whether there are significant noise components in the audio frame. Signal-to-noise ratio (SNR) estimation aims to quantify the intensity of the speech signal relative to background noise in an audio frame. A higher SNR generally indicates better audio frame quality and less noise interference. In practical applications, SNR can be estimated using various algorithms, such as those based on statistical models or spectral subtraction. Furthermore, the instantaneous zero-crossing rate refers to the number of times an audio signal crosses the zero axis per unit time. Speech signals and different types of noise signals exhibit different characteristics in terms of instantaneous zero-crossing rate; therefore, monitoring the instantaneous zero-crossing rate can serve as an auxiliary indicator for determining whether an audio frame is contaminated by noise.

[0086] Furthermore, after obtaining the spectral flatness value, signal-to-noise ratio, and instantaneous zero-crossing rate, the system comprehensively analyzes these indicators to determine whether an audio frame is contaminated by noise. For example, corresponding thresholds can be set; if the spectral flatness value is higher than a certain threshold, the signal-to-noise ratio is lower than a certain threshold, or the instantaneous zero-crossing rate exceeds the normal range, the audio frame is considered to be potentially contaminated by noise. Only when an audio frame is determined to be free from noise contamination will its timbre features be extracted and stored in the clean timbre buffer. This aims to ensure that the timbre features stored in the clean timbre buffer are of high quality and uncontaminated, thereby providing a reliable timbre benchmark for subsequent filler audio synthesis.

[0087] This application's solution effectively addresses the aforementioned problems by introducing a multi-dimensional noise detection mechanism. Specifically, by performing spectral flatness analysis, signal-to-noise ratio (SNR) estimation, and instantaneous zero-crossing rate monitoring on the received audio frames, the noise level of the audio frames can be quantified from different perspectives. Spectral flatness reflects the energy distribution characteristics of the audio signal; generally, noisy signals have higher spectral flatness. The SNR directly quantifies the intensity of the speech signal relative to background noise; a high SNR indicates better signal quality. The instantaneous zero-crossing rate provides the signal's temporal variation characteristics; speech and noise differ in their zero-crossing rates. Because these indicators complementarily reflect the noise status of the audio frames, the system can more accurately identify audio frames uncontaminated by noise. Therefore, only the timbre features of audio frames deemed clean are extracted and stored in the clean timbre buffer, ensuring the purity of the timbre features in the buffer and laying the foundation for subsequent generation of high-quality fill audio.

[0088] Through the above technical solution, this application effectively avoids storing noise-contaminated timbre features in a clean timbre buffer, thereby significantly improving the synthesis quality and naturalness of the fill audio. Compared to relying solely on preset conditions to determine whether timbre features are clean, this application performs multi-dimensional, real-time noise contamination assessment on audio frames, ensuring the purity of timbre features in the buffer. This allows for the synthesis of fill audio that more closely resembles the original speech using high-quality timbre features even when audio data packets are lost, further enhancing the user experience. This mechanism ensures the continuity and high-quality output of the audio stream even in complex communication environments.

[0089] In some preferred embodiments, a specific example is given below. Assume the system receives an audio frame. First, spectral flatness analysis is performed on the audio frame, calculating a spectral flatness value, for example, 0.7. Next, the signal-to-noise ratio (SNR) of the audio frame is estimated, yielding an SNR of 20 dB. Simultaneously, the instantaneous zero-crossing rate of the audio frame is monitored, for example, 50 times per millisecond. The system presets a series of thresholds; for example, when the spectral flatness value is below 0.8, the SNR is above 15 dB, and the instantaneous zero-crossing rate is between 40 and 60 times per millisecond, the audio frame is determined to be free from noise contamination. In this example, since all indicators meet the preset conditions, the system determines the audio frame to be clean. Subsequently, its timbre features are extracted from the audio frame and stored in a clean timbre buffer. When subsequent audio data packets are lost, the system extracts information from this buffer containing clean timbre features to synthesize high-quality filler audio, thereby effectively avoiding the impact of noise on the quality of the synthesized speech and ensuring a smooth transition of the audio stream.

[0090] In some embodiments described above in this application, a method is proposed to determine whether audio data packet loss exists based on the information of the received audio data packets. However, in practical applications, when the audio data packet information is a received signal strength indicator value, a simple signal strength drop may not be sufficient to accurately identify all types of packet loss, especially fragmented packet loss caused by signal jitter. This can lead to a deterioration in audio transmission quality that is difficult to detect and compensate for effectively.

[0091] In response, this application further proposes an audio transmission control method, wherein the information of the audio data packet is a received signal strength indication value, and the step of determining whether there is audio data packet loss based on the received signal strength indication value includes: Preset signal jitter warning threshold; Calculate the standard deviation of the most recent Y received signal strength indicators; If the standard deviation exceeds the signal jitter warning threshold, it is determined that there is audio data packet fragmentation and loss.

[0092] Specifically, when using the received signal strength indicator (RSI) to determine audio data packet loss, a signal jitter warning threshold must first be preset. This threshold is a pre-set value used to measure the acceptable range of fluctuations in the RSI. Its setting can be based on empirical data, network environment characteristics, or actual test results, aiming to distinguish between normal signal fluctuations and severe jitter that may lead to packet loss. Subsequently, the system continuously calculates the standard deviation of the most recent Y RSI values. Y RSI values ​​are the number of samples used to calculate the standard deviation, where Y is a natural number greater than 1. By calculating the standard deviation, the dispersion or fluctuation amplitude of the RSI over a short period can be quantified. A larger standard deviation indicates more severe signal fluctuations. Finally, the calculated standard deviation is compared with the preset signal jitter warning threshold. If the standard deviation exceeds the threshold, the system determines that audio data packet fragmentation and loss have occurred. This type of fragmented packet loss is usually not due to a complete signal interruption, but rather to the partial loss or damage of data packets caused by unstable signal quality, transient interference, or multipath effects, which manifests as drastic fluctuations in the received signal strength indication value.

[0093] This application's solution, by monitoring the fluctuations in the received signal strength indicator value rather than just its absolute value, can more precisely identify audio data packet fragmentation and loss caused by signal instability. Traditional packet loss detection methods may mainly rely on sequence number discontinuities or complete signal loss, but these methods may not be effective in identifying data packet corruption or partial loss caused by severe jitter in the signal strength, even if it exists. By calculating the standard deviation of the received signal strength indicator value, the stability of the signal can be quantified. When the standard deviation exceeds a preset threshold, it indicates a significant instantaneous deterioration in signal quality, which is often the root cause of audio data packet fragmentation and loss. Therefore, this solution can promptly detect and respond to such highly concealed packet loss situations.

[0094] Through the above technical solution, this application can effectively identify audio data packet fragmentation and loss caused by signal jitter, overcoming the shortcomings of relying solely on sequence numbers or simple signal strength judgments. This detection mechanism based on the fluctuation of received signal strength indicators enables the system to more accurately determine the type of packet loss when facing complex wireless environments or transient interference, thus providing a more precise basis for subsequent retransmission suppression and filler audio generation. This helps improve the robustness of audio transmission, especially in scenarios with unstable signals or transient interference, significantly enhancing the user experience and reducing audio stuttering or distortion caused by fragmented packet loss.

[0095] In some preferred embodiments, a specific example is given below. Assume the preset signal jitter warning threshold is 3dB. The system continuously monitors the received signal strength indication (RSI) values ​​of received audio data packets. For example, within a certain time period, the most recent five RSI values ​​are: -60dBm, -58dBm, -65dBm, -59dBm, and -62dBm. The system calculates the standard deviation of these five values. If the calculated standard deviation is 4.5dB, exceeding the preset 3dB jitter warning threshold, the system determines that audio data packet fragmentation and loss exist. In this case, even without detecting a significant discontinuity in sequence numbers, the system can identify potential audio quality problems and trigger corresponding retransmission suppression and filler audio generation mechanisms to maintain the continuity and quality of the audio stream. Conversely, if the standard deviation is 1.5dB, not exceeding the threshold, the signal fluctuation is considered acceptable and not classified as fragmentation and loss.

[0096] In some of the embodiments described above, this application proposes an audio transmission control method that effectively addresses the problem of audio data packet loss and maintains the continuity of the audio stream by generating filler audio. However, in actual deployment and operation of this method, a specific system architecture is required to support and coordinate various functions to ensure its efficient and stable execution. Therefore, this application proposes an audio transmission control system, aiming to provide a modular and operable implementation framework for the aforementioned audio transmission control method. Through clear functional division and collaborative operation, it ensures the effective implementation of the audio transmission control method.

[0097] Please refer to Figure 2 This application proposes an audio transmission control system, which, when using the aforementioned audio transmission control method, includes: Information monitoring module 11 is used to continuously monitor the information of received audio data packets, the information including serial number or received signal strength indication value; The speech feature extraction module 12 is used to analyze the received audio data packets, extract speech feature information from the audio frames, and cache it. The packet loss judgment module 13 is used to determine whether there is audio data packet loss based on the sequence number or the received signal strength indication value; The retransmission suppression module 14 is used to suppress retransmission requests for lost audio data packets; The filler audio generation module 15 is used to read the voice feature information cached before the audio data packet loss when there is audio data packet loss, and generate filler audio with matching duration based on the number of audio data packets lost. The audio stream integration module 16 is used to place the filler audio at the location where the audio data packet is lost, forming a continuous audio stream and outputting it.

[0098] Specifically, the information monitoring module 11 is configured to continuously monitor audio data packets received from the network and extract key information, such as the sequence number of the audio data packets or the received signal strength indicator value. This information forms the basis for subsequent judgments on the integrity and quality of the audio data packets. The speech feature extraction module 12 is designed to perform in-depth analysis of the received audio data packets, extracting speech feature information contained in the audio frames, such as pitch feature information, timbre feature information, and energy envelope representing instantaneous loudness changes in the speech signal. The extracted speech feature information is cached for use in filling audio generation when audio data packets are lost. The packet loss judgment module 13 performs logical judgments based on the sequence number or received signal strength indicator value provided by the information monitoring module 11 to determine whether audio data packets are lost. For example, when a discontinuous sequence number or abnormal jitter in the received signal strength indicator value is detected, it can be determined that packet loss exists. The retransmission suppression module 14 is activated when the packet loss judgment module 13 determines that audio data packets are lost, to prevent the system from initiating retransmission requests for these lost audio data packets. This can be achieved by setting specific control flags in the communication protocol stack, thereby avoiding unnecessary network overhead and latency. After confirming the loss of audio data packets, the filler audio generation module 15 reads the extracted speech feature information from the cache before the loss and, combined with the number of lost audio data packets, generates a filler audio that matches the duration of the loss. This filler audio aims to simulate the characteristics of the original speech for a smooth transition. The audio stream integration module 16 is responsible for precisely inserting the generated filler audio into the position where the original audio data packets were lost. In this way, even with data packet loss, a continuous and uninterrupted audio stream can be formed and output to the user, thereby significantly improving the user experience.

[0099] The solution proposed in this application decouples the various functional steps in the audio transmission control method and maps them to independent modules, thereby achieving a systematic and structured processing flow. Specifically, the information monitoring module 11 serves as the data entry point, continuously acquiring the status of audio data packets; the speech feature extraction module 12 processes the audio content in parallel, providing the necessary data foundation for subsequent padding operations. When the packet loss judgment module 13 identifies a lost data packet, the retransmission suppression module 14 immediately intervenes to optimize network resource utilization and avoid invalid retransmissions. Simultaneously, the padding audio generation module 15 intelligently synthesizes the missing audio content using pre-cached speech feature information. Finally, the audio stream integration module 16 seamlessly connects the original audio with the synthesized padding audio, ensuring the continuity of the output audio stream. This modular collaborative working mechanism enables the complex audio transmission control logic to be executed efficiently and reliably, effectively solving problems such as high functional coupling and maintenance difficulties that may be encountered in actual system deployment.

[0100] Through the above technical solution, the audio transmission control system of this application provides a clear, scalable, and easy-to-maintain architecture for implementing complex audio transmission control methods. This modular design clarifies the responsibilities of each functional unit, reduces the complexity of system development, and improves the system's robustness. Furthermore, by concretizing the method steps into specific modules, the system can respond more efficiently to changes in network conditions, such as quickly identifying and handling audio packet loss while avoiding unnecessary retransmissions, thereby optimizing network bandwidth utilization. Thus, this system can ensure a high-quality, continuous audio experience even in the event of network instability or packet loss, significantly improving user satisfaction.

[0101] In some preferred embodiments, a specific example is given below. Assume that in a real-time voice call application, audio data packets are transmitted via a wireless network. When network instability causes some audio data packets to be lost, the audio transmission control system of this application will come into play. Specifically, the information monitoring module 11 continuously receives and analyzes the incoming audio data packets, recording their sequence number and received signal strength indication value. For example, when the information monitoring module 11 detects that the sequence number jumps from N to N+4, indicating that audio data packets with sequence numbers N+1, N+2, and N+3 are lost, this information is passed to the packet loss judgment module 13. After confirming that the audio data packets with sequence numbers N+1, N+2, and N+3 are indeed lost, the packet loss judgment module 13 notifies the retransmission suppression module 14. The retransmission suppression module 14 then sets a control flag in the communication protocol stack to prevent the system from sending retransmission requests independently for the lost audio data packets N+1, N+2, and N+3, thereby avoiding additional delays and network congestion introduced by retransmission. Meanwhile, the speech feature extraction module 12 has already extracted and cached the speech feature information, such as pitch and timbre, of the audio data packets N+1, N+2, and N+3 before they are lost. Once the loss of audio data packets N+1, N+2, and N+3 is confirmed, the filler audio generation module 15 reads this cached speech feature information and, combined with the number of lost audio data packets (three in this example), generates filler audio that matches the duration of the lost data packets. For example, this filler audio can continue the pitch trend and timbre characteristics of the speech before the loss. Finally, the audio stream integration module 16 precisely inserts the generated filler audio into the positions of audio data packets N+1, N+2, and N+3 in the original audio stream. In this way, even if the original data packets are lost, the audio stream heard by the user is continuous and smooth, without noticeable stuttering or interruption, thus greatly improving the quality of voice calls and the user experience.

[0102] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An audio transmission control method, characterized in that, Including the following steps: Continuously monitor information of received audio data packets, including sequence numbers or received signal strength indicators; Analyze the received audio data packets, extract speech feature information from the audio frames, and cache it; Based on the sequence number or received signal strength indicator value, determine whether there is audio data packet loss. If so, suppress retransmission requests for lost audio data packets; if not, continue to monitor the information of received audio data packets. In the event of audio data packet loss, read the voice feature information cached before the audio data packet loss, and combine it with the number of audio data packets lost to generate duration-matched filler audio; The filler audio is placed at the location where the audio data packet is missing, forming a continuous audio stream, which is then output.

2. The audio transmission control method according to claim 1, characterized in that, The information in the audio data packet is a serial number. Determining whether an audio data packet is lost based on the serial number includes the following steps: Cases of discontinuous sequence numbers are treated as lost audio data packets and counted accordingly. When the number of lost audio data packets exceeds a preset threshold, it is determined that there is a cluster of lost audio data packets.

3. The audio transmission control method according to claim 2, characterized in that, The step of treating discontinuous sequence numbers as lost audio data packets and counting them includes: After receiving an audio data packet with sequence number N, if the sequence number of the next audio data packet is ≥ N + K, the audio data packet is considered lost. N is a natural number greater than or equal to 1, and K is a natural number greater than 1. An internal continuous packet loss counting mechanism is activated to record the number of consecutively lost audio data packets; the internal continuous packet loss counting mechanism includes: The number of lost audio data packets is incremented each time a loss is detected. If consecutive audio data packets with consecutive sequence numbers are received before the number of lost packets reaches a preset threshold, the audio data packets are considered to have been lost in a non-clustered manner, and the accumulated number of lost packets is reset to zero; or If the sequence number of the next audio data packet is N+M and M is greater than the preset threshold before the number of lost data packets reaches the preset threshold, it is considered a new audio data packet loss situation, and the accumulated number of lost data packets is reset to zero and recounted.

4. The audio transmission control method according to claim 1, characterized in that, The speech feature information includes pitch feature information, timbre feature information, and an energy envelope representing the instantaneous loudness change of the speech signal. The steps for extracting the pitch feature information include: Calculate the autocorrelation function of the audio frame; Find the maximum peak value of the autocorrelation function at a non-zero delay; The speech period is obtained based on the maximum peak value, and the pitch feature information is obtained based on the speech period.

5. The audio transmission control method according to claim 1, characterized in that, The step of suppressing retransmission requests for lost audio data packets includes: A control flag is set in the communication protocol stack to prevent audio data, including the receiver, from generating and sending independent retransmission request messages.

6. The audio transmission control method according to claim 1, characterized in that, The step of extracting and caching speech feature information from audio frames includes: Extract pitch feature information from the current audio frame; Determine whether the timbre features in the current audio frame are distorted; if so, extract clean timbre feature information from the clean timbre buffer, which pre-stores timbre feature information corresponding to audio frames that have not been contaminated by transient electromagnetic interference within a certain period in the past; if not, extract timbre feature information from the current audio frame. The pitch feature information and the timbre feature information are cached.

7. The audio transmission control method according to claim 6, characterized in that, The steps to generate fill audio include: Based on the extracted pitch feature information, a periodic excitation signal is generated; The timbre feature information is used as the coefficients of a digital filter to construct a speech synthesis filter; The excitation signal is passed through the speech synthesis filter to synthesize a filled audio that has the original timbre and continues the pitch trend before the loss.

8. The audio transmission control method according to claim 6, characterized in that, The steps for storing clean tone feature information in the clean tone buffer include: Perform spectral flatness analysis on the received audio frame to obtain the spectral flatness value; estimate the signal-to-noise ratio (SNR) of the audio frame to obtain the SNR; monitor the instantaneous zero-crossing rate of the audio frame; Based on the spectral flatness value, the signal-to-noise ratio, and the instantaneous zero-crossing rate, determine the... Is the audio frame contaminated with noise? When the audio frame is not contaminated by noise, the timbre features of the audio frame are extracted and stored in a clean timbre buffer.

9. The audio transmission control method according to claim 1, characterized in that, The information in the audio data packet is a received signal strength indication value. The step of determining whether an audio data packet has been lost based on the received signal strength indication value includes: Preset signal jitter warning threshold; Calculate the standard deviation of the most recent Y received signal strength indicators; If the standard deviation exceeds the signal jitter warning threshold, it is determined that there is audio data packet fragmentation and loss.

10. An audio transmission control system, employing the audio transmission control method according to any one of claims 1-9, characterized in that, include: The information monitoring module is used to continuously monitor the information of the received audio data packets, including the sequence number or the received signal strength indicator value; The speech feature extraction module is used to analyze the received audio data packets, extract speech feature information from the audio frames, and cache it. The packet loss detection module is used to determine whether there is any audio data packet loss based on the sequence number or the received signal strength indicator value. The retransmission suppression module is used to suppress retransmission requests for lost audio data packets; The filler audio generation module is used to read the voice feature information cached before the audio data packets were lost, and generate filler audio with matching duration based on the number of audio data packets lost. The audio stream integration module is used to place the filler audio at the location where the audio data packet is lost, forming a continuous audio stream and outputting it.

Images