Voice QoS optimizing and playing method, device and equipment facing end-to-end information source encryption and medium

Abstract

The invention discloses a voice QoS optimizing and playing method, device and equipment for end-to-end information source encryption and a medium, and relates to the field of instant messaging, and the method comprises the steps: storing a received encrypted voice data packet into a jitter buffer area in a ciphertext form; determining a current target buffer water level of the jitter buffer based on the current network jitter degree and a delay factor introduced by the current decryption processing; generating a play control strategy and a decryption scheduling strategy according to the current actual buffer water level and the current target buffer water level of the jitter buffer; the decryption scheduling strategy comprises a decryption operation frequency, a triggering opportunity and an execution priority, and the decryption scheduling strategy is adjusted based on a real-time playing control strategy and a jitter buffer area state; and scheduling the corresponding encrypted voice data packet from the jitter buffer area by using a decryption scheduling strategy for decryption, and decoding and playing the decrypted voice data packet according to a playing control strategy. According to the invention, the overall fluency of encrypted real-time voice communication is improved.

Description

Technical Field

[0001] This application relates to the field of instant messaging, and in particular to a method, apparatus, device and medium for voice QoS optimization and playback with end-to-end source encryption. Background Technology

[0002] In real-time voice communication (such as VoIP, Voice over Internet Protocol), QoS (Quality of Service) mechanisms are crucial to ensure smooth and clear calls. Due to inherent packet loss, latency, and jitter in network transmission, the receiving end typically employs jitter buffering technology to ensure playback continuity. NetEQ, as an advanced adaptive jitter control and packet loss compensation algorithm, effectively improves voice quality under typical network conditions by dynamically adjusting the buffer and performing operations such as acceleration, deceleration, and packet loss compensation.

[0003] With the increasing demand for communication security, end-to-end source encryption has become a standard means of ensuring privacy in real-time voice communication. The sending end encrypts the encoded voice data, while the receiving end must decrypt it before subsequent QoS processing and playback. Current mainstream implementations typically use a simple serial connection between the encryption / decryption module and QoS processing modules like NetEQ. This traditional serial processing architecture has significant drawbacks: First, the decryption operation itself introduces additional latency and jitter. This overhead, combined with network jitter, exacerbates the overall jitter of NetEQ processing, increasing the probability of voice playback stuttering. Second, in the traditional serial processing architecture, the encryption / decryption module and the QoS processing module are independent. This design not only increases the complexity of system integration and maintenance but also makes it difficult to achieve deep collaboration between decryption scheduling and QoS processing mechanisms, limiting the rapid deployment and performance optimization of encrypted voice communication solutions. Third, in scenarios with sudden packet arrivals or increased network jitter, the limited encryption / decryption computing resources in the serial architecture cannot handle the immediate decryption needs of a large number of packets. A large number of packets accumulate in the decryption stage, delaying the timing of subsequent QoS processing. Meanwhile, some data packets that are ultimately dropped due to excessive latency will still be decrypted, resulting in unnecessary consumption of computing resources. These two issues together weaken the system's ability to maintain voice continuity and smooth playback under high load or unstable network conditions. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, apparatus, device, and medium for voice QoS optimization and playback with end-to-end source encryption. By introducing and compensating for the latency and jitter caused by the encryption and decryption process in the real-time voice QoS processing mechanism, the overall smoothness of encrypted real-time voice communication is effectively improved. The specific solution is as follows:

[0005] Firstly, this application provides a voice QoS optimization and playback method for end-to-end source encryption, including:

[0006] Receive encrypted voice data packets from an end-to-end source and store the encrypted voice data packets in ciphertext form in a jitter buffer;

[0007] Based on the current network jitter level and the delay factor introduced by the current decryption process, the current target buffer level of the jitter buffer is determined; wherein, the network jitter level is used to reflect the fluctuation of network transmission delay, and the delay factor introduced by the decryption process includes the average decryption time and the decryption time jitter level of the encrypted voice data packets within a preset time period.

[0008] Based on the current actual buffer level of the jitter buffer and the current target buffer level, a playback control strategy and a decryption scheduling strategy are generated; wherein, the decryption scheduling strategy includes the frequency, triggering timing and execution priority of decryption operations, and the decryption scheduling strategy is adjusted based on the real-time playback control strategy and jitter buffer status;

[0009] The corresponding encrypted voice data packets are scheduled from the jitter buffer using the decryption scheduling strategy for decryption, and the decrypted voice data packets are decoded and played according to the playback control strategy.

[0010] Optionally, the receiving end-to-end encrypted voice data packet and storing the encrypted voice data packet in ciphertext form in a jitter buffer includes:

[0011] Receive encrypted voice data packets from an end-to-end encrypted source and perform validity checks on the encrypted voice data packets; the validity checks include determining whether the data packets arrived after a timeout and whether the data packets are duplicate data packets;

[0012] If the encrypted voice data packet is detected to have arrived after a timeout or is a duplicate data packet, the encrypted voice data packet is determined to be an invalid data packet and is discarded directly. If the encrypted voice data packet is detected to have arrived without a timeout and is a non-duplicate data packet, the encrypted voice data packet is determined to be a valid data packet and is directly stored in the jitter buffer.

[0013] Optionally, determining the current target buffer level of the jitter buffer based on the current network jitter level and the latency introduced by the current decryption process includes:

[0014] The target formula is constructed based on the base buffer level corresponding to the jitter buffer, the degree of network jitter, the average decryption time and the degree of jitter of decryption time, the security factor and the security margin of encrypted voice data packets within a preset time period;

[0015] Wherein, the basic buffer level is a pre-set default value that meets the preset playback smoothness standard; the safety factor is a pre-set weight value used to amplify the compensation intensity of decryption time jitter; the safety margin is an additional buffer reserve value used to cope with sudden decryption delays or network fluctuations; and the target formula is:

[0016] Target buffer level = Base buffer level + Network jitter level + (Average decryption time + Security factor × Decryption time jitter level) + Security margin;

[0017] The current target buffer level of the jitter buffer is determined using the target formula.

[0018] Optionally, a playback control strategy is generated based on the current actual buffer level of the jitter buffer and the current target buffer level, including:

[0019] Compare the current actual buffer level of the jitter buffer with the current target buffer level;

[0020] If the current actual buffer level is higher than the current target buffer level, a preset accelerated playback strategy is generated.

[0021] If the current actual buffer level is equal to the current target buffer level, a preset default playback strategy is generated;

[0022] If the current actual buffer level is lower than the current target buffer level, a preset deceleration playback strategy is generated.

[0023] If packet loss is detected and the current actual buffer level is insufficient to compensate for the playback gap caused by packet loss, a preset packet loss compensation strategy is generated to maintain voice continuity.

[0024] After executing the packet loss compensation strategy, if normal data packets are received again, a preset fusion processing strategy is generated to smoothly connect the compensation data and the normal data.

[0025] Optionally, the execution priority includes the scheduling priority of the decryption task in the overall computing resources of the system, and / or the processing order priority of each encrypted voice data packet within the decryption task;

[0026] Accordingly, adjusting the decryption scheduling strategy based on the real-time playback control strategy and the jitter buffer state includes:

[0027] If the current playback control strategy is the preset accelerated playback strategy, and the current actual buffer level of the jitter buffer is higher than the first preset water level threshold or the data packet accumulation rate exceeds the preset accumulation rate threshold, then the frequency of the decryption operation will be adjusted to the preset highest frequency, the triggering time of the decryption operation will be advanced, and the scheduling priority of the decryption task in the overall computing resources of the system will be adjusted to the first priority that meets the preset high priority standard.

[0028] If the current playback control strategy is the preset default playback strategy, and the current actual buffer level of the jitter buffer is within the preset normal level range, then the frequency of the decryption operation will be adjusted to the preset base frequency, the triggering time of the decryption operation will be determined according to the receiving order of the data packets and the expected playback timestamp, and the scheduling priority of the decryption task in the overall computing resources of the system will be adjusted to the second priority that meets the preset priority standard.

[0029] If the current playback control strategy is a preset deceleration playback strategy, and the current actual buffer level of the jitter buffer is lower than the second preset level threshold, the frequency of the decryption operation will be adjusted to a preset proportion of the preset base frequency, the triggering time of the decryption operation will be delayed, and the scheduling priority of the decryption task in the overall computing resources of the system will be adjusted to the third priority that meets the preset low priority standard.

[0030] Optionally, the process of scheduling corresponding encrypted voice data packets from the jitter buffer for decryption using the decryption scheduling strategy further includes:

[0031] Record the actual time taken for each encrypted voice data packet from the start of decryption to the completion of decryption in real time, and generate a decryption time log;

[0032] When the preset statistical period is reached, or the cumulative number of decrypted data packets reaches the preset threshold, a new average decryption time and decryption time jitter are determined based on the decryption time log, and the cumulative number of decrypted data packets is recounted.

[0033] The new average decryption time and decryption time jitter are fed back to the step of determining the current target buffer level of the jitter buffer to obtain the new current target buffer level.

[0034] Optionally, after decoding and playing the decrypted voice data packets according to the playback control strategy, the method further includes:

[0035] The new current actual buffer level corresponding to the jitter buffer is determined, and the new current actual buffer level is fed back to the step of generating a playback control strategy and a decryption scheduling strategy based on the current actual buffer level of the jitter buffer and the current target buffer level, so as to obtain a new playback control strategy and a decryption scheduling strategy.

[0036] Secondly, this application provides a voice QoS optimization and playback device for end-to-end source encryption, comprising:

[0037] The data receiving module is used to receive encrypted voice data packets encrypted from an end-to-end source and store the encrypted voice data packets in ciphertext form in a jitter buffer.

[0038] The water level determination module is used to determine the current target buffer water level of the jitter buffer based on the current network jitter level and the delay factor introduced by the current decryption process; wherein, the network jitter level is used to reflect the fluctuation of network transmission delay, and the delay factor introduced by the decryption process includes the average decryption time and the decryption time jitter level of the encrypted voice data packets within a preset time period.

[0039] The strategy generation module is used to generate a playback control strategy and a decryption scheduling strategy based on the current actual buffer level of the jitter buffer and the current target buffer level; wherein, the decryption scheduling strategy includes the frequency, triggering timing and execution priority of decryption operations, and adjusts the decryption scheduling strategy based on the real-time playback control strategy and jitter buffer status;

[0040] The decryption and playback module is used to schedule corresponding encrypted voice data packets from the jitter buffer for decryption using the decryption scheduling strategy, and to decode and play the decrypted voice data packets according to the playback control strategy.

[0041] Thirdly, this application provides an electronic device, comprising:

[0042] Memory, used to store computer programs;

[0043] A processor is used to execute the computer program to implement the aforementioned voice QoS optimization and playback method oriented towards end-to-end source encryption.

[0044] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned voice QoS optimization and playback method oriented towards end-to-end source encryption.

[0045] In this application, encrypted voice data packets are received from an end-to-end source and stored in a jitter buffer in ciphertext form. Based on the current network jitter level and the latency introduced by the current decryption process, the current target buffer level of the jitter buffer is determined. The network jitter level reflects the fluctuation of network transmission delay, and the latency introduced by the decryption process includes the average decryption time and the jitter level of the decryption time within a preset time period. A playback control strategy and a decryption scheduling strategy are generated based on the current actual buffer level and the current target buffer level of the jitter buffer. The decryption scheduling strategy includes the frequency, triggering timing, and execution priority of decryption operations, and is adjusted based on the real-time playback control strategy and the jitter buffer state. The decryption scheduling strategy is used to schedule corresponding encrypted voice data packets from the jitter buffer for decryption, and the decrypted voice data packets are decoded and played according to the playback control strategy. As can be seen from the above, on the one hand, when determining the target buffer level of the jitter buffer, this application not only considers the degree of network jitter but also introduces the latency factor brought by the decryption process. This enables buffer management to proactively predict and compensate for the latency and jitter introduced by the decryption process, and to absorb the overall jitter by dynamically adjusting the buffer size, thereby reducing the misjudgment of playback timing caused by decryption latency fluctuations, thus reducing the probability of voice stuttering and improving playback smoothness. On the other hand, the frequency, triggering timing, and priority of decryption operations are dynamically adjusted according to the real-time playback control strategy and the jitter buffer status, so that the decryption operation is no longer an isolated fixed process, but closely adapted to the needs of QoS processing. This integrated control mechanism simplifies the system architecture, reduces the complexity of external module splicing, and optimizes the connection efficiency between decryption and playback through internal collaboration, which is conducive to the rapid deployment and performance improvement of encrypted voice communication solutions. In addition, storing encrypted voice data packets directly into the jitter buffer in ciphertext form and triggering decryption on demand based on the decryption scheduling strategy changes the mode in traditional serial processing where all data packets must be decrypted immediately. When data packets arrive in bursts, the decryption rhythm can be flexibly adjusted according to the buffer level and playback control strategy to avoid the backlog and blockage of the decryption process. Meanwhile, for data packets that might be dropped due to excessive latency, decryption operations can be avoided or delayed through decryption scheduling strategies, thereby reducing invalid decryption calculations and saving computing resources. This combination of on-demand decryption and dynamic scheduling enhances the system's ability to maintain continuous voice playback under high load or unstable network conditions. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0047] Figure 1 This application discloses a flowchart of a voice QoS optimization and playback method for end-to-end source encryption.

[0048] Figure 2 This is a schematic diagram of a specific system architecture disclosed in this application;

[0049] Figure 3 This is a schematic diagram of the communication process of encrypted voice data in the sending and receiving directions disclosed in this application;

[0050] Figure 4 This is a schematic diagram illustrating the interaction between a decryption module and an operation decision module as disclosed in this application;

[0051] Figure 5 This is a schematic diagram of the system architecture of a NetEQ processing module that supports encrypted real-time voice, as disclosed in this application.

[0052] Figure 6 This is a schematic diagram of the voice QoS optimization and playback device for end-to-end source encryption disclosed in this application;

[0053] Figure 7 This is a schematic diagram of the structure of an electronic device disclosed in this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] Current mainstream implementations typically use a simple serial connection between the encryption / decryption module and QoS processing modules such as NetEQ. This traditional serial processing architecture has significant drawbacks, specifically: First, it fails to consider the impact of the encryption / decryption process, leading to inaccurate decision-making and voice stuttering. Second, the encryption / decryption module and the QoS processing module are independent, making it difficult to achieve deep collaboration between decryption scheduling and QoS processing mechanisms, thus limiting the rapid deployment and performance optimization of encrypted voice communication solutions. Third, in the traditional serial architecture, the decryption process easily becomes a performance bottleneck, resulting in invalid decryption and resource waste. Therefore, this application provides a voice QoS optimization and playback method for end-to-end source encryption. By introducing and compensating for the latency and jitter caused by the encryption / decryption process in the real-time voice QoS processing mechanism, it effectively improves the overall smoothness of encrypted real-time voice communication.

[0056] See Figure 1 As shown in the figure, this application discloses a voice QoS optimization and playback method for end-to-end source encryption, including:

[0057] Step S11: Receive the encrypted voice data packet from the end-to-end source and store the encrypted voice data packet in ciphertext form in the jitter buffer.

[0058] like Figure 2 As shown, this embodiment adds an encryption / decryption module to the existing MCU and DSP modules in the NetEQ component of WebRTC (Web Real-Time Communications, a real-time communication technology), to realize the decryption scheduling and decryption of RTP (Real-time Transport Protocol) ciphertext.

[0059] like Figure 3 As shown, in the transmitting direction, after acquiring speech PCM (Pulse Code Modulation) data from the microphone device, preprocessing is first performed, including but not limited to acoustic echo cancellation (AEC), noise suppression (NS), and automatic gain control (AGC). Then, speech encoding is performed, and the encoded speech data is encrypted at the source level by a data encryption module. The encrypted speech data packets are then transmitted through the transmitting thread. In the receiving direction, the encrypted speech data packets are received through a receiving thread. The NetEQ processing module, which supports encrypted real-time speech, performs data decryption and QoS processing, completing the decryption and QoS processing of RTP data packets in the source encryption scenario.

[0060] The interaction flow of each part in the NetEQ processing module that supports encrypted real-time voice is as follows:

[0061] First, it can receive encrypted voice data packets from an end-to-end encrypted source and perform validity checks on the encrypted voice data packets. Validity checks include, but are not limited to, determining whether the data packet arrived after a timeout and whether the data packet is a duplicate data packet. If an encrypted voice data packet is detected to have arrived after a timeout or is a duplicate data packet, the encrypted voice data packet can be determined as an invalid data packet and discarded directly. If an encrypted voice data packet is detected to have arrived before a timeout and is a non-duplicate data packet, the encrypted voice data packet can be determined as a valid data packet and directly stored in the jitter buffer.

[0062] Step S12: Determine the current target buffer level of the jitter buffer based on the current network jitter level and the delay factor introduced by the current decryption process; wherein, the network jitter level is used to reflect the fluctuation of network transmission delay, and the delay factor introduced by the decryption process includes the average decryption time and the decryption time jitter level of the encrypted voice data packets within a preset time period.

[0063] In this embodiment, determining the current target buffer level of the jitter buffer based on the current network jitter level and the latency introduced by the current decryption process can include: firstly, constructing a target formula based on the base buffer level corresponding to the jitter buffer, the network jitter level, the average decryption time and decryption time jitter level of encrypted voice data packets within a preset time period, the security factor, and the security margin; wherein, the base buffer level is a preset default value or fixed benchmark value that meets the preset playback smoothness standard, the security factor is a preset weight value used to amplify the compensation strength of the decryption time jitter level to cover abnormal fluctuations, and the security margin is an additional buffer reserve value used to cope with sudden decryption latency or network fluctuations, which can be fixed or dynamically adjusted, and the network jitter level can be determined by the root mean square (RMS) value of the network transmission jitter of the most recent preset number of data packets. The jitter level of decryption time can be represented by the root mean square (RMS) of the decryption time of the most recent preset number of data packets. The target formula is: Target buffer level = Base buffer level + Network jitter level + (Average decryption time + Security factor × Decryption time jitter level) + Security margin. Then, the current target buffer level of the jitter buffer is determined using the target formula.

[0064] Understandably, when significant network latency fluctuations or large decryption delays are detected, the target buffer level of the jitter buffer can be increased to reduce the possibility of packet loss. Conversely, when network conditions are good, the target buffer level of the jitter buffer can be decreased to minimize playback latency.

[0065] Step S13: Generate a playback control strategy and a decryption scheduling strategy based on the current actual buffer level of the jitter buffer and the current target buffer level; wherein, the decryption scheduling strategy includes the frequency, triggering timing and execution priority of decryption operations, and adjusts the decryption scheduling strategy based on the real-time playback control strategy and jitter buffer status.

[0066] After determining the target buffer level of the jitter buffer, the operation decision module can use real-time analysis of the current actual buffer level and the current target buffer level to output precise playback control strategies, such as accelerated playback, decelerated playback, fusion processing, normal playback, and packet loss compensation, to solve problems such as playback stuttering or excessive latency caused by network jitter, latency, and decryption fluctuations. Specifically, this can include comparing the current actual buffer level with the current target buffer level. If the current actual buffer level is higher than the current target buffer level, it indicates excess buffer data and a risk of excessive latency. In this case, a preset accelerated playback strategy is generated to quickly consume redundant buffer data and reduce end-to-end latency. If the current actual buffer level is equal to the current target buffer level, it indicates the buffer is in an optimal state. In this case, a preset default playback strategy, also known as a preset normal playback strategy, is generated to maintain a balance between latency and smoothness. If the current actual buffer level is lower than the current target buffer level, it indicates insufficient buffer data and a risk of stuttering. In this case, a preset decelerated playback strategy is generated to increase buffer reserves and avoid stuttering. If packet loss is detected and the current buffer level is insufficient to compensate for the playback gap caused by the packet loss, a preset packet loss compensation strategy is generated to maintain voice continuity. After the packet loss compensation strategy is executed, if normal data packets are received again, a preset fusion processing strategy is generated to smoothly connect the compensated data and normal data to improve the coherence of the voice data.

[0067] Furthermore, the decryption module can determine the decryption scheduling strategy based on the real-time playback control strategy and the jitter buffer status, including but not limited to the frequency, triggering timing, and execution priority of decryption operations. The execution priority includes the scheduling priority of the decryption task within the overall system computing resources and / or the processing order priority of each encrypted voice data packet within the decryption task. Specifically, this can include: if the current playback control strategy is a preset accelerated playback strategy, and the current actual buffer level of the jitter buffer is higher than a first preset level threshold or the data packet accumulation rate exceeds a preset accumulation rate threshold, then the frequency of the decryption operation is adjusted to a preset highest frequency, the triggering timing of the decryption operation is advanced, and the scheduling priority of the decryption task within the overall system computing resources is adjusted to the first priority that meets the preset high priority standard. If the current playback control strategy is a preset default playback strategy, and the current actual buffer level of the jitter buffer is within a preset normal level range, then the frequency of the decryption operation is adjusted to a preset base frequency, the triggering timing of the decryption operation is determined based on the data packet reception order and the expected playback timestamp, and the scheduling priority of the decryption task within the overall system computing resources is adjusted to the second priority that meets the preset medium priority standard. If the current playback control strategy is the preset deceleration playback strategy, and the current actual buffer level of the jitter buffer is lower than the second preset level threshold, the frequency of the decryption operation will be adjusted to a preset proportion of the preset base frequency, the triggering time of the decryption operation will be delayed, and the scheduling priority of the decryption task in the overall computing resources of the system will be adjusted to the third priority that meets the preset low priority standard.

[0068] Then, the decryption module can use the decryption scheduling strategy to schedule the corresponding encrypted voice data packets from the jitter buffer for decryption, and send the decrypted plaintext data to the decoder.

[0069] In addition, the operation decision module can also send future playback plans to the decryption module, such as the expected playback time of the next 5 data packets. After receiving the future playback plan, the decryption module decrypts the corresponding data packets in advance, reducing the delay of on-demand decryption.

[0070] Step S14: Decrypt the corresponding encrypted voice data packets from the jitter buffer using the decryption scheduling strategy, and decode and play the decrypted voice data packets according to the playback control strategy.

[0071] In this embodiment, the process of scheduling corresponding encrypted voice data packets from the jitter buffer for decryption using a decryption scheduling strategy may further include: recording the actual time taken for each encrypted voice data packet from the start of decryption to completion in real time, and generating a decryption time log. When a preset statistical period is reached, or the cumulative number of decrypted data packets reaches a preset threshold, a new average decryption time and decryption time jitter level are determined based on the decryption time log, and the cumulative number of decrypted data packets is recounted. Then, the new average decryption time and decryption time jitter level are fed back to the step of determining the current target buffer level of the jitter buffer to obtain the new current target buffer level.

[0072] Furthermore, after decoding and playing the decrypted voice data packets according to the playback control strategy, the actual level of the jitter buffer will be updated in real time. Therefore, the new current actual buffer level corresponding to the jitter buffer can be determined, and the new current actual buffer level can be fed back to the step of generating the playback control strategy and decryption scheduling strategy based on the current actual buffer level of the jitter buffer and the current target buffer level, so as to obtain the new playback control strategy and decryption scheduling strategy.

[0073] For example Figure 4 As shown, on the one hand, the new average decryption time and decryption time jitter obtained by the decryption module after executing the decryption scheduling strategy are fed back to the target level calculation module. The target level calculation module recalculates the current target buffer level of the jitter buffer based on the new average decryption time and decryption time jitter, as well as the current network jitter level. On the other hand, after decoding and playing the decrypted voice data packets according to the playback control strategy, the actual level of the jitter buffer is updated in real time. The operation decision module generates a new playback control strategy based on the updated current actual buffer level and the current target buffer level. The new playback control strategy is also fed back to the decryption module in real time, and the decryption module generates a new decryption scheduling strategy. In this way, a dynamic closed loop is formed, continuously adapting to changes in network conditions.

[0074] The following is based on Figure 5 The technical solution in this embodiment is explained using a schematic diagram of a NetEQ processing module that supports encrypted real-time voice as an example.

[0075] Figure 5The process is divided into two main parts: encrypted NetEQ input and encrypted NetEQ output. On the input side, the received encrypted RTP media stream is processed by RTP packet reception and stored in ciphertext form in the jitter buffer. Simultaneously, the network jitter estimation module calculates the average and peak jitter based on packet arrival conditions. Combining the decryption delay and jitter parameters fed back by the decryption module, the target water level calculation module comprehensively calculates the current target water level. On the output side, the operation decision module, based on the current buffer size, current target water level, and previous operation type, determines the specific operation type, such as acceleration, deceleration, normal playback, merging, or packet loss compensation. The decryption scheduling strategy module generates a decryption scheduling strategy based on the operation type and buffer state, controlling the decryption operation to retrieve RTP ciphertext data from the buffer for decryption, obtaining plaintext data. Further, the audio decoding module decodes the plaintext data, obtaining decoded data and storing it in the buffer queue. The decoded data performs corresponding actions in the algorithm buffer according to the operation type. Finally, the processed data is inserted into the playback queue, forming played data and data to be played. It completes a closed-loop processing flow from encrypted reception, adaptive buffering, intelligent decision-making, on-demand decryption to smooth playback.

[0076] As can be seen from the above, this application, while retaining the existing NetEQ's anti-packet loss and anti-jitter capabilities, introduces encryption / decryption latency and jitter as dynamic parameters into jitter buffer management and playback decisions. Through a scheduling mechanism that combines on-demand decryption and pre-decryption, it significantly reduces voice stuttering caused by decryption delay and sudden arrival of data packets, thereby achieving a more continuous and smooth real-time voice experience in encrypted communication scenarios. At the same time, it is provided in the form of encapsulated modules, which facilitates application integration and eliminates the need to concern oneself with the underlying encryption / decryption and QoS coordination details.

[0077] See Figure 6 As shown in the embodiments of this application, a voice QoS optimization and playback device for end-to-end source encryption is also disclosed, comprising:

[0078] Data receiving module 11 is used to receive encrypted voice data packets encrypted from end-to-end source encryption and store the encrypted voice data packets in ciphertext form into a jitter buffer.

[0079] The water level determination module 12 is used to determine the current target buffer water level of the jitter buffer based on the current network jitter level and the delay factor introduced by the current decryption process; wherein, the network jitter level is used to reflect the fluctuation of network transmission delay, and the delay factor introduced by the decryption process includes the average decryption time and the decryption time jitter level of the encrypted voice data packets within a preset time period.

[0080] The strategy generation module 13 is used to generate a playback control strategy and a decryption scheduling strategy based on the current actual buffer level of the jitter buffer and the current target buffer level; wherein, the decryption scheduling strategy includes the frequency, triggering timing and execution priority of decryption operations, and adjusts the decryption scheduling strategy based on the real-time playback control strategy and jitter buffer status;

[0081] The decryption and playback module 14 is used to schedule corresponding encrypted voice data packets from the jitter buffer for decryption using the decryption scheduling strategy, and to decode and play the decrypted voice data packets according to the playback control strategy.

[0082] In some specific embodiments, the data receiving module 11 includes:

[0083] The data receiving unit is used to receive encrypted voice data packets from an end-to-end encrypted source and to perform validity checks on the encrypted voice data packets; the validity checks include determining whether the data packets arrived after a timeout and whether the data packets are duplicate data packets;

[0084] The data caching unit is used to determine the encrypted voice data packet as invalid and discard it directly if the encrypted voice data packet is detected to have arrived after a timeout or is a duplicate data packet; and to determine the encrypted voice data packet as valid and store it directly in the jitter buffer if the encrypted voice data packet is detected to have arrived before a timeout and is a non-duplicate data packet.

[0085] In some specific embodiments, the water level determination module 12 includes:

[0086] The formula construction unit is used to construct a target formula based on the basic buffer level corresponding to the jitter buffer, the degree of network jitter, the average decryption time and the degree of decryption time jitter, the security factor and the security margin of the encrypted voice data packets within a preset time period.

[0087] Wherein, the basic buffer level is a pre-set default value that meets the preset playback smoothness standard; the safety factor is a pre-set weight value used to amplify the compensation intensity of decryption time jitter; the safety margin is an additional buffer reserve value used to cope with sudden decryption delays or network fluctuations; and the target formula is:

[0088] Target buffer level = Base buffer level + Network jitter level + (Average decryption time + Security factor × Decryption time jitter level) + Security margin;

[0089] The first water level determination unit is used to determine the current target buffer water level of the jitter buffer using the target formula.

[0090] In some specific embodiments, the strategy generation module 13 includes:

[0091] The comparison unit is used to compare the current actual buffer level of the jitter buffer with the current target buffer level;

[0092] The first strategy generation unit is used to generate a preset accelerated playback strategy if the current actual buffer level is higher than the current target buffer level.

[0093] The second strategy generation unit is used to generate a preset default playback strategy if the current actual buffer level is equal to the current target buffer level.

[0094] The third strategy generation unit is used to generate a preset deceleration playback strategy if the current actual buffer level is lower than the current target buffer level.

[0095] The fourth strategy generation unit is used to generate a preset packet loss compensation strategy to maintain the continuity of voice if packet loss is detected and the current actual buffer level is insufficient to make up for the playback gap caused by packet loss.

[0096] The fifth strategy generation unit is used to generate a preset fusion processing strategy for smoothly connecting compensation data and normal data if normal data packets are received again after the packet loss compensation strategy is executed.

[0097] In some specific implementations, the execution priority includes the scheduling priority of the decryption task in the overall system computing resources, and / or the processing order priority of each encrypted voice data packet within the decryption task.

[0098] In some specific embodiments, the strategy generation module 13 includes:

[0099] The first processing unit is configured to adjust the frequency of the decryption operation to the preset highest frequency, advance the triggering time of the decryption operation, and adjust the scheduling priority of the decryption task in the overall computing resources of the system to the first priority that meets the preset high priority standard if the current playback control strategy is the preset accelerated playback strategy and the current actual buffer level of the jitter buffer is higher than the first preset water level threshold or the data packet accumulation rate exceeds the preset accumulation rate threshold.

[0100] The second processing unit is used to adjust the frequency of the decryption operation to a preset base frequency if the current playback control strategy is the preset default playback strategy and the current actual buffer level of the jitter buffer is within the preset normal level range, determine the triggering time of the decryption operation according to the receiving order of the data packets and the expected playback timestamp, and adjust the scheduling priority of the decryption task in the overall computing resources of the system to the second priority that meets the preset priority standard.

[0101] The third processing unit is used to adjust the frequency of the decryption operation to a preset proportion of the preset reference frequency, delay the triggering time of the decryption operation, and adjust the scheduling priority of the decryption task in the overall computing resources of the system to the third priority that meets the preset low priority standard if the current playback control strategy is a preset deceleration playback strategy and the current actual buffer level of the jitter buffer is lower than the second preset level threshold.

[0102] In some specific embodiments, the decryption and playback module 14 further includes:

[0103] The log generation unit is used to record the actual time taken for each encrypted voice data packet from the start of decryption to the completion of decryption in real time, and generate a decryption time log.

[0104] The data determination unit is used to determine a new average decryption time and decryption time jitter based on the decryption time log when a preset statistical period is reached or the cumulative number of decrypted data packets reaches a preset threshold, and to re-count the cumulative number of decrypted data packets.

[0105] The second water level determination unit is used to feed back the new average decryption time and the degree of decryption time jitter to the step of determining the current target buffer water level of the jitter buffer, so as to obtain the new current target buffer water level.

[0106] In some specific embodiments, the decryption and playback module 14 further includes:

[0107] The strategy update unit is used to determine the new current actual buffer level corresponding to the jitter buffer, and feed back the new current actual buffer level to the step of generating a playback control strategy and a decryption scheduling strategy based on the current actual buffer level of the jitter buffer and the current target buffer level, so as to obtain a new playback control strategy and a decryption scheduling strategy.

[0108] Furthermore, embodiments of this application also disclose an electronic device, Figure 7 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0109] Figure 7This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the voice QoS optimization and playback method for end-to-end source encryption disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be a computer.

[0110] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0111] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0112] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the voice QoS optimization and playback method for end-to-end source encryption disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0113] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned disclosed voice QoS optimization and playback method oriented towards end-to-end source encryption. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0114] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0115] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0116] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0117] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0118] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A voice QoS optimization and playing method for end-to-end source encryption, characterized in that, The method comprises the following steps: receiving encrypted voice data packets encrypted by an end-to-end source, and storing the encrypted voice data packets in a jitter buffer in ciphertext form; determining a current target buffer water level of the jitter buffer based on a current network jitter degree and a delay factor introduced by current decryption processing, wherein the network jitter degree is used to reflect the fluctuation degree of network transmission delay, and the delay factor introduced by decryption processing includes a decryption average time consumption and a decryption time consumption jitter degree of encrypted voice data packets completing decryption within a preset time period; generating a playback control strategy and a decryption scheduling strategy according to a current actual buffer water level of the jitter buffer and the current target buffer water level, wherein the decryption scheduling strategy includes a frequency, a triggering time and an execution priority of a decryption operation, and the decryption scheduling strategy is adjusted based on a real-time playback control strategy and a jitter buffer state; decrypting corresponding encrypted voice data packets from the jitter buffer by using the decryption scheduling strategy, and decoding and playing the decrypted voice data packets according to the playback control strategy.

2. The voice QoS optimization and play-out method for end-to-end source encryption of claim 1, wherein, The method of receiving encrypted voice data packets encrypted by an end-to-end source, and storing the encrypted voice data packets in a jitter buffer in ciphertext form comprises the following steps: receiving encrypted voice data packets encrypted by an end-to-end source, and performing validity detection on the encrypted voice data packets, wherein the validity detection includes judging whether the data packets arrive overtime and whether the data packets are duplicate data packets; if it is detected that the encrypted voice data packets arrive overtime or are duplicate data packets, the encrypted voice data packets are determined as invalid data packets and are discarded directly, and if it is detected that the encrypted voice data packets do not arrive overtime and are not duplicate data packets, the encrypted voice data packets are determined as valid data packets and are stored in the jitter buffer directly.

3. The voice QoS optimization and play-out method for end-to-end source encryption of claim 1, wherein, The method of determining a current target buffer water level of the jitter buffer based on a current network jitter degree and a delay factor introduced by current decryption processing comprises the following steps: constructing a target formula based on a basic buffer water level corresponding to the jitter buffer, the network jitter degree, a decryption average time consumption and a decryption time consumption jitter degree of encrypted voice data packets completing decryption within a preset time period, a security coefficient and a security margin, wherein the basic buffer water level is a default value pre-set to meet a preset playback fluency standard, the security coefficient is a weight value pre-set to amplify the compensation strength of the decryption time consumption jitter degree, the security margin is an additional buffer reserved value for coping with sudden decryption delay or network fluctuation, and the target formula is: target buffer water level = basic buffer water level + network jitter degree + (decryption average time consumption + security coefficient × decryption time consumption jitter degree) + security margin; determining the current target buffer water level of the jitter buffer by using the target formula. The method of generating a playback control strategy according to a current actual buffer water level of the jitter buffer and the current target buffer water level comprises the following steps:

4. The voice QoS optimization and play-out method for end-to-end source encryption of claim 1, wherein, comparing the current actual buffer water level of the jitter buffer with the current target buffer water level; if the current actual buffer water level is higher than the current target buffer water level, a preset accelerated playback strategy is generated. ​ generating a preset default playing strategy if the current actual buffer level is equal to the current target buffer level; generating a preset decelerated playing strategy if the current actual buffer level is lower than the current target buffer level; generating a preset packet loss compensation strategy for maintaining voice continuity if packet loss is detected and the current actual buffer level is insufficient to compensate for the playing vacancy caused by the packet loss; generating a preset fusion processing strategy for smoothly connecting the compensation data and the normal data if normal data packets are received again after the packet loss compensation strategy is executed.

5. The voice QoS optimization and play-out method for end-to-end source encryption of claim 4, wherein, The execution priority includes a scheduling priority of the decryption task in the overall computing resources of the system and / or a processing order priority of each encrypted voice data packet in the decryption task; Correspondingly, the decryption scheduling strategy is adjusted based on the real-time playing control strategy and the jitter buffer state, including: if the current playing control strategy is a preset accelerated playing strategy and the current actual buffer level of the jitter buffer is higher than a first preset water level threshold or a data packet accumulation rate exceeds a preset accumulation rate threshold, adjusting the frequency of the decryption operation to a preset highest frequency, advancing the triggering time of the decryption operation, and adjusting the scheduling priority of the decryption task in the overall computing resources of the system to a first priority that meets a preset high priority standard; if the current playing control strategy is a preset default playing strategy and the current actual buffer level of the jitter buffer is within a preset normal water level range, adjusting the frequency of the decryption operation to a preset reference frequency, determining the triggering time of the decryption operation according to the receiving order and the expected playing time stamp of the data packet, and adjusting the scheduling priority of the decryption task in the overall computing resources of the system to a second priority that meets a preset medium priority standard; if the current playing control strategy is a preset decelerated playing strategy and the current actual buffer level of the jitter buffer is lower than a second preset water level threshold, adjusting the frequency of the decryption operation to a preset proportion of the preset reference frequency, delaying the triggering time of the decryption operation, and adjusting the scheduling priority of the decryption task in the overall computing resources of the system to a third priority that meets a preset low priority standard.

6. The voice QoS optimization and play-out method for end-to-end source encryption of claim 1, wherein, The process of scheduling corresponding encrypted voice data packets from the jitter buffer for decryption using the decryption scheduling strategy further includes: real-time recording of the actual time consumption of each encrypted voice data packet from starting decryption to completing decryption, and generating a decryption time consumption log; when a preset statistical period is reached or the cumulative number of completed decryption data packets reaches a preset threshold, determining a new decryption average time consumption and a decryption time consumption jitter degree based on the decryption time consumption log, and re-counting the cumulative number of completed decryption data packets; feeding the new decryption average time consumption and the decryption time consumption jitter degree back to the step of determining the current target buffer level of the jitter buffer to obtain a new current target buffer level.

7. The voice QoS optimization and play-out method for end-to-end source encryption, according to claim 1, characterized by, After the decoded voice data packets are processed according to the playing control strategy, the process further includes: determine a new current actual buffer level corresponding to the jitter buffer, and feed back the new current actual buffer level to a step of generating a play control strategy and a decryption scheduling strategy according to the current actual buffer level of the jitter buffer and the current target buffer level, to obtain a new play control strategy and a new decryption scheduling strategy.

8. A voice QoS optimization and play-out apparatus oriented to end-to-end source encryption, characterized by The method comprises the steps of: a data receiving module, configured to receive an end-to-end source encryption encrypted voice data packet, and store the encrypted voice data packet in a jitter buffer in ciphertext form; a water level determining module, configured to determine a current target buffer level of the jitter buffer based on a current network jitter degree and a decryption processing introduced delay factor; wherein the network jitter degree is used to reflect a fluctuation degree of network transmission delay, and the decryption processing introduced delay factor comprises a decryption average time consumption and a decryption time consumption jitter degree of the encrypted voice data packet completing decryption within a preset time period; a strategy generating module, configured to generate a play control strategy and a decryption scheduling strategy according to the current actual buffer level of the jitter buffer and the current target buffer level; wherein the decryption scheduling strategy comprises a frequency, a triggering time and an execution priority of a decryption operation, and the decryption scheduling strategy is adjusted based on a real-time play control strategy and a jitter buffer state; a decryption and play module, configured to schedule corresponding encrypted voice data packets from the jitter buffer for decryption by using the decryption scheduling strategy, and decode and play the decrypted voice data packet according to the play control strategy.

9. An electronic device, comprising: The method comprises the steps of: a memory, configured to save a computer program; a processor, configured to execute the computer program to realize the end-to-end source encryption oriented voice QoS optimization and play method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is saved, and the computer program is executed by a processor to realize the end-to-end source encryption oriented voice QoS optimization and play method according to any one of claims 1 to 7.

Images