An adaptive speech semantic communication method based on hierarchical temporal importance

Abstract

The application relates to an adaptive voice semantic communication method based on hierarchical time sequence importance, and belongs to the voice analysis field, which comprises the following steps: a sending end divides a discrete feature matrix according to the priority of voice features and time sequence attributes, and filters the features according to the importance score in the feature matrix encapsulation and selection link; a channel adaptive scheduling module is interacted with wireless communication, a channel feedback mechanism is introduced, the current channel state information is perceived, and the transmission power and resource allocation strategy are dynamically adjusted; after signal demodulation is completed, a receiving end inputs the obtained sparse feature stream to a voice repair module, and global reconstruction is performed on the received features by using deep pre-trained voice prior knowledge. The application combines a hierarchical voice feature extraction technology based on a discrete codebook and a generative semantic repair technology, so that the voice word error rate in a severe channel environment is maximally reduced, and the semantic intelligibility of a receiving end is improved.

Description

Technical Field

[0001] This invention relates to the field of speech analysis, and more particularly to an adaptive speech-semantic communication method based on hierarchical temporal importance. Background Technology

[0002] Faced with the urgent need for highly reliable and low-latency voice interaction in future communication systems, sixth-generation mobile networks (6G) need to achieve higher-quality semantic transmission under limited spectrum resources. Semantic communication, by utilizing artificial intelligence technology to extract deep semantic features from raw speech, can maintain the effectiveness of information while reducing data volume. Currently, speech editing technology based on discrete codebooks (such as SpeechTokenizer) has become a research hotspot in this field due to its high compression rate and anti-interference potential. However, existing speech semantic communication technologies still face the following significant problems in practical applications: 1. Lack of feature importance assessment: Existing technologies usually treat the multi-level discrete features (tokens) extracted by neural networks as equally important information flows; in fact, the semantic content (Semantics) and acoustic details (Acoustics) of speech are distributed at different feature levels, and there is a distinction between skeleton and redundancy on the time axis. Existing methods lack a fine distinction of this dual importance of hierarchy and time sequence, resulting in the average allocation of channel resources and the inability to prioritize the protection of core information. 2. Insufficient robustness under extreme channel conditions: In environments with extremely low signal-to-noise ratios (e.g., SNR ≤ -15dB), traditional uniform protection strategies result in all data packets being contaminated by noise. The decoded speech often contains a large amount of noise or is even completely incomprehensible. Existing methods lack a channel-aware, non-uniform protection mechanism to ensure the survival of the core framework. 3. Limited receiver-side repair mechanisms: Existing semantic receivers mostly rely on deterministic decoding algorithms. When some non-critical features are lost during transmission, the system cannot utilize the prior knowledge of generative models to "fill in" the missing acoustic details, leading to a sharp decline in the perceived quality of the reconstructed speech under adverse channel conditions. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide an adaptive speech-semantic communication method based on hierarchical temporal importance, thus solving the deficiencies of the prior art.

[0004] The objective of this invention is achieved through the following technical solution: an adaptive speech-semantic communication method based on hierarchical temporal importance, the communication method comprising:

[0005] Step 1: The sending end prioritizes the discrete feature matrix according to the hierarchy and temporal attributes of the speech features. In the feature matrix encapsulation and selection stage, features are filtered according to importance scores.

[0006] Step 2: The filtered feature packets enter the physical layer and interact with wireless communication through the channel adaptive scheduling module. A channel feedback mechanism is introduced to dynamically adjust the transmission power and resource allocation strategy by sensing the current channel state information.

[0007] Step 3: After completing signal demodulation, the receiving end inputs the acquired sparse feature stream into the speech restoration module and uses the deep pre-trained speech prior knowledge to globally reconstruct the received features.

[0008] The filtering in step one includes: core features with the highest priority are encapsulated into a transmission queue with enhanced protection, while less important detailed features are masked or discarded according to preset rules or real-time channel conditions, thereby achieving semantic-level adaptive content compression at the source.

[0009] The priority division of the discrete feature matrix in step one includes: a two-dimensional importance assessment of hierarchical and temporal dimensions, as well as a priority mapping and discarding strategy.

[0010] The hierarchical importance assessment includes: using the semantic evaluation module to analyze the 8 quantization layers in the discrete feature matrix extracted from the discrete codebook, measuring the contribution of each layer to the semantic reconstruction of the original speech; based on the contribution score, identifying the 0th layer carrying the core semantic information as the semantic base, dividing the 0th layer into the first priority region to ensure that it has the highest protection level during transmission, serving as the basis for subsequent speech reconstruction.

[0011] The temporal dimension importance assessment includes: in the temporal dimension, introducing a key temporal frame sampling mechanism, filtering the temporal feature columns by setting a fixed sampling step size K to obtain key temporal frames; defining the feature columns that meet the sampling conditions as the temporal skeleton, and dividing the key temporal frames into second priority regions in the feature layers other than the 0th layer.

[0012] The priority mapping and discarding strategy includes mapping the evaluation results of hierarchical dimension importance assessment and temporal dimension importance assessment to a unified importance weight map, which includes a core protection area and a flexible detail area.

[0013] The core protection zone consists of a first priority area and a second priority area. During resource allocation, priority is given to ensuring the complete transmission of the multi-level discrete features of the core protection zone.

[0014] The elastic detail region includes a third priority region. When the wireless channel environment is harsh or the bandwidth is limited, the multi-level discrete features of this region are actively discarded according to adaptive rules. The resulting acoustic detail gaps are then generatively filled at the receiver by the speech restoration module in combination with the semantic base and the temporal skeleton, thereby achieving high-quality communication.

[0015] Step three is specifically achieved through several steps, including the construction of a comprehensive importance scoring model, a channel-aware adaptive selection strategy, dynamic threshold mapping, mask generation and sparse transmission, and optimization of the objective function for generative reconstruction.

[0016] The construction of the comprehensive importance scoring model includes:

[0017] Let the discrete feature matrix extracted from the discrete codebook be... ,in For the number of levels, For any feature unit in the matrix, the number of time-series frames is given. Its overall importance score Defined as a linear weighted combination of hierarchical semantic weights and temporal structure weights, I(l,t) = α·H(l) + β·T(t), where H(l) is the hierarchical importance decay function. For time-importance sampling functions, These are weighting coefficients. Indicates the first layer. Comprehensive transmission priority score for frame feature units;

[0018] Considering that layer 0 carries core semantic information, while subsequent layers carry acoustic detail information, a piecewise exponential model is used to model the layer weights as H(l) = I(l=0) + ηe -μl • (1 - I(l=0)) to ensure that the semantic base has the highest global basic weight, where, For indicator functions, The hierarchical attenuation coefficient is... This is the scaling factor for the detail layer;

[0019] To preserve the skeleton frames that maintain the prosodic structure of speech in the time domain, a discrete combinator function is used. When t is an integer multiple of K, This frame is identified as a critical timing frame. For Kronek function, This is the preset grid sampling step size.

[0020] The channel-aware adaptive selection strategy includes: extracting the signal-to-noise ratio parameter based on the channel state information fed back by the physical layer in real time. And available bandwidth B, a dynamic truncation threshold is constructed based on channel state. And generate a binary transmission mask matrix. ;

[0021] The dynamic threshold mapping includes: in order to achieve smooth adaptive control, [the following steps are taken]... Set as , and These correspond to the threshold boundaries under adverse and ideal channel conditions, respectively. This is the sensitivity coefficient. The center offset is used as the mapping relationship, which enables the system to automatically raise the decision threshold in low signal-to-noise ratio environments and retain only high-priority features.

[0022] The mask generation and sparse transmission include: transmission mask matrix. Through the unit step function Generate, obtain The sparse feature matrix that finally enters the physical layer transmission queue The Hadamard product of the original matrix and the mask matrix, i.e. ;

[0023] The objective function optimization of the generative reconstruction includes: at the receiving end, the speech restoration module is configured to solve for the optimal solution under constraints, assuming... For parameters Deep generative networks to reconstruct speech Its training and inference processes aim to minimize the multi-objective joint loss function. ,Right now ,in, For semantic consistency loss, To combat perceptual loss.

[0024] The present invention has the following advantages: an adaptive speech semantic communication method based on hierarchical temporal importance, which combines hierarchical speech feature extraction technology based on discrete codebook and generative semantic repair technology, considers communication scenarios with extremely low signal-to-noise ratio, and designs a channel adaptive weight allocation strategy and hybrid decoding algorithm based on the dual importance of hierarchy and temporal order, so as to minimize the speech word error rate in harsh channel environments and improve the semantic intelligibility of the receiver. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the architecture of the present invention;

[0026] Figure 2 A schematic diagram illustrating a two-dimensional feature importance selection strategy that integrates hierarchical semantic evaluation and temporal grid sampling;

[0027] Figure 3 This is a schematic diagram comparing the end-to-end voice intelligibility (WER) of the present invention and the MP3 scheme under different packet loss rates in a packet loss channel.

[0028] Figure 4This diagram illustrates the comparison of end-to-end voice perceived quality (DNSMOS) of the present invention and the MP3 scheme under different packet loss rates in a packet loss channel.

[0029] Figure 5 This is a schematic diagram comparing the end-to-end speech intelligibility (WER) of the present invention and the Encodec scheme under different signal-to-noise ratios in an additive white Gaussian noise channel.

[0030] Figure 6 This diagram illustrates the comparison of end-to-end speech perception quality (DNSMOS) of the present invention and the Encodec scheme under different signal-to-noise ratios in an additive white Gaussian noise channel. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application provided below with reference to the accompanying drawings is not intended to limit the scope of protection of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application. The present invention will be further described below with reference to the accompanying drawings.

[0032] like Figure 1 As shown, this invention specifically relates to an adaptive speech semantic communication method based on hierarchical temporal importance. It proposes an end-to-end, importance-aware semantic communication architecture. By introducing a spatiotemporal dual importance modeling module at the sending end and an importance-based resource allocation module at the physical layer, efficient semantic communication is achieved. Specifically, it consists of three parts: the sending end, the physical layer, and the receiving end.

[0033] The transmitting end processes the following: First, the raw speech signal passes through a coding layer and a two-dimensional importance evaluation layer. Based on the hierarchical and temporal attributes of the speech features, the discrete feature matrix is ​​prioritized. In the feature matrix encapsulation and selection stage, the system filters features based on their importance scores. The core features with the highest priority (i.e., the first and second priority regions) are encapsulated into an enhanced protection transmission queue; while less important (third priority region) detailed features are masked or discarded according to preset rules or real-time channel conditions, thus achieving semantic-level adaptive content compression at the source.

[0034] Physical Layer Transmission: The filtered feature packets enter the physical layer and interact with the wireless channel through the channel adaptive scheduling module. A channel feedback mechanism is introduced to dynamically adjust transmission power and resource allocation strategies by sensing the current channel state information, ensuring the reliability of core semantic stream transmission in the wireless environment.

[0035] Receiver reconstruction: After demodulation, the receiver inputs the acquired sparse feature stream to the speech restoration module. This speech restoration module is always active, utilizing deep pre-trained speech prior knowledge to globally reconstruct the received features.

[0036] Even in environments with extremely low signal-to-noise ratios, the speech restoration module can still utilize the remaining core semantic base and temporal skeleton to fill in the lost speech details through generative reasoning, ensuring the high clarity and intelligibility of the output speech.

[0037] In a favorable channel environment, the speech restoration module combines more complete feature information to further improve the naturalness and fidelity of the reconstructed speech.

[0038] Furthermore, such as Figure 2 As shown, in order to achieve fine-grained management of speech features, this invention designs a dual evaluation mechanism that operates in both hierarchical and temporal parallel modes, transforming the discrete feature matrix into a multi-level priority system, specifically including the following:

[0039] Hierarchical Importance Assessment: The semantic evaluation module analyzes the eight quantization layers in the discrete feature matrix extracted from the discrete codebook, measuring the contribution of each layer to the semantic reconstruction of the original speech. Semantic Base Locking: Based on the contribution score, Layer 0, which carries the core semantic information, is identified as the semantic base. Layer 0 is then designated as the first priority region (i.e., Figure 2 The uppermost region ensures the highest level of protection during transmission, serving as the basis for subsequent voice reconstruction.

[0040] Temporal Importance Assessment: In the temporal dimension, a key temporal frame sampling mechanism (Grid) is introduced. By setting a fixed sampling step size K, feature columns in the temporal dimension are filtered. Temporal Skeleton Locking: Feature columns that meet the sampling conditions are defined as the temporal skeleton. In feature layers other than Layer 0, these key temporal frames are divided into second-priority regions (i.e., specific column regions in Figure 2). These key temporal frames provide alignment anchors on the time axis for the speech restoration module at the receiver, maintaining the rhythm and coherence of the reconstructed speech.

[0041] Priority mapping and discarding strategy: The evaluation results of the above two dimensions are mapped to a unified importance weight map (as shown on the right side of Figure 2):

[0042] Core Protection Zone: As indicated by the curly braces in Figure 2, it consists of the first priority zone and the second priority zone. During resource scheduling, priority is given to ensuring the complete transmission of the token (discrete feature code) in this zone.

[0043] The flexible detail region, also known as the third priority region, is the blank grid area in Figure 2 excluding the first and second priority regions. In harsh wireless channel environments or when bandwidth is limited, the system proactively discards tokens in this region according to adaptive rules. The resulting acoustic detail gaps are then generatively filled at the receiver by the voice restoration module, combining the semantic base and temporal skeleton, thereby achieving high-quality communication at extremely low bit rates.

[0044] Furthermore, this method proposes a mathematical model based on the decoupling of hierarchical and temporal features to quantify the transmission priority of feature units and guide adaptive channel selection and receiver reconstruction, specifically including the following:

[0045] Construction of the comprehensive importance scoring model: Let the discrete feature matrix extracted by SpeechTokenizer be... ,in For the number of levels, This represents the number of time-series frames. For any feature unit in the matrix... (in ), its overall importance score Defined as a linear weighted combination of hierarchical semantic weights and temporal structure weights:

[0046] I(l,t)=α·H(l)+β·T(t),

[0047] in, This is the hierarchy importance decay function. Indicates the first layer. The overall transmission priority score of frame feature units.

[0048] Considering that layer 0 carries core semantic information, while subsequent layers carry acoustic detail information, this embodiment uses a piecewise exponential model to model the layer weights:

[0049] H(l) = I(l=0) + ηe -μl ·(1- I(l=0))

[0050] in, For an indicator function, if and only if The value is 1 at this time. This refers to the hierarchical attenuation coefficient. This is the scaling factor for the detail layer. This function ensures that the semantic base (Layer 0) has the highest global base weight.

[0051] For temporal importance sampling functions, in order to preserve the skeleton frames that maintain the prosodic structure of speech in the time domain, a discrete comb function is used:

[0052] ,

[0053] in, For Kronek function, This is the preset grid sampling step size. When for When it is an integer multiple of, This frame is identified as a critical timing frame. These are weighting coefficients. To ensure strict ordering of priorities, constraints are set. This establishes that the semantic base has a higher priority than the temporal skeleton, and the temporal skeleton has a higher priority than the transmission layer of acoustic details.

[0054] Channel-aware adaptive selection strategy: Extract signal-to-noise ratio parameters based on channel state information (CSI) fed back from the physical layer in real time. (SNR) and available bandwidth Based on channel state, a dynamic truncation threshold is constructed. And generate a binary transmission mask matrix. .

[0055] Dynamic threshold mapping: To achieve smooth adaptive control, the following is implemented: Set as:

[0056] ,

[0057] and These correspond to the threshold boundaries under adverse and ideal channel conditions, respectively. This is the sensitivity coefficient. The center offset is used as the mapping relationship, which enables the system to automatically raise the decision threshold in low signal-to-noise ratio environments and retain only high-priority features.

[0058] Mask generation and sparse transmission: Transmission mask matrix Through the unit step function generate:

[0059] ,

[0060] The sparse feature matrix that finally enters the physical layer transmission queue The Hadamard product of the original matrix and the mask matrix is:

[0061] ,

[0062] This part implements joint source-channel coding at the feature level, that is, when channel capacity is limited, actively discarding components with importance scores lower than [a certain value]. Redundant feature units.

[0063] Generative reconstruction objective function optimization: At the receiver, the speech restoration module is configured to solve for the optimal solution under constraints. Let... For parameters Deep generative networks to reconstruct speech Its training and inference processes aim to minimize the following multi-objective joint loss function. :

[0064] ,

[0065] in, This is the semantic consistency loss. It is used to constrain the reconstructed speech to be semantically consistent with the original speech in the frequency domain. To maintain consistency, it is defined as the Mel-Spectrogram space. distance:

[0066] ,

[0067] This constraint uses the remaining Layer 0 features to lock in the accuracy of the speech content.

[0068] To counteract perceptual loss, a discriminator loss form from a Generative Adversarial Network (GAN) is employed to recover high-frequency acoustic details lost due to feature discarding.

[0069] ,

[0070] in As a discriminator network, this constraint drives the generator network to infer the lost acoustic texture using prior distributions, thereby improving the subjective listening quality of the reconstructed speech.

[0071] like Figure 3 The diagram illustrates the comparison of speech intelligibility (WER) between the present invention and the MP3 solution under different packet loss rates. The horizontal axis represents the packet loss rate (e.g., increasing from 0% to 20%), and the vertical axis represents the WER value. Figure 3As can be seen, the WER of the MP3 scheme increases sharply with the increase of packet loss rate, indicating that its speech intelligibility drops significantly under poor network conditions. This is because traditional source coding is extremely sensitive to packet loss; the loss of some data can lead to decoding failure or severe distortion. In contrast, the WER curve of this invention changes more gradually with the increase of packet loss rate, remaining at a low level. This indicates that this invention utilizes semantic features for transmission, and through a joint source-channel coding mechanism, it enables semantic information to have better distribution characteristics in the channel. Even if partial packet loss occurs, the receiver can still recover highly intelligible speech through contextual semantics.

[0072] like Figure 4 As shown, the comparison results of the present invention and the MP3 solution in terms of speech perception quality are presented under different packet loss rates. Figure 4 As shown, at low packet loss rates, the scores of the two solutions are quite similar; however, as the packet loss rate increases, the DNSMOS score of the MP3 solution drops significantly, and the listening quality deteriorates noticeably. In contrast, the DNSMOS score of this invention decreases slowly, maintaining a high speech quality score even at high packet loss rates (e.g., 15%-20%). Figure 3 and Figure 4 It can be seen that the present invention has significantly better robustness than the MP3 scheme under packet loss channel conditions.

[0073] like Figure 5 The figure shows a comparison of the speech intelligibility (WER) of the present invention and the Encoder scheme under different signal-to-noise ratio (SNR) conditions. The horizontal axis represents the signal-to-noise ratio (dB), and the vertical axis represents the WER. From Figure 5 As can be seen, in low signal-to-noise ratio (i.e., high noise) environments, the WER of the Encoder scheme is extremely high, making it almost impossible to decode intelligible speech. This is a typical "cliff effect" in digital communication systems, where performance drops precipitously when channel quality falls below a certain threshold. However, the WER curve of this invention remains low even under low signal-to-noise ratio conditions, without exhibiting a significant cliff effect, demonstrating that analog semantic communication methods are better suited to harsh channel environments.

[0074] like Figure 6 As shown, the comparison results of the present invention and the Encoder scheme in terms of speech perception quality (DNSMOS) are presented under different signal-to-noise ratios. Figure 6As shown, the speech quality of the Encoder degrades rapidly with decreasing signal-to-noise ratio (SNR), while the present invention exhibits a more graceful degradation characteristic. Particularly in the low SNR region, the DNSMOS score of the present invention is significantly higher than that of the Encoder scheme. This further verifies that the present invention can effectively utilize channel bandwidth in low SNR environments, recovering a more natural and clearer speech signal than traditional digital coding schemes.

[0075] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and improvements, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. An adaptive speech-semantic communication method based on hierarchical temporal importance, characterized in that: The communication method includes: Step 1: The sending end prioritizes the discrete feature matrix according to the hierarchy and temporal attributes of the speech features. In the feature matrix encapsulation and selection stage, features are filtered according to importance scores. Step 2: The filtered feature packets enter the physical layer and interact with wireless communication through the channel adaptive scheduling module. A channel feedback mechanism is introduced to dynamically adjust the transmission power and resource allocation strategy by sensing the current channel state information. Step 3: After the receiver completes signal demodulation, it inputs the acquired sparse feature stream into the speech restoration module and uses the deep pre-trained speech prior knowledge to globally reconstruct the received features. The priority division of the discrete feature matrix in step one includes: a two-dimensional importance assessment of hierarchical dimension importance assessment and temporal dimension importance assessment, as well as a priority mapping and discarding strategy; The hierarchical importance assessment includes: using the semantic assessment module to analyze the 8 quantization layers in the discrete feature matrix extracted from the discrete codebook, measuring the contribution of each layer to the semantic reconstruction of the original speech; based on the contribution score, identifying the 0th layer carrying the core semantic information as the semantic base, dividing the 0th layer into the first priority region to ensure that it has the highest protection level during transmission, serving as the basis for subsequent speech reconstruction. The temporal dimension importance assessment includes: in the temporal dimension, introducing a key temporal frame sampling mechanism, filtering the feature columns in the temporal dimension by setting a fixed sampling step size K to obtain key temporal frames; defining the feature columns that meet the sampling conditions as the temporal skeleton, and dividing the key temporal frames into second priority regions in the feature layers other than the 0th layer; The priority mapping and discarding strategy includes mapping the evaluation results of hierarchical dimension importance assessment and temporal dimension importance assessment to a unified importance weight map, which includes a core protection area and a flexible detail area. The core protection zone consists of a first priority area and a second priority area. During resource allocation, priority is given to ensuring the complete transmission of the multi-level discrete features of the core protection zone. The elastic detail region includes a third priority region. When the wireless channel environment is harsh or the bandwidth is limited, the multi-level discrete features of this region are actively discarded according to adaptive rules. The resulting acoustic detail gaps are then generatively filled at the receiver by the speech restoration module in combination with the semantic base and the temporal skeleton, thereby achieving high-quality communication.

2. The adaptive speech-semantic communication method based on hierarchical temporal importance according to claim 1, characterized in that: The filtering in step one includes: core features with the highest priority are encapsulated into a transmission queue with enhanced protection, while less important detailed features are masked or discarded according to preset rules or real-time channel conditions, thereby achieving semantic-level adaptive content compression at the source.

3. The adaptive speech-semantic communication method based on hierarchical temporal importance according to claim 1, characterized in that: Step three is specifically achieved through several steps, including the construction of a comprehensive importance scoring model, a channel-aware adaptive selection strategy, dynamic threshold mapping, mask generation and sparse transmission, and optimization of the objective function for generative reconstruction.

4. The adaptive speech-semantic communication method based on hierarchical temporal importance according to claim 3, characterized in that: The construction of the comprehensive importance scoring model includes: Let the discrete feature matrix extracted from the discrete codebook be... ,in For the number of levels, For any feature unit in the matrix, the number of time-series frames is given. Its overall importance score Defined as a linear weighted combination of hierarchical semantic weights and temporal structure weights, I(l,t) = α·H(l) + β·T(t), where H(l) is the hierarchical importance decay function. For timing importance sampling function, These are weighting coefficients. Indicates the first layer. Comprehensive transmission priority score for frame feature units; Considering that layer 0 carries core semantic information, while subsequent layers carry acoustic detail information, a piecewise exponential model is used to model the layer weights as H(l) = I(l=0) + ηe -μl • (1 - I(l=0)) to ensure that the semantic base has the highest global basic weight, where, For indicator functions, The hierarchical attenuation coefficient is... This is the scaling factor for the detail layer; To preserve the skeleton frames that maintain the prosodic structure of speech in the time domain, a discrete combinator function is used. When t is an integer multiple of K, This frame is identified as a critical timing frame. For Kronek function, This is the preset grid sampling step size.

5. The adaptive speech-semantic communication method based on hierarchical temporal importance according to claim 4, characterized in that: The channel-aware adaptive selection strategy includes: extracting the signal-to-noise ratio parameter based on the channel state information fed back by the physical layer in real time. And available bandwidth B, a dynamic truncation threshold is constructed based on channel state. And generate a binary transmission mask matrix. ; The dynamic threshold mapping includes: in order to achieve smooth adaptive control, [the following steps are taken]... Set as , and These correspond to the threshold boundaries under adverse and ideal channel conditions, respectively. This is the sensitivity coefficient. The center offset is used as the mapping relationship, which enables the system to automatically raise the decision threshold in low signal-to-noise ratio environments and retain only high-priority features.

6. The adaptive speech-semantic communication method based on hierarchical temporal importance according to claim 5, characterized in that: The mask generation and sparse transmission include: transmission mask matrix. Through the unit step function Generate, obtain The sparse feature matrix that finally enters the physical layer transmission queue The Hadamard product of the original matrix and the mask matrix, i.e. ; The objective function optimization of the generative reconstruction includes: at the receiving end, the speech restoration module is configured to solve for the optimal solution under constraints, assuming... For parameters Deep generative networks to reconstruct speech Its training and inference processes aim to minimize the multi-objective joint loss function. ,Right now ,in, For semantic consistency loss, To combat perceptual loss.

Images