Modularized semantic communication network construction method and system

Abstract

The invention discloses a modular semantic communication network construction method and system, and aims to solve the problems that an existing semantic communication method is high in coupling, poor in universality, difficult to adapt to multi-user requirements and the like. The method comprises the steps that a multi-code-element knowledge base is constructed, adjustable compression expression of input semantic features is achieved through weighted combination of multiple sets of coding vectors, and self-adaptive rate transmission under different delay constraints is supported; the task filtering module is used for decoupling task semantic selection from an end-to-end backbone network and supporting quick switching of multi-task semantics; and the private module, the private encoder and the private knowledge base module introduce an incremental learning mechanism for new data, so that the communication service quality of new users is effectively improved. Experiments show that the modular design idea provided by the invention not only improves the flexibility and expandability of the system, but also provides an efficient solution for multi-user semantic communication.

Description

Technical Field

[0001] This invention belongs to the field of network communication technology, specifically relating to a modular semantic communication network construction method and system. Background Technology

[0002] With the exponential growth in the number of communication users, the capacity of existing communication networks is gradually approaching its theoretical limit. In the foreseeable future, network resource saturation will lead to an inability to support new user access, resulting in problems such as increased communication latency and decreased service quality. To address these challenges, semantic communication has emerged as a new communication paradigm. Semantic communication no longer relies on the complete transmission of raw information, but is task-semantic oriented, transmitting only information that is actually meaningful to the receiver, thereby significantly improving communication efficiency and reducing bandwidth consumption.

[0003] Existing semantic communication research largely focuses on end-to-end (E2E) architectures. Typical methods employ deep neural networks to construct encoders and decoders, achieving compression and reconstruction of input data. In this architecture, the encoder typically compresses the input data into a fixed-length feature representation, and the decoder reconstructs the information based on this representation. While these methods perform well in terms of compression efficiency, their tightly coupled network structure necessitates retraining the entire network should a change in feature dimensions occur. This fixed feature length mechanism limits the flexibility of communication systems in multi-user environments; different users cannot dynamically adjust the compression ratio and reconstruction quality according to their individual needs, making it difficult to meet diverse transmission preferences.

[0004] Furthermore, E2E networks typically embed task semantic filtering and recognition functions into specific layers of the neural network. This means that the encoder and decoder must be retrained every time the communication task is changed (e.g., switching from vehicle recognition to pedestrian recognition), severely limiting the flexibility of task transfer. Some studies have proposed replacing task semantics through independent task encoding modules to achieve a certain degree of task transfer capability. However, this method is only effective for pre-trained tasks and cannot adapt to the immediate replacement requirements of new tasks, still requiring retraining of the entire network.

[0005] More importantly, deep neural networks have inherent limitations in generalization ability, making it difficult to adapt to diverse user characteristics and data types, severely impacting the scalability of semantic communication systems. Existing research attempts to ensure consistency between new users' communication behavior and that of existing users by distributing pre-trained network modules through a central server. However, this method still shows insufficient adaptability when facing unknown data types or sudden tasks. To address this, some studies have introduced federated learning mechanisms to support dynamic updates of communication modules. However, frequent model parameter synchronization not only increases the communication burden but may also lead to "catastrophic forgetting," where the system loses its ability to reconstruct old data while learning new data.

[0006] In summary, traditional end-to-end semantic communication systems face the following key problems in multi-user, multi-task environments: (1) the b encoding method is fixed, making it difficult to adaptively adjust the transmission rate; (2) the task semantics are strongly coupled with the network, making it difficult to support flexible task replacement; (3) the generalization ability is limited, making it difficult to cope with transmission requests from unknown users or data, thus limiting the scalability and adaptability of the system.

[0007] Therefore, there is an urgent need for a modular semantic communication network construction method that can decouple the semantic communication system from a single, closed E2E structure, realize pluggable and scalable module combinations, and adapt to the complex communication needs of multiple users, multiple tasks and multiple data types. Summary of the Invention

[0008] Purpose of the Invention: To address the difficulty in scalability of tightly coupled end-to-end networks, this invention proposes a strategy for decoupling network modules, namely a modular semantic communication network construction method and system. This invention improves the scalability of the network by decoupling network encoding, semantic filtering, and incremental learning functions into independent modules.

[0009] Technical Solution: A modular semantic communication network construction method, which decomposes the key functions of the semantic communication network into three independent modules through a network decoupling strategy, enabling flexible network expansion and multi-scenario adaptation, specifically including: (1) Based on the requirements of the task scenario for data volume and accuracy, a multi-code knowledge base is constructed, and the adaptive encoding function is independently encapsulated. The multi-code knowledge base includes a global knowledge base and a private knowledge base, which supports dynamic selection of quantization resolution based on spectrum resource conditions, and realizes flexible adjustment of transmission rate. (2) Based on the task requirements of the receiving end, a task filtering module is constructed, and the semantic filtering function is independently encapsulated. The task filtering module automatically removes semantically irrelevant feature sub-blocks through intelligent agent decision-making, and only filters the quantized feature sub-blocks that are most relevant to the semantics of the current task for transmission. (3) Based on the transmission needs of new users, a private module is constructed to independently encapsulate the incremental learning function. The private module includes a private encoder and a private knowledge base, which improves the communication service quality of new users while ensuring the accuracy of transmission for original users.

[0010] The semantic communication network performs the following semantic communication methods based on the aforementioned multi-codeme knowledge base, task filtering module, and private module: Sending end: Selects data to be sent from the data source. The data to be sent is sent to the global encoder and the private encoder respectively to extract potential global semantic features and private semantic features. The global semantic features are quantized in the global knowledge base. The global knowledge base inputs the quantization results and quantization index of the features. Transmission end: The quantization index is used as the semantic transmission object, and the quantization result is sent to the task filtering module. The task filtering module divides the input quantization features into multiple sub-blocks and selects the sub-block with the most relevant semantics for transmission. At the receiving end: the filtered quantized index is dequantized into quantized global semantic features and private semantic features in the global knowledge base and private knowledge base, respectively. The features dequantized in the private knowledge base are summed and sent to the global decoder to recover the desired semantics.

[0011] Based on the implementation of the modular semantic communication network construction method described above, the present invention can obtain a modular semantic communication network system, which includes: The multi-code knowledge base module is used to implement adaptive encoding functions, including a global knowledge base and a private knowledge base, and supports dynamic adjustment of quantization strategy and transmission rate based on task scenario and spectrum resource status. The task filtering module is used to implement semantic filtering function. It uses an agent trained by reinforcement learning to filter semantically relevant feature sub-blocks for transmission and removes semantically irrelevant information. The private module is used to implement incremental learning functions, including a private encoder and a private knowledge base, which improves the communication service quality for new users without affecting the communication quality of the original users.

[0012] The system also includes a global encoder and a global decoder, which are used to extract and reconstruct global semantic features from the input data, respectively. The global encoder and global decoder are composed of convolutional networks, activation layers, deconvolutional networks, and activation layers.

[0013] This invention, through the design of the aforementioned three modules, decouples tightly coupled end-to-end modules, enabling dynamic rate adjustment, task switching, and data updates. It can provide uninterrupted and reliable semantic communication services and can be extended to multi-user networks.

[0014] Furthermore, the multi-code knowledge base module, task filtering module, and private module are designed to be pluggable, supporting module combination and configuration according to communication scenario requirements. The multi-code knowledge base module can adapt to different spectrum resource conditions by adjusting the quantization resolution, and the task filtering module can adapt to different communication tasks by switching the task mask. Attached Figure Description

[0015] Figure 1 An architectural diagram of the method described in the invention; Figure 2 This is a schematic diagram of the data processing process at the sending end; Figure 3 This is a schematic diagram of the data processing process at the receiving end; Figure 4 This is a rendering of the reconstructed vehicle data; Figure 5 A rendering of the reconstruction of architectural data. Detailed Implementation

[0016] The present invention will now be described in detail with reference to the accompanying drawings and specific examples.

[0017] Semantic communication has attracted much attention due to its potential to overcome the limitations of traditional communication. Current research mainly focuses on end-to-end communication scenarios, using deep neural networks (DNNS) to model communication modules that are intertwined and coupled with each other. This specific embodiment uses image data as an example to extend the service of unknown end-to-end semantic communication to multiple user scenarios.

[0018] This invention primarily addresses the problem that existing deep neural network-based semantic communication frameworks are difficult to scale to multiple users due to tight coupling. To achieve this, this invention mainly employs a network decoupling method, decoupling key functions into three independent modules to enable flexible network expansion. Specifically, it includes the following: (1) The adaptive coding function is separated into a multi-code element knowledge base. It realizes flexible data quantization, and users can choose an appropriate quantization strategy according to the current task and communication environment, thereby adaptively adjusting the transmission rate; (2) Semantic filtering is separated into a task filtering module. It can automatically select the most semantically relevant information to transmit based on the current task; (3) The incremental learning function is made into a private module. It can improve the service quality of new users without affecting the communication quality of the original users.

[0019] The specific transmission process is as follows Figure 1-3As shown, the sender first selects data to be sent from the data source. The data is then sent to both the global encoder and the private encoder to extract potential global and private semantic features. The global semantic features are then quantized in the global knowledge base. During this process, the global knowledge base provides the quantization result and quantization index of the input features. To reduce data transmission, this invention chooses to transmit the quantization index instead of the quantization result. Furthermore, the quantization index of the private semantic features in the private knowledge base is consistent with the global quantization index. Therefore, quantization of the private semantic features is unnecessary here. The quantization result is then sent to the task filtering module. This module divides the input quantized features into multiple sub-blocks and selects the sub-block most relevant to the current semantics for transmission. At the receiving end, the filtered quantization index is dequantized in both the global and private knowledge bases to obtain quantized global and private semantic features. The dequantized features are summed and sent to the global decoder to recover the desired semantics.

[0020] The working details of the main coupling module constructed in this invention will be described in detail below: (1) Construction of a multi-code knowledge base Let's take a global knowledge base as an example to illustrate the working details of a multi-codeme knowledge base. The process for a private knowledge base is the same as that for the global knowledge base. Given a global encoder... Global decoder and global knowledge base The global semantic encoder and global semantic decoder consist of convolutional networks, activation layers, and deconvolutional networks, also with activation layers. The global knowledge base contains... A quantization vector. Taking image data as an example, the sender first obtains the data from the data source. Extracting images to be sent .then First, it is sent to the global encoder to obtain the implicit global semantic features. ,in , Representing global semantic features respectively The channel, with its length and width dimensions, is then used. The semantic features are then sent to a global knowledge base for quantization. This global knowledge base contains... The size is The quantized vector. During quantization, the global semantic features are first... Flattened The size is The vectors are then used to calculate the Euclidean distance between each vector and the quantized vector. This expression represents The first vectors and Inner The distance between quantized vectors. Given the quantization range. Take the closest one One quantization vector is used as the candidate quantization vector.

[0021] Further calculations are needed to determine the weight of each quantization vector, as follows:

[0022] The above equation can be viewed as a distance standardization, where the numerator is the distance standardization. The distance between the selected vector and the input vector is calculated, with the denominator being the sum of the distances to all other selected vectors except this one; the candidate vector is the first... The weights of each quantization vector. Then the input quantization result is:

[0023] in For the selected quantization resolution, At the receiving end, the global decoder receives... And reconstruct it back to the desired image:

[0024] To train the multi-codeme knowledge base and encoding / decoding, the following loss function is used:

[0025] The first term is the reconstruction loss, used to improve image quality. The second and third terms respectively guide the features to move closer to the knowledge base and guide the knowledge base to move closer to the features.

[0026] (2) Construction of the task filtering module

[0027] By training core module 1, a basic image reconstruction network can be obtained. Then, to improve transmission efficiency and remove semantically irrelevant parts from the quantized features, this invention designs a task filtering module. Specifically, this module is modeled as an intelligent agent. Its input state... For quantification features Number of sub-blocks Output action For the sub-block index of the transmission, i.e. To train this agent, this invention employs reinforcement learning (…). And its return function. Defined as:

[0028] in This is the task mask. If the sub-block is a semantically related sub-block, its corresponding value is 1; otherwise, its value is 0. Indicates at time The information transmitted in each sub-block. It contains... A binary variable is defined such that if a sub-block at a certain position has been transmitted, that position is marked as 1; otherwise, it is marked as 0. Correspondingly... This represents information about sub-blocks that have not yet been transmitted. express Whether a new sub-block is transmitted at any given time. In other words, This indicates whether a previously transmitted sub-block has been retransmitted. If a retransmission occurred, all values ​​are 0; otherwise, the position of the newly transmitted sub-block is marked as 1. This result is consistent with... Multiplication represents if New sub-blocks are transmitted continuously, and it is determined whether each sub-block is task-related. If task-related, the reward is 1; if not, the reward is 0; if a duplicate transmission occurs, the reward is -1. Multiple sub-blocks of data are then randomly generated to train the agent.

[0029] Its loss function is expressed as:

[0030] in and They represent and Moment value. Indicates the formation Number of networks. To improve training stability, Use multiple Network. By randomly generating multiple screens, the above method can learn to select semantically relevant sub-blocks (such as...) while sending data without overlap. Figure 4 and Figure 5 As shown: Among them Figure 4 Focus on vehicle-related data, and Figure 5 (Focus on construction-related data).

[0031] (3) Construction of private modules

[0032] To improve the communication service quality for new users (data from outside the training set), this invention designs a private module containing a private encoder. With private knowledge base At the sending end, the private encoder extracts private semantic features from the data:

[0033] To reduce data transmission, this invention aligns the private knowledge base index with the global knowledge base index. At the receiving end, the private knowledge base uses the knowledge base index to find quantified private semantic features. Then this feature is combined with the quantified global semantic features. Summation is performed to reduce the error of global semantic features. Finally, the desired image is reconstructed by the global decoder. .

[0034] Its training objective is expressed as:

[0035] The first term represents the new reconstruction error. The second term modifies the private knowledge base based on private semantic features, and the third term, conversely, modifies the private semantic features based on the private knowledge base.

[0036] In summary, this invention extends the end-to-end semantic communication network to multi-user scenarios by decoupling the existing end-to-end network into three independent modules: 1) a multi-code knowledge base that can dynamically adjust the recovery effect and compression ratio; 2) a task filtering module that can perform task replacement while maintaining existing communication; and 3) a private knowledge base and encoder for expanding network users and adapting to new data types. Experimental results show that the multi-code knowledge base supports different transmission preferences, and its performance in 8-code transmission is comparable to that of the variational autoencoder (VAE) and surpasses that of the vector quantization autoencoder (VQ-VAE). Notably, under 8-code quantization, the proposed method compresses the dataset for each transmission to only 1.17% of the original size. The task filtering module can effectively capture the feature distribution of specific tasks, demonstrating excellent recognition capabilities. Furthermore, the private knowledge base and encoder, by integrating new datasets, improve data accuracy recovery by 22%.

Claims

1. A modular method for constructing a semantic communication network, characterized in that, By employing a network decoupling strategy, the key functions of the semantic communication network are divided into three independent modules, enabling flexible network expansion and multi-scenario adaptation. Specifically, this includes: (1) Based on the requirements of the task scenario for data volume and accuracy, a multi-code knowledge base is constructed, and the adaptive encoding function is independently encapsulated. The multi-code knowledge base includes a global knowledge base and a private knowledge base, which supports dynamic selection of quantization resolution based on spectrum resource conditions, and realizes flexible adjustment of transmission rate. (2) Based on the task requirements of the receiving end, a task filtering module is constructed, and the semantic filtering function is independently encapsulated. The task filtering module automatically removes semantically irrelevant feature sub-blocks through intelligent agent decision-making, and only filters the quantized feature sub-blocks that are most relevant to the semantics of the current task for transmission. (3) Based on the transmission needs of new users, a private module is constructed to independently encapsulate the incremental learning function. The private module includes a private encoder and a private knowledge base, which improves the communication service quality of new users while ensuring the accuracy of transmission for original users.

2. The modular semantic communication network construction method according to claim 1, characterized in that, The semantic communication network performs the following semantic communication methods based on the aforementioned multi-codeme knowledge base, task filtering module, and private module: Sending end: Selects data to be sent from the data source. The data to be sent is sent to the global encoder and the private encoder respectively to extract potential global semantic features and private semantic features. The global semantic features are quantized in the global knowledge base. The global knowledge base inputs the quantization results and quantization index of the features. Transmission end: The quantization index is used as the semantic transmission object, and the quantization result is sent to the task filtering module. The task filtering module divides the input quantization features into multiple sub-blocks and selects the sub-block with the most relevant semantics for transmission. At the receiving end: the filtered quantized index is dequantized into quantized global semantic features and private semantic features in the global knowledge base and private knowledge base, respectively. The features dequantized in the private knowledge base are summed and sent to the global decoder to recover the desired semantics.

3. The modular semantic communication network construction method according to claim 1 or 2, characterized in that, The workflow of the multi-code knowledge base is as follows: The sending end encodes the input data using a global encoder to obtain global semantic features. ,in For input data, For global encoder, , The channels, representing semantic features, are the length and width dimensions, respectively. global semantic features Flattened The size is Given a vector, calculate the Euclidean distance between each vector and the N quantized vectors in the multi-code knowledge base: The above formula represents The first vectors and Inner The distance between quantized vectors; Take the closest Each candidate quantization vector is selected as a potential quantization vector, and the weight of each candidate quantization vector is calculated. in Let be the distance between the i-th vector and the k-th candidate quantized vector; According to quantization resolution Calculation of quantization results And only transmit the quantization index to the receiving end: The receiving end reconstructs the quantized features after inversion using a global decoder to obtain the recovered data. ,in For global decoder; The multi-codeme knowledge base and encoder / decoder are trained using a loss function, which is defined as follows: Where sg(·) is the stopping gradient operation. , The first term represents the reconstruction loss, the second term guides the features to move closer to the knowledge base, and the third term guides the knowledge base to move closer to the features.

4. The modular semantic communication network construction method according to claim 1, characterized in that, The task filtering module employs reinforcement learning. The algorithm trains the intelligent agent, and its specific implementation includes: The input state of the intelligent agent For quantification features Number of sub-blocks Output action For the sub-block index of the transmission, its revenue function is... Defined as: In the formula, For a moment, This is the task mask. If the sub-block is a semantically related sub-block, its corresponding value is 1; otherwise, its value is 0. Indicates at time The information transmitted in each sub-block, corresponding to This represents sub-block information that has not yet been transmitted; express Whether to transmit a new sub-block at any given time; The loss function is defined as: in and They represent and Moment value, Indicates the formation Number of networks.

5. The modular semantic communication network construction method according to claim 1 or 4, characterized in that, Use multiple The network trains the agent by randomly generating multiple sub-data blocks, enabling the agent to learn to prioritize semantically relevant sub-blocks without retransmitting data.

6. The modular semantic communication network construction method according to claim 1 or 2, characterized in that, The private module includes a private encoder. With private knowledge base The workflow is as follows: Construct a private encoder to obtain private semantic features within the data; its composition is similar to that of the global encoder. and global knowledge base Consistent The sending end extracts private semantic features from the input data using a private encoder. ; Private knowledge bases use global indexes Corresponding quantization vectors and private semantic features Alignment; At the receiving end, the received global quantization features With private quantitative characteristics The summation is performed, and the result is sent to the global decoder for decoding to obtain the reconstruction information. .

7. A modular semantic communication network system obtained by implementing the modular semantic communication network construction method as described in any one of claims 1-6, characterized in that, The system includes: The multi-code knowledge base module is used to implement adaptive encoding functions, including a global knowledge base and a private knowledge base, and supports dynamic adjustment of quantization strategy and transmission rate based on task scenario and spectrum resource status. The task filtering module is used to implement semantic filtering function. It uses an agent trained by reinforcement learning to filter semantically relevant feature sub-blocks for transmission and removes semantically irrelevant information. The private module is used to implement incremental learning functions, including a private encoder and a private knowledge base, which improves the communication service quality for new users without affecting the communication quality of the original users.

8. The modular semantic communication network system according to claim 7, characterized in that, The system also includes a global encoder and a global decoder, which are used to extract and reconstruct global semantic features from the input data, respectively. The global encoder and global decoder are composed of convolutional networks, activation layers, deconvolutional networks, and activation layers.

9. The modular semantic communication network system according to claim 7, characterized in that, The multi-code knowledge base module, task filtering module, and private module are designed to be pluggable, allowing for module combination and configuration according to communication scenario requirements. The multi-code knowledge base module can adapt to different spectrum resource conditions by adjusting the quantization resolution, and the task filtering module can adapt to different communication tasks by switching the task mask.

Images