A multi-modal feature adaptive fusion and compression semantic communication and classification method and device

Abstract

The application discloses a multi-modal feature adaptive fusion and compression semantic communication and classification method and device, the multi-modal feature adaptive fusion and compression semantic communication method comprises the following steps: multi-modal feature extraction is performed on collected multi-modal signals to obtain multi-modal features; the multi-modal features are adaptively fused to obtain fused features; the fused features are entropy encoded and compressed to obtain fused and compressed features, and the fused and compressed features are converted into binary code streams through an encoder-decoder mechanism, which are used for realizing semantic communication with a task module. The application has the characteristics of high fusion efficiency, friendly compression, low calculation cost, small communication cost and small bandwidth occupation, and is suitable for the dual requirements of intelligent perception and task execution ability under the conditions of limited bandwidth or limited computing power.

Description

Technical Field

[0001] This invention belongs to the field of intelligent sensing and communication technology, specifically relating to a semantic communication and classification method and apparatus for multimodal feature adaptive fusion and compression. Background Technology

[0002] With the continuous advancement of artificial intelligence, the Internet of Things, and robotics, multimodal perception systems have been widely applied in various task-oriented scenarios, such as robot grasping, autonomous driving, and remote intelligent monitoring. In these systems, the visual modality provides appearance information of objects, including shape, color, and texture, while the tactile modality acquires physical properties of objects, such as surface roughness, hardness, and friction. Effectively fusing these two types of modal information can significantly improve the perception accuracy and decision robustness of the system in complex environments. However, existing multimodal fusion methods still face many challenges in practical applications. First, most methods employ fixed weighting or static attention mechanisms during the fusion process, failing to dynamically adjust the importance of modalities according to specific task requirements or input data characteristics. This easily introduces redundant or inefficient information, affecting the fusion effect and system performance. Second, in edge computing or cloud-based collaborative processing scenarios, original images or high-dimensional features often need to be transmitted over the network. Existing methods generally lack efficient semantic compression strategies for fusion features, resulting in high communication costs and large bandwidth consumption, making it difficult to meet the actual needs in resource-constrained environments. In addition, while some attention mechanisms with a certain degree of adaptability have improved performance, the additional computational complexity and parameter size they introduce limit their deployment and application in lightweight devices or real-time tasks. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a semantic communication and classification method and apparatus based on multimodal feature adaptive fusion and compression. It features high fusion efficiency, compression friendliness, low computational overhead, low communication cost, and low bandwidth usage, and is suitable for the dual requirements of intelligent perception and task execution capabilities under bandwidth-constrained or computing-constrained conditions.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0005] Firstly, a semantic communication method based on adaptive fusion and compression of multimodal features is provided, comprising: extracting multimodal features from the acquired multimodal signals to obtain multimodal features; adaptively fusing the multimodal features to obtain fused features; entropy encoding compression of the fused features to obtain fused compressed features; and converting the fused compressed features into a binary code stream through an encoder-decoder mechanism for semantic communication with a task module.

[0006] Furthermore, the multimodal signal includes: visual image and tactile signal; the multimodal feature extraction includes: inputting the acquired visual image and tactile signal into independent but structurally consistent feature extraction networks to obtain image features and tactile features.

[0007] Furthermore, the feature extraction network uses a pre-trained ResNet-18 neural network as its backbone structure. Its last fully connected classification head is removed, and only the convolutional module and the global average pooling layer are retained to extract semantically rich but dimensionally compressed feature representations.

[0008] Furthermore, adaptive fusion of the multimodal features includes: first, constructing initial fusion weights. Generated through sampling from a Beta distribution, its parameters The modal splicing features are calculated through a fully connected layer:

[0009] ;

[0010] in, This represents the maximum value of the shape parameter of the Beta distribution, used to limit the adjustment range of the fusion weights. This represents the minimum value of the shape parameter of the Beta distribution, used to prevent the fused weights from degenerating into a single mode. The Sigmoid activation function is used to map the output of a fully connected layer to the (0,1) interval. This is the mapping function for the fully connected layer used to compute the shape parameters of the Beta distribution. For the extracted image features, For the extracted tactile features, || denotes the vector concatenation operation;

[0011] Subsequently, an auxiliary attention module is introduced to correct the fusion weights; the input features are processed by a multilayer perceptron to output attention weights. :

[0012] ;

[0013] in, It is a multilayer perceptron used to adaptively generate attention weights based on multimodal features;

[0014] Ultimately, the fusion weights Revised to:

[0015] ;

[0016] in, The corrected fusion weights;

[0017] The fused features are represented as follows:

[0018] ;

[0019] in, For the fused feature representation, The number of residual fusion terms introduced is used to enhance stability and preserve original feature information.

[0020] Furthermore, the fused features are subjected to entropy encoding compression, specifically by fitting the probability density distribution of the fused features to achieve information entropy encoding compression.

[0021] Furthermore, the task module includes a classification decision module, which is used to receive the fused compressed features decoded from the binary code stream and output the corresponding classification result.

[0022] Furthermore, the classifier in the classification decision module includes two fully connected layers. The input dimension of the first fully connected layer is the dimension of the fused compressed features, and the output is a set dimension as the hidden representation. The output of the second fully connected layer is the number of categories. A Dropout mechanism is inserted into the fully connected layer to enhance the generalization ability, and a ReLU activation function is used after the hidden layer to improve the non-linear expression ability.

[0023] Secondly, a classification method based on adaptive fusion and compression of multimodal features is provided, comprising: acquiring multimodal signals of a target, transmitting the multimodal signals to a classification decision module through the semantic communication method based on adaptive fusion and compression of multimodal features described in the first aspect, and outputting a classification result corresponding to the multimodal signals.

[0024] Thirdly, a semantic communication device for adaptive fusion and compression of multimodal features is provided, comprising: a feature extraction module for extracting multimodal features from acquired multimodal signals to obtain multimodal features; an adaptive fusion module for adaptively fusing the multimodal features to obtain fused features; and an entropy-aware compression module for entropy encoding compression of the fused features to obtain fused compressed features, and converting the fused compressed features into a binary code stream through an encoder-decoder mechanism for semantic communication with a task module.

[0025] Fourthly, a classification device for adaptive fusion and compression of multimodal features is provided, comprising: a signal acquisition module for acquiring multimodal signals of a target; a semantic communication device for adaptive fusion and compression of multimodal features for transmitting the multimodal signals to a classification decision module via the semantic communication method for adaptive fusion and compression of multimodal features described in the first aspect; and a classification decision module for outputting a classification result corresponding to the multimodal signals.

[0026] Compared with existing technologies, the beneficial effects achieved by this invention are as follows: This invention extracts multimodal features from the acquired multimodal signals, adaptively fuses the multimodal features, compresses the fused features using entropy coding, and converts the fused and compressed features into a binary code stream through an encoder-decoder mechanism for semantic communication with the task module. It features high fusion efficiency, compression friendliness, low computational overhead, low communication cost and bandwidth usage, and adapts to the dual requirements of intelligent perception and task execution capabilities under bandwidth-limited or computing-power-limited conditions. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of a semantic communication architecture for adaptive fusion and compression of multimodal features provided in an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of the modified ResNet-18 neural network in an embodiment of the present invention. Detailed Implementation

[0029] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0030] Example 1

[0031] A semantic communication method based on adaptive fusion and compression of multimodal features includes: extracting multimodal features from acquired multimodal signals to obtain multimodal features; adaptively fusing the multimodal features to obtain fused features; entropy encoding compression of the fused features to obtain fused compressed features; and converting the fused compressed features into a binary code stream through an encoder-decoder mechanism for semantic communication with a task module.

[0032] like Figure 1 The diagram shows a schematic of the semantic communication architecture MUSCLE, which features adaptive fusion and compression of multimodal features. It includes a visual modality input module, a tactile modality input module, a feature extraction module, an adaptive fusion module (FuseMix), an entropy-aware compression module, and a classification decision module. This method is suitable for efficient visual-tactile task classification in environments with limited bandwidth and computational resources.

[0033] First, multimodal feature extraction is performed on the acquired multimodal signals to obtain multimodal features.

[0034] Multimodal signals include visual images and tactile signals. Visual images and tactile signals serve as input modalities, each fed into an independent but structurally consistent feature extraction network. The feature extraction network uses a pre-trained ResNet-18 neural network as its backbone, with the last fully connected classification head removed, retaining only convolutional modules and a global average pooling (GAP) layer to extract semantically rich but dimensionally compressed feature representations, such as... Figure 2 As shown. Specifically, the feature output f extracted by the network from the input visual image or tactile signal x is:

[0035] (1)

[0036] like Figure 1 As shown, when the input visual image x is given, the feature output f is the extracted image features, denoted as f(x). When the input tactile signal y is received, the feature output f is the extracted tactile feature, denoted as y. Both are 512-dimensional feature vectors; ConvBlock represents a set of convolutional modules consisting of multiple convolutional operations, used to extract high-level semantic features from the input signal; GAP represents global average pooling operation, used to compress spatial dimensions to obtain fixed-dimensional feature representations; Flatten represents flattening operation, used to map multi-dimensional features to one-dimensional feature vectors; R represents the real number field, and the feature vectors are defined in the real number space.

[0037] The feature extraction module of this invention is based on a pre-trained ResNet-18 network. The classifier has a simple structure, containing only two fully connected layers. It has a small number of parameters, stable training, lightweight model, and low computational cost, making it suitable for deployment on edge devices and in real-time scenarios.

[0038] Second, adaptively fuse the multimodal features to obtain the fused features.

[0039] Based on the feature extraction in step one, in order to dynamically adjust the modality contribution ratio, this invention proposes an adaptive fusion method, FuseMix, based on Beta distribution and attention guidance. Its fusion process consists of the following multiple steps.

[0040] First, construct the initial fusion weights. Generated through sampling from a Beta distribution, its parameters The modal splicing features are calculated through a fully connected layer:

[0041] (2)

[0042] in This represents the maximum value of the shape parameter of the Beta distribution, used to limit the adjustment range of the fusion weights; This represents the minimum value of the shape parameter of the Beta distribution, used to prevent the fused weights from degenerating into a single mode. This is used to prevent extreme fusion; This is the Sigmoid activation function, used to map the output of the fully connected layer to the (0,1) interval; This is the mapping function for the fully connected layer used to calculate the shape parameters of the Beta distribution; For the extracted image features; The extracted tactile features; || represents the vector concatenation operation.

[0043] Subsequently, an auxiliary attention module is introduced to correct the fusion weights; the input features are processed by a multilayer perceptron (MLP) to output attention weights. :

[0044] (3)

[0045] Ultimately, the fusion weights Revised to:

[0046] (4)

[0047] in, The corrected fusion weights;

[0048] The fused features are represented as follows:

[0049] (5)

[0050] in, For the fused feature representation, The number of residual fusion terms introduced is used to enhance stability and preserve original feature information.

[0051] The adaptive fusion module (FuseMix) proposed in this invention can dynamically adjust the contribution ratio of visual and tactile modalities in the fusion based on the features of the input data, thereby achieving adaptive regulation of modal importance and significantly improving the semantic expressiveness and task adaptability of the fusion.

[0052] This fusion strategy is lightweight, learnable, and dynamically adjustable, enabling it to adapt the fusion ratio in real time according to task requirements, thereby improving the performance of downstream tasks.

[0053] Third, the fused features are entropy encoded and compressed to obtain fused compressed features, and the fused compressed features are converted into binary code streams through an encoder-decoder mechanism for semantic communication with the task module.

[0054] To further reduce communication bandwidth consumption, the fused features The data is fed into the entropy-aware compression module for semantic compression. This module performs semantic compression by fitting... Information entropy encoding is achieved by using the probability density distribution, which approximately reaches the theoretical compression lower bound. A compression model is used to estimate the probability distribution for each feature dimension, and a binary bitstream is generated through an encoder-decoder mechanism.

[0055] Communication compression effectiveness is measured by two metrics:

[0056] (1) Bits Per Pixel (BPP): Represents the average number of bits per pixel;

[0057] (2) Average bit stream length: indicates the average number of compressed bits transmitted per sample.

[0058] This invention performs probability density modeling based on the fused feature dimension and introduces a lightweight entropy coding module, effectively reducing the feature bit rate and significantly improving communication efficiency. Compared to traditional uncompressed transmission methods, the average bit stream length is reduced by more than 60%, exhibiting strong communication adaptability in bandwidth-constrained environments; it also significantly reduces feature transmission costs, achieving a compression bit reduction of up to 63.95% in experiments.

[0059] IV. Classification based on tasks.

[0060] The entropy-aware compression module communicates with the task module via a binary code stream. The type and function of the task module can be configured according to specific needs; for example, it can be a classification decision module.

[0061] In this invention, the decoded fused features are fed into the classification decision module. The classifier in this module consists of two fully connected layers. The first layer has an input dimension of fused feature dimension (e.g., 512) and an output dimension of 800, serving as the hidden representation. The second layer outputs the number of classes (num_classes). A Dropout mechanism is inserted to enhance generalization ability, and a ReLU activation function is used after the hidden layers to improve non-linear expressive power. The specific network structure is as follows:

[0062] fc1: Linear (input_dim, 800); where fc1 represents the first fully connected layer in the classifier; Linear (⋅) represents the linear mapping operation, used to implement the linear transformation of the feature vector; input_dim represents the dimension of the fused feature vector input to the classifier.

[0063] Activation function: ReLU linear rectified function;

[0064] Dropout(p=0.1); where Dropout(⋅) represents the random deactivation operation, used to randomly disable some neurons during training to enhance the model's generalization ability; p represents the probability that a neuron is randomly deactivated.

[0065] fc2: Linear(800, num_classes); where fc2 represents the second fully connected layer in the classifier; num_classes represents the number of classes corresponding to the classification task.

[0066] This module accepts the compressed and decoded fused feature vector as input and outputs the corresponding classification result. It has strong generalization ability, improves model robustness through Dropout and attention-guided mechanisms, and supports end-to-end training. It can improve the overall system performance by co-optimizing the various stages of multimodal feature fusion, compression, and task recognition, and optimizes it through backpropagation using the cross-entropy loss function between the module and the label.

[0067] The parameters in the fusion module of this invention are all learned by the downstream task loss function, which can significantly improve the quality of multimodal fusion and communication efficiency without increasing a lot of computational overhead.

[0068] This invention addresses the issues of limited communication bandwidth and redundant modal transmission in multimodal perception scenarios. The method constructs a unified multimodal semantic communication architecture, MUSCLE (Multimodal Unified Semantic Communication with Lightweight Entropy Coding), which integrates visual and tactile modal information. It has wide applications, suitable for task classification scenarios in robot grasping, object recognition, intelligent manufacturing, autonomous driving, and edge intelligent devices, and is particularly suitable for distributed multimodal intelligent perception systems sensitive to communication resources.

[0069] In this invention, a convolutional neural network feature extraction module is first used to extract deep features from visual images and tactile spectrograms, generating fixed-dimensional modal feature vectors. Subsequently, an adaptive fusion module based on probability distribution is designed, dynamically adjusting the weights of each modal feature through a Beta distribution to achieve adaptive modeling of modal importance, effectively avoiding the performance bottleneck caused by fixed-weight fusion. To further reduce communication overhead, the fused features are entropy-encoded and compressed using probability density estimation, obtaining an encoded representation close to the theoretical minimum bit rate, thus improving communication efficiency.

[0070] To verify the effectiveness of this invention, experiments were conducted using the LMT-TextureDB multimodal dataset. The results show that, compared to traditional weighted fusion and attention fusion methods, the proposed FuseMix fusion strategy improves classification accuracy by 1.34%–11.61% and reduces average bitstream by 63.95% while maintaining a similar number of model parameters. It achieves an effective balance between accuracy (recognition performance), communication efficiency, and model deployment efficiency, demonstrating significant engineering practical value and potential for widespread application.

[0071] Example 2

[0072] Based on the semantic communication method of adaptive fusion and compression of multimodal features described in Embodiment 1, this embodiment provides a classification method of adaptive fusion and compression of multimodal features, including:

[0073] Collect multimodal signals from the target.

[0074] The multimodal signal is transmitted to the classification decision module using the semantic communication method of multimodal feature adaptive fusion and compression described in Example 1;

[0075] Output the classification result corresponding to the multimodal signal.

[0076] Example 3

[0077] Based on the semantic communication method of adaptive fusion and compression of multimodal features described in Embodiment 1, this embodiment provides a semantic communication device of adaptive fusion and compression of multimodal features, comprising:

[0078] The feature extraction module is used to extract multimodal features from the acquired multimodal signals to obtain multimodal features;

[0079] An adaptive fusion module is used to adaptively fuse the multimodal features to obtain fused features;

[0080] The entropy-aware compression module is used to entropy-encode and compress the fused features to obtain fused compressed features, and converts the fused compressed features into a binary code stream through an encoder-decoder mechanism for semantic communication with the task module.

[0081] Example 4

[0082] Based on the semantic communication method of adaptive fusion and compression of multimodal features described in Embodiment 1, and the semantic communication device of adaptive fusion and compression of multimodal features described in Embodiment 2, this embodiment provides a classification device of adaptive fusion and compression of multimodal features, including:

[0083] The signal acquisition module is used to acquire multimodal signals from the target being detected.

[0084] A semantic communication device for adaptive fusion and compression of multimodal features is used to transmit the multimodal signal to the classification decision module through the semantic communication method for adaptive fusion and compression of multimodal features described in Embodiment 1.

[0085] The classification decision module is used to output the classification result corresponding to the multimodal signal.

[0086] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0088] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A semantic communication method for adaptive fusion and compression of multimodal features, characterized in that, include: Multimodal features are extracted from the acquired multimodal signals to obtain multimodal features; The multimodal features are adaptively fused to obtain the fused features; The fused features are entropy-encoded and compressed to obtain fused compressed features, which are then converted into binary code streams through an encoder-decoder mechanism for semantic communication with the task module. The multimodal signals include visual images and tactile signals; the multimodal feature extraction includes: inputting the acquired visual images and tactile signals into independent but structurally consistent feature extraction networks to obtain image features and tactile features. Adaptive fusion of the multimodal features includes: First, construct the initial fusion weights. Generated through sampling from a Beta distribution, its parameters The modal splicing features are calculated through a fully connected layer: ； in, This represents the maximum value of the shape parameter of the Beta distribution, used to limit the adjustment range of the fusion weights; This is the minimum value of the shape parameter of the Beta distribution, used to avoid the fusion weights from degenerating into a single mode; This is the Sigmoid activation function, used to map the output of the fully connected layer to the (0,1) interval; This is the mapping function for the fully connected layer used to calculate the shape parameters of the Beta distribution; For the extracted image features; The extracted tactile features; || represents the vector concatenation operation; Subsequently, an auxiliary attention module is introduced to correct the fusion weights; the input features are processed by a multilayer perceptron to output attention weights. : ； in, It is a multilayer perceptron used to adaptively generate attention weights based on multimodal features; Ultimately, the fusion weights Revised to: ； in, The corrected fusion weights; The fused features are represented as follows: ； in, For the fused feature representation, The number of residual fusion terms introduced is used to enhance stability and preserve original feature information.

2. The semantic communication method for adaptive fusion and compression of multimodal features according to claim 1, characterized in that, The feature extraction network uses a pre-trained ResNet-18 neural network as its backbone structure. Its last fully connected classification head is removed, and only the convolutional module and the global average pooling layer are retained to extract semantically rich but dimensionally compressed feature representations.

3. The semantic communication method for adaptive fusion and compression of multimodal features according to claim 1, characterized in that, The fused features are compressed by entropy encoding, specifically by fitting the probability density distribution of the fused features to achieve information entropy encoding compression.

4. The semantic communication method for adaptive fusion and compression of multimodal features according to claim 1, characterized in that, The task module includes a classification decision module, which receives the fused compressed features decoded from the binary code stream and outputs the corresponding classification result.

5. The semantic communication method for adaptive fusion and compression of multimodal features according to claim 4, characterized in that, The classifier in the classification decision module includes two fully connected layers. The input dimension of the first fully connected layer is the dimension of the fused compressed features, and the output is a hidden representation with a set dimension. The output of the second fully connected layer is the number of categories. A Dropout mechanism is inserted into the fully connected layer to enhance generalization ability, and a ReLU activation function is used after the hidden layer to improve non-linear expression ability.

6. A classification method for adaptive fusion and compression of multimodal features, characterized in that, include: Collect multimodal signals from the target. The multimodal signal is transmitted to the classification decision module using the semantic communication method of multimodal feature adaptive fusion and compression as described in any one of claims 4 to 5; Output the classification result corresponding to the multimodal signal.

7. A semantic communication device for adaptive fusion and compression of multimodal features, characterized in that, include: The feature extraction module is used to extract multimodal features from the acquired multimodal signals to obtain multimodal features; An adaptive fusion module is used to adaptively fuse the multimodal features to obtain fused features; The entropy-aware compression module is used to entropy-encode and compress the fused features to obtain fused compressed features, and converts the fused compressed features into a binary code stream through an encoder-decoder mechanism for semantic communication with the task module. The multimodal signals include visual images and tactile signals; the multimodal feature extraction includes: inputting the acquired visual images and tactile signals into independent but structurally consistent feature extraction networks to obtain image features and tactile features. Adaptive fusion of the multimodal features includes: First, construct the initial fusion weights. Generated through sampling from a Beta distribution, its parameters The modal splicing features are calculated through a fully connected layer: ； in, This represents the maximum value of the shape parameter of the Beta distribution, used to limit the adjustment range of the fusion weights; This is the minimum value of the shape parameter of the Beta distribution, used to avoid the fusion weights from degenerating into a single mode; This is the Sigmoid activation function, used to map the output of the fully connected layer to the (0,1) interval; This is the mapping function for the fully connected layer used to calculate the shape parameters of the Beta distribution; For the extracted image features; The extracted tactile features; || represents the vector concatenation operation; Subsequently, an auxiliary attention module is introduced to correct the fusion weights; the input features are processed by a multilayer perceptron to output attention weights. : ； in, It is a multilayer perceptron used to adaptively generate attention weights based on multimodal features; Ultimately, the fusion weights Revised to: ； in, The corrected fusion weights; The fused features are represented as follows: ； in, For the fused feature representation, The number of residual fusion terms introduced is used to enhance stability and preserve original feature information.

8. A classification device for adaptive fusion and compression of multimodal features, characterized in that, include: The signal acquisition module is used to acquire multimodal signals from the target being detected. A semantic communication device for adaptive fusion and compression of multimodal features, used to transmit the multimodal signal to a classification decision module through the semantic communication method for adaptive fusion and compression of multimodal features as described in any one of claims 4 to 5; The classification decision module is used to output the classification result corresponding to the multimodal signal.

Images