A rate-controllable semantic communication method for body-equipped multi-agent semantic sharing

Abstract

This invention discloses a rate-controllable semantic communication method for semantic sharing among embodied multi-agent systems, relating to the fields of semantic communication, computer vision, and multi-agent collaboration. It addresses the problem of poor semantic map sharing performance among embodied multi-agent systems. The method includes: converting a projected map into semantic feature representations after semantic feature extraction and a semantic segmentation map into semantic feature representations after wavelet transform semantic feature extraction; performing rate-controllable feature selection encoding on the semantic feature representations and sending them into a wireless channel; and performing semantic decoding on the received signal transmitted through the wireless channel to obtain the reconstructed projected map or reconstructed semantic segmentation map. This invention effectively improves the compression efficiency and transmission robustness of the transmitting end in multi-embodied multi-agent map sharing, achieves dynamic control of the transmission rate, and enhances the semantic map sharing effect.

Description

Technical Field

[0001] This invention relates to the fields of semantic communication, computer vision, and multi-agent collaboration, and in particular to a rate-controllable semantic communication method and system for semantic sharing among embodied multi-agents. Background Technology

[0002] Embodied agents represent one of the ultimate forms of artificial intelligence. The collaborative efficiency of multi-embodied agent systems highly depends on the accurate and efficient sharing of environmental semantic information. In multi-embodied agent collaborative scenarios, a single agent can only perceive a local environment and cannot grasp global information, making it difficult to achieve efficient coordination.

[0003] Bird's-eye view (BEV) semantic maps have become the mainstream representation of semantic information in embodied multi-agent environments due to their strong spatial consistency, clear semantic structure, and high compatibility with downstream tasks. The transmission and sharing of BEV semantic maps can break down information barriers between agents, enabling all agents to obtain a unified global understanding of the environment. This is a core prerequisite for ensuring synchronous collaboration among multiple agents and the successful completion of complex tasks, and thus possesses strong practical necessity.

[0004] However, signal fading, interference, and bandwidth limitations in wireless channels lead to information loss and transmission delays during BEV semantic map transmission, severely reducing the performance of multi-agent collaboration. Semantic communication, as an effective paradigm for improving anti-interference capabilities and reducing bandwidth consumption, has been applied in the field of image transmission. However, existing semantic communication methods are mainly designed for natural images and do not consider the spatial sparsity and channel redundancy characteristics of semantic segmentation maps in BEV semantic maps, making it impossible to achieve efficient compression and robust transmission of BEV semantic maps. At the same time, existing methods lack the ability to dynamically adjust the transmission rate, making it difficult to adapt to the dynamic changes of wireless channels in multi-agent systems. Under harsh communication conditions such as low signal-to-noise ratio and low bandwidth, the accuracy of map reconstruction drops significantly.

[0005] Therefore, how to adapt to the features of BEV semantic maps and support semantic communication with controllable rate, and solve the problems of low compression efficiency, poor transmission robustness and inability to dynamically adjust the rate in multi-agent map sharing, has become an important technical challenge in the current field of multi-agent collaborative perception and semantic communication. Summary of the Invention

[0006] This invention addresses the technical problem of poor performance in existing multi-agent semantic map sharing by proposing a rate-controllable semantic communication method and system for multi-agent semantic sharing. This method can effectively improve compression efficiency and transmission robustness in multi-agent semantic sharing and achieve dynamic control of transmission rate.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] In a first aspect, the present invention provides a rate-controllable semantic communication method for semantic sharing among embodied multi-agent systems, comprising the following steps:

[0009] S1: Project the map After semantic feature extraction, it is transformed into semantic feature representation. ;

[0010] S2: Transform the semantic segmentation graph After semantic feature extraction via wavelet transform, it is transformed into a semantic feature representation. ;

[0011] S3: Semantic feature representation obtained in step S1 Or the semantic feature representation obtained in step S2 Rate-controlled feature selection coding is performed to obtain the transmitted signal. And sent into the wireless channel, where the rate factor is ;

[0012] S4: Received signal after transmission via wireless channel Semantic decoding is performed to obtain the reconstructed projection map. Or reconstruct the semantic segmentation graph result.

[0013] Furthermore, step S1 uses a four-layer cascaded convolutional residual block module to project the map. Semantic extraction is performed, and its mathematical expression is:

[0014] ,

[0015] in, It is a semantic feature representation; It is a semantic feature extractor based on convolutional residual blocks.

[0016] Furthermore, step S2 uses a four-layer cascaded wavelet transform convolutional residual block module to process the semantic segmentation map. The mathematical expression for wavelet transform semantic extraction is as follows:

[0017] ,

[0018] in, It is a semantic feature representation; It is a semantic feature extractor based on wavelet transform convolutional residual blocks.

[0019] Furthermore, in step S2, the implementation process of wavelet transform semantic feature extraction is as follows:

[0020] S2.1: Convolve the input of the wavelet transform The transformation result is obtained by wavelet transform. , , , ,in These are the low-frequency components of the wavelet transform; , , These are the high-frequency components of the wavelet transform;

[0021] S2.2: Reduce low-frequency components The transformation result is obtained by wavelet transform. , , , ;

[0022] S2.3: Transformation results , , , Perform a convolution operation, followed by an inverse wavelet transform, to obtain the wavelet transform convolution result. ;

[0023] S2.4: To , , , Perform a convolution operation and combine the convolution result with the wavelet transform convolution result. Add them together to get the convolution sum result. , , , ;

[0024] S2.5: For , , , Perform an inverse wavelet transform to obtain the wavelet transform convolution result. ;

[0025] S2.6: For input Perform a convolution operation and combine the convolution result with the wavelet transform convolution result. Add them together to get the final output. .

[0026] Furthermore, the specific implementation process of step S3 is as follows:

[0027] S3.1: Representation of semantic features or Channel importance is evaluated using an attention-based mechanism to obtain a channel importance weight vector. ;

[0028] S3.2: Based on the channel importance weight vector and rate factor Preserving the pre-semantic feature representation One important channel, among which The ratio of the total number of channels representing semantic features is equal to ;

[0029] S3.3: Perform channel pruning on the remaining channels to obtain the transmitted signal. ;

[0030] Its mathematical expression is:

[0031] ,

[0032] in, It is a semantic feature representation or ; It is a rate-controllable feature selection encoder based on the attention mechanism.

[0033] Furthermore, the specific implementation process of step S4 is as follows:

[0034] Use a four-layer cascaded deconvolution block to process the received signal. Decoding is performed, and its mathematical expression is:

[0035] ,

[0036] in, It is a semantic decoder. It is to reconstruct the projection map Or reconstruct the semantic segmentation graph .

[0037] Furthermore, the aforementioned rate-controllable semantic communication method for embodied multi-agent semantic sharing also includes a training step S5, which involves projecting a map. Semantic segmentation graph Rate factor Reconstructing the projected map and reconstructing semantic segmentation graphs The model is trained using the training set, and the trained model is saved after training for map sharing. The model includes: a semantic feature extractor, a wavelet transform semantic feature extractor, a rate-controlled feature selection encoder, and a semantic decoder.

[0038] Furthermore, the specific implementation process of step S5, the training process, is as follows:

[0039] S5.1: Training the Semantic Feature Extractor Rate-controllable feature selection encoder and semantic decoder ;

[0040] S5.2: Training the Wavelet Transform Semantic Feature Extractor Rate-controllable feature selection encoder and semantic decoder ;

[0041] S5.3: Freeze the semantic feature extractors obtained in S5.1 and S5.2. and wavelet transform semantic feature extractor The parameters, mixed data retraining rate controllable feature selection encoder and semantic decoder ;

[0042] S5.4: Freeze the rate-controllable feature selection encoder obtained in S5.3 and semantic decoder The parameters were then fine-tuned again for the semantic feature extractor. Wavelet transform semantic feature extractor Save the training model.

[0043] Secondly, the present invention proposes a rate-controllable semantic communication system for embodied multi-agent semantic sharing, comprising the following modules to implement the method described in any of the above:

[0044] Semantic feature extractor: used to extract projected maps Encoding as semantic feature representation ;

[0045] Wavelet transform semantic feature extractor: used to extract semantic segmentation maps Encoding as semantic feature representation ;

[0046] Rate-controlled feature selection encoder: Used to adjust the transmission rate, and obtains the transmitted signal by performing channel importance assessment and channel pruning on semantic feature representations. ;

[0047] Wireless channel, used to receive transmitted signals And output the received signal ;

[0048] Semantic decoder: used to decode received signals Semantic encoding is performed to obtain the reconstructed projection map recovered after wireless channel transmission. Or reconstruct the semantic segmentation graph .

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0050] This invention proposes a rate-controllable semantic communication method for semantic sharing among embodied multi-agent systems. It employs a differentiated semantic feature extraction scheme, using four-layer cascaded convolutional residual blocks to process the projected map and four-layer cascaded wavelet transform convolutional residual blocks to process the semantic segmentation map. By combining the edge feature extraction advantages of wavelet transform, it achieves deep mining of map semantic information, providing a high-quality feature foundation for subsequent transmission and reconstruction, demonstrating significant application potential in embodied multi-agent collaborative scenarios. Simultaneously, this invention employs a rate-controllable feature selection encoding scheme based on an attention mechanism. By flexibly adjusting the number of transmission channels through a rate factor α, and combining channel importance assessment and pruning strategies, it achieves adaptive control of the transmission rate, enhancing the robustness of embodied agents under complex communication conditions such as resource constraints and channel fluctuations.

[0051] This invention proposes a rate-controlled semantic communication system for embodied multi-agent semantic sharing. It integrates a semantic feature extractor, a wavelet transform semantic feature extractor, a rate-controlled feature selection encoder, and a semantic decoder, thereby achieving efficient transmission and reconstruction of semantic maps in wireless channel scenarios, significantly improving the accuracy and reliability of embodied multi-agent map sharing. Furthermore, this invention achieves collaborative optimization of each module through a multi-stage training process. The system also adopts a modular design (semantic feature extractor, wavelet transform semantic feature extractor, rate-controlled feature selection encoder, and semantic decoder), which can flexibly adapt to the map sharing needs of embodied multi-agents, improving the system's scalability and generalization ability, and further enhancing the collaborative capabilities of embodied multi-agents. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0053] Figure 1 This is a flowchart of a rate-controllable semantic communication method for semantic sharing among embodied multi-agents, provided in an embodiment of the present invention.

[0054] Figure 2 This is a flowchart of the wavelet transform semantic feature extractor method provided in an embodiment of the present invention.

[0055] Figure 3 This is a schematic diagram of the rate-controllable feature selection encoding process provided in an embodiment of the present invention.

[0056] Figure 4This is an illustration of the rate-controllable semantic communication method for semantic sharing among embodied multi-agents provided in an embodiment of the present invention.

[0057] Figure 5 This is a block diagram of the structure of a rate-controllable semantic communication system for embodied multi-agent semantic sharing provided in an embodiment of the present invention. Detailed Implementation

[0058] To better understand this technical solution, the method of the present invention will be described in detail below with reference to the accompanying drawings.

[0059] This invention proposes a rate-controllable semantic communication method for semantic sharing among embodied multi-agent systems. The overall system architecture and process are as follows: Figure 1 As shown, the specific steps include:

[0060] Step S1: Project the map After semantic feature extraction, it is transformed into semantic feature representation. .

[0061] Specifically, the implementation process of step S1 is as follows:

[0062] A four-layer cascaded convolutional residual block module is used to project the map. Semantic extraction is performed, and its mathematical expression is:

[0063] ,

[0064] in, It is a semantic feature representation; It is a semantic feature extractor based on convolutional residual blocks.

[0065] Step S2: Transform the semantic segmentation map After semantic feature extraction via wavelet transform, it is transformed into a semantic feature representation. .

[0066] Specifically, such as Figure 2 As shown, the implementation process of step S2 is as follows:

[0067] A four-layer cascaded wavelet transform convolutional residual block module is used to segment the semantic map. The mathematical expression for wavelet transform semantic extraction is as follows:

[0068] ,

[0069] in, It is a semantic feature representation; It is a semantic feature extractor based on wavelet transform convolutional residual blocks.

[0070] The implementation process of wavelet transform convolution is as follows:

[0071] S2.1: Convolve the input of the wavelet transform The transformation result is obtained by wavelet transform. , , , ,in These are the low-frequency components of the wavelet transform; , , These are the high-frequency components of the wavelet transform;

[0072] S2.2: Reduce low-frequency components The transformation result is obtained by wavelet transform. , , , ;

[0073] S2.3: Transformation results , , , Perform a convolution operation, followed by an inverse wavelet transform, to obtain the wavelet transform convolution result. ;

[0074] S2.4: To , , , Perform a convolution operation and combine the convolution result with the wavelet transform convolution result. Add them together to get the convolution sum result. , , , ;

[0075] S2.5: For , , , Perform an inverse wavelet transform to obtain the wavelet transform convolution result. ;

[0076] S2.6: For input Perform a convolution operation and combine the convolution result with the wavelet transform convolution result. Add them together to get the final output. .

[0077] This invention employs a four-layer cascaded wavelet transform convolutional residual block to process semantic segmentation maps. By combining the edge feature extraction advantages of wavelet transform, it achieves effective semantic feature extraction and demonstrates great potential in subsequent communication processes.

[0078] Step S3: Represent the semantic features obtained in step S1 Or the semantic feature representation obtained in step S2 Perform rate-controlled feature selection coding and send it into the wireless channel.

[0079] Specifically, such as Figure 3 As shown, the implementation process of step S3 is as follows:

[0080] S3.1: Representation of semantic features or Channel importance is evaluated using an attention-based mechanism to obtain a channel importance weight vector. ;

[0081] S3.2: Based on the channel importance weight vector and rate factor Preserving the pre-semantic feature representation One important channel, among which The percentage of the total number of channels representing semantic features is ;

[0082] S3.3: Perform channel pruning on the remaining channels to obtain the transmitted signal. ;

[0083] Its mathematical expression is:

[0084] ,

[0085] in, It is a semantic feature representation or ; It is a rate-controllable feature selection encoder based on the attention mechanism.

[0086] Step S3 of the present invention specifically adopts a rate-controllable feature selection coding scheme based on an attention mechanism. By flexibly adjusting the number of transmission channels through a rate factor α and combining channel importance assessment and pruning strategies, adaptive regulation of the transmission rate is achieved, thereby enhancing the robustness of the embodied agent under complex communication conditions such as resource constraints and channel fluctuations.

[0087] Step S4: Receive the signal Perform semantic decoding and convert it into a reconstructed projection map. Or reconstruct the semantic segmentation graph .

[0088] Specifically, the implementation process of step S4 is as follows:

[0089] Use a four-layer cascaded deconvolution block to process the received signal. Decoding is performed, and its mathematical expression is:

[0090] ,

[0091] in, It is a semantic decoder. It is to reconstruct the projection map Or reconstruct the semantic segmentation graph .

[0092] Step S5: Project the map Semantic segmentation graph Rate factor Reconstructing the projected map and reconstructing semantic segmentation graphs Use this as a training dataset to train the semantic feature extractor. Wavelet transform semantic feature extractor Rate-controllable feature selection encoder and semantic decoder .

[0093] Specifically, in step S5, the training process is implemented as follows:

[0094] S5.1: Training the Semantic Feature Extractor Rate-controllable feature selection encoder and semantic decoder ;

[0095] S5.2: Training the Wavelet Transform Semantic Feature Extractor Rate-controllable feature selection encoder and semantic decoder ;

[0096] S5.3: Freeze the semantic feature extractors obtained in S5.1 and S5.2. and wavelet transform semantic feature extractor The parameters, mixed data retraining rate controllable feature selection encoder and semantic decoder ;

[0097] S5.4: Freeze the rate-controllable feature selection encoder obtained in S5.3 and semantic decoder The parameters were then fine-tuned again for the semantic feature extractor. Wavelet transform semantic feature extractor .

[0098] Step S6: Save the training model, which can then be applied to embodied multi-agent map sharing under low-quality communication conditions.

[0099] To verify the effectiveness of the rate-controllable semantic communication method for embodied multi-agent semantic sharing provided by the present invention, the following experiments were conducted to verify the performance of the method of the present invention.

[0100] The test results are as follows Figure 4 As shown, the horizontal axis represents different channel conditions. It is the signal-to-noise ratio, such as Figure 4 In the text, (1) and (3) represent the method performance under AWGN channel conditions; such as Figure 4 (2) and (4) in the text represent the performance of the method under Rayleigh fading channel conditions. Figure 4 (1) and (2) in the figure represent the performance of the projection map sharing method. The evaluation index is the peak signal-to-noise ratio (PSNR) score. The higher the value, the better the map reconstruction effect. Figure 4 In the text, (3) and (4) represent the performance of the semantic segmentation map sharing method. The evaluation index is the peak signal-to-noise ratio (PSNR) score. The higher the value, the better the map reconstruction effect.

[0101] The above experimental results show that, under different noise levels, the rate-controllable semantic communication method for embodied multi-agent semantic sharing provided by this invention has good robustness.

[0102] It should be understood that the method of the present invention can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, it can be considered a sequenced list of executable instructions for implementing logical functions, which can be stored in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as computer-based systems, including processor-based systems, and other systems that can fetch and execute instructions from, or in conjunction with, such instruction execution systems, apparatus, or devices). For the purposes of this description, a computer-readable medium can be any means that contains, stores, communicates, propagates, or transmits programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0103] Specifically, more specific examples of computer-readable media (a non-exhaustive list) include, but are not limited to: electrical connections (electronic devices) having one or more wirings, portable computer disks (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM).

[0104] This invention integrates a semantic feature extractor, a wavelet transform semantic feature extractor, a rate-controlled feature selection encoder, and a semantic decoder, thereby enabling efficient transmission and reconstruction of semantic maps in dynamic, resource-constrained wireless channel scenarios, and improving the accuracy and reliability of embodied multi-agent map sharing.

[0105] Secondly, this invention provides a rate-controllable semantic communication system for semantic sharing among embodied multi-agent systems, the overall system architecture and process of which are as follows: Figure 5 As shown, it mainly includes the following modules: semantic feature extraction module, wavelet transform semantic feature extraction module, rate-controllable feature selection encoding module, and semantic decoding module.

[0106] Semantic feature extraction module: used to extract the projected map Encoding as semantic feature representation ;

[0107] Wavelet transform semantic feature extraction module: used to extract semantic segmentation maps Encoding as semantic feature representation ;

[0108] Rate-controllable feature selection coding module: used to adjust the transmission rate, and obtain the transmitted signal by performing channel importance assessment and channel pruning on semantic feature representations. ;

[0109] Semantic decoding module: used to decode received signals Semantic encoding is performed to obtain the reconstructed projection map recovered after wireless channel transmission. Or reconstruct the semantic segmentation graph .

[0110] In summary, the present invention integrates an encoder for semantic feature extraction, a wavelet transform encoder, a feature selection encoding module for dynamic rate control, and a semantic decoding module for map restoration. This effectively improves the compression efficiency and transmission robustness of the transmitting end in embodied multi-agent map sharing, thereby flexibly adapting to the map sharing needs of embodied multi-agents, enhancing the system's scalability and generalization ability, and further strengthening the collaborative capabilities of embodied multi-agents.

[0111] It should be understood that the various functional units of the system of the present invention can be implemented in hardware, software, firmware, or a combination thereof. For example, if implemented in hardware, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), or field-programmable gate arrays (FPGAs), etc. Furthermore, the various functional units of the system of the present invention can be integrated into a single module, or each functional unit can exist physically separately, or two or more functional units can be integrated into a single module.

[0112] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. However, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A rate-controllable semantic communication method for embodied multi-agent semantic sharing, characterized in that, Includes the following steps: S1: Project the map After semantic feature extraction, it is transformed into semantic feature representation. ; S2: semantic segmentation map After wavelet transform semantic feature extraction is converted into semantic feature representation ; S3: the semantic feature representation obtained in step S1 is processed or the semantic feature representation obtained in step S2 is processed a rate-controllable feature selection encoding is performed to obtain a transmission signal and sent to a wireless channel, wherein the rate factor is ; S4: receive signal after transmission through wireless channel performing semantic decoding to obtain a reconstructed projection map or a reconstructed semantic segmentation map Results.

2. The rate-controllable semantic communication method for embodied multi-agent semantic sharing according to claim 1, wherein, Step S1 uses a four-layer cascaded convolution residual block module to project the map Semantic extraction is performed, and the mathematical expression is: ， in, It is a semantic feature representation; It is a semantic feature extractor based on convolutional residual blocks.

3. The rate-controllable semantic communication method for embodied multi-agent semantic sharing according to claim 1, wherein, Step S2 uses a four-layer cascaded wavelet transform convolutional residual block module to segment the semantic map. The mathematical expression for wavelet transform semantic extraction is as follows: ， wherein, is a semantic feature representation; is a semantic feature extractor based on wavelet transform convolution residual block.

4. The rate-controllable semantic communication method for embodied multi-agent semantic sharing according to claim 1, characterized in that, In step S2, the implementation process of wavelet transform semantic feature extraction is as follows: S2.1: Convolve the input of the wavelet transform The transformation result is obtained by wavelet transform. , , , ,in These are the low-frequency components of the wavelet transform; , , These are the high-frequency components of the wavelet transform; S2.2: low frequency components are removed The transform result is obtained by wavelet transform , , , ; S2.3: Transformation results , , , Perform a convolution operation, followed by an inverse wavelet transform, to obtain the wavelet transform convolution result. ; S2.4: To , , , Perform a convolution operation and combine the convolution result with the wavelet transform convolution result. Add them together to get the convolution sum result. , , , ; S2.5: For , , , Perform an inverse wavelet transform to obtain the wavelet transform convolution result. ; S2.6: For input Perform a convolution operation and combine the convolution result with the wavelet transform convolution result. Add them together to get the final output. .

5. The rate-controllable semantic communication method for embodied multi-agent semantic sharing according to claim 1, wherein, The specific implementation process of step S3 is as follows: S3.1: Representation of semantic features or Channel importance is evaluated using an attention-based mechanism to obtain a channel importance weight vector. ; S3.2: according to the channel importance weight vector and the rate factor reserve the top important channels in the semantic feature representation, where the ratio of the number of reserved channels to the total number of channels in the semantic feature representation equals ; S3.3: Perform channel pruning on the remaining channels to obtain the transmitted signal. ; Its mathematical expression is: ， in, It is a semantic feature representation or ; It is a rate-controllable feature selection encoder based on the attention mechanism.

6. The rate-controllable semantic communication method for embodied multi-agent semantic sharing according to claim 1, wherein, The specific implementation process of step S4 is as follows: A four-layered cascaded deconvolution block is used to process the received signal for decoding, which is mathematically expressed as ， wherein, is a semantic decoder, is a reconstructed projection map or a reconstructed semantic segmentation map .

7. The rate-controllable semantic communication method for embodied multi-agent semantic sharing according to claim 1, wherein, It also includes training step S5, which involves projecting the map. Semantic segmentation graph Rate factor Reconstructing the projected map and reconstructing semantic segmentation graphs The model is trained using the training set, and the trained model is saved after training for map sharing. The model includes: a semantic feature extractor, a wavelet transform semantic feature extractor, a rate-controlled feature selection encoder, and a semantic decoder.

8. The rate-controllable semantic communication method for embodied multi-agent semantic sharing according to claim 7, wherein, The specific implementation process of step S5 training is as follows: S5.1 : training a semantic feature extractor , a rate-controllable feature selection encoder and a semantic decoder ; S5.2: training the wavelet transform semantic feature extractor , rate-controllable feature selection encoder and semantic decoder ; S5.3: Freeze the semantic feature extractors obtained in S5.1 and S5.

2. and wavelet transform semantic feature extractor The parameters, mixed data retraining rate controllable feature selection encoder and semantic decoder ; S5.4: Freeze the rate-controllable feature selection encoder obtained in S5.3 and semantic decoder The parameters were then fine-tuned again for the semantic feature extractor. Wavelet transform semantic feature extractor Save the training model.

9. A rate-controllable semantic communication system for embodied multi-agent semantic sharing, characterized in that, For performing the method as described in any one of claims 1 to 8, comprising: semantic feature extractor: for encoding the projected map into semantic feature representations ; wavelet transform semantic feature extractor: for converting semantic segmentation maps encoding into semantic feature representations ; Rate-controllable feature selection encoder: used to adjust the sending rate, pass the semantic feature representation through channel importance evaluation and channel pruning to obtain the transmission signal ; wireless channel for receiving a transmission signal and outputting a received signal ; Semantic decoder: used to decode received signals Semantic encoding is performed to obtain the reconstructed projection map recovered after wireless channel transmission. Or reconstruct the semantic segmentation graph .

Images