Knowledge base enhanced semantic communication method and electronic equipment
By judging channel parameter characteristics in the channel knowledge base and using indexed transmission, combined with source-channel joint coding and error control, the communication cliff effect and bandwidth occupation caused by the decrease in signal-to-noise ratio are solved, and accurate transmission and recovery of information are achieved.
Patent Information
- Application Number
- CN202510842191.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-23
- Publication Date
- 2025-11-18
AI Technical Summary
Existing technologies suffer from a communication cliff effect when the signal-to-noise ratio decreases, and joint source-channel coding increases channel bandwidth resource consumption or reduces the accuracy of transmitted information due to limitations in the number of semantic knowledge bases.
By receiving the semantic feature map of the target image, the channel parameter features are determined and input into a pre-trained channel knowledge base. It is then determined whether to use indexed transmission. Combined with source-channel joint coding and error control, accurate information transmission is achieved.
Mitigating the cliff effect under low signal-to-noise ratio conditions, reducing transmission bandwidth usage, and ensuring accurate information recovery.
Smart Images
Figure CN120979602A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of communication technology, and in particular to a knowledge base-enhanced semantic communication method and electronic device. Background Technology
[0002] Semantic communication has been proven to achieve efficient information transmission in multiple scenarios, while overcoming the "cliff effect" caused by the decrease in signal-to-noise ratio in traditional communication. As a form of prior knowledge, semantic knowledge bases can further enhance the performance of system transmission by aligning knowledge at the sending and receiving ends and integrating this prior information into the semantic encoding and decoding processes of information.
[0003] Conventional source-channel joint coding increases the channel bandwidth resources used; and the semantic knowledge base obtained through learning and training will reduce the accuracy of transmitted information due to its limited quantity. Summary of the Invention
[0004] In view of this, the purpose of this application is to propose a knowledge base-enhanced semantic communication method and electronic device to solve or partially solve the above-mentioned technical problems.
[0005] To achieve the above objectives, this application provides a knowledge base-enhanced semantic communication method applied at the sending end, the method comprising:
[0006] Receive a target image and determine the semantic feature map of the target image;
[0007] Determine channel parameter features based on the semantic feature map;
[0008] The channel parameter features are input into a pre-trained first channel knowledge base to obtain channel parameter constraint values;
[0009] The channel parameter features are judged using the channel parameter limit values to determine whether the feature points in the semantic feature map corresponding to the channel parameter features are transmitted using indexes, thereby obtaining indexed transmission data;
[0010] The semantic feature map is jointly encoded by source and channel to obtain a semantic codeword sequence, and the semantic codeword sequence is transmitted to the receiving end;
[0011] The index transmission data is transmitted to the receiving end through error control.
[0012] Based on the same inventive concept, this application also provides a knowledge base-enhanced semantic communication method, applied at a receiving end, the method comprising:
[0013] Receive semantic codeword sequences and index transmission data sent by the sending end;
[0014] The semantic codeword sequence is subjected to source-channel joint decoding to obtain a source-channel joint coded transmission feature map;
[0015] The corresponding target feature vector is determined based on the index transmission data, and the index transmission feature map is determined based on the target feature vector.
[0016] Obtain the signal-to-noise ratio during transmission;
[0017] The source-channel jointly coded transmission feature map and the indexed transmission feature map, along with the signal-to-noise ratio during transmission, are input into a bidirectional gated error correction module to obtain the error-corrected feature map.
[0018] The semantic recovery of the corrected feature map is performed to obtain the target image.
[0019] Based on the same inventive concept, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.
[0020] As can be seen from the above, the knowledge base-enhanced semantic communication method and electronic device provided in this application can determine channel parameter features based on the semantic feature map of the target image. These channel parameter features can then be input into a pre-trained first channel knowledge base to obtain corresponding channel parameter constraint values. The channel parameter constraint values are then used to constrain and judge the channel parameter features, determining whether the feature points corresponding to these features need to be transmitted using indexing, thereby identifying indexed transmission data that can be transmitted using indexing. The semantic feature map is then jointly encoded through the source and channel to obtain a semantic codeword sequence, which can then be transmitted to the receiving end via a traditional channel, enabling accurate information response. The indexed transmission data is transmitted using an error control method. Because the amount of indexed transmission data is relatively small, this reduces the required transmission bandwidth and effectively alleviates the cliff effect under low signal-to-noise ratio transmission. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1 This is a flowchart illustrating a semantic communication method for knowledge base enhancement applied to the sending end according to an embodiment of this application;
[0023] Figure 2 This is a flowchart illustrating a semantic communication method for knowledge base enhancement applied to the receiving end according to an embodiment of this application;
[0024] Figure 3 This is a schematic diagram of information interaction between the sending end and the receiving end in an embodiment of this application;
[0025] Figure 4 This is a schematic diagram of the error correction module of the bidirectional gating error correction module in an embodiment of this application;
[0026] Figure 5 This is a schematic diagram of the peak signal-to-noise ratio (PSNR) for quantitative analysis in an embodiment of this application.
[0027] Figure 6 This is a schematic diagram of the quantitative analysis of multi-scale structural similarity (MS-SSIM) in an embodiment of this application;
[0028] Figure 7 This is a diagram of JSCC (w / KB) transmission when SNR = 0dB, as shown in the qualitative analysis of this application embodiment.
[0029] Figure 8 This is a diagram of JSCC (w / o KB) transmission when SNR=0dB, as shown in the qualitative analysis of an embodiment of this application.
[0030] Figure 9 This is a structural block diagram of the transmitting end according to an embodiment of this application;
[0031] Figure 10 This is a structural block diagram of the receiving end according to an embodiment of this application;
[0032] Figure 11 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation
[0033] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.
[0034] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0035] Definitions:
[0036] PSNR: Peak signal-to-noise ratio.
[0037] MS-SSIM: Multi Scale Structural Similarity Index Measure.
[0038] CRC: Cyclic Redundancy Check.
[0039] LDPC: Low Density Parity Check Code, a block error correction code with a sparse parity check matrix.
[0040] SNR: Signal-to-Noise Ratio.
[0041] CSI: Channel State Information, which refers to the channel attribute parameters of a communication link.
[0042] In related technology 1, information is efficiently transmitted by performing semantic extraction and joint source-channel coding at the transmitting end, and joint source-channel decoding and semantic recovery at the receiving end. In related technology 2, information is transmitted using a semantic knowledge base, which is constructed as a codebook. The transmitting end obtains a feature map through semantic coding, and the feature map is sent to the semantic knowledge base for feature vector matching to obtain the matching index. During transmission, only the index needs to be transmitted, which greatly reduces the resources required for communication.
[0043] In related technology 1, a suitable source-channel joint coding structure needs to be designed to achieve lossless transmission and recovery of information as much as possible. Although this can improve the accuracy of information recovery, it may increase the channel bandwidth resources occupied. In related technology 2, a semantic knowledge base suitable for representing the training dataset needs to be iteratively generated during the training process. Although the transmission index can reduce communication resources to some extent, due to the limited number of codebooks, the matching process can only match similar vectors, thereby reducing the accuracy of information transmission under the semantic knowledge base.
[0044] The embodiments of this application will be described in detail below with reference to the accompanying drawings.
[0045] The knowledge base-enhanced semantic communication method proposed in the embodiments of this application is applied to the sending end, such as... Figure 1 Combination Figure 3 As shown, the method includes:
[0046] Step 101: Receive the target image and determine the semantic feature map of the target image.
[0047] In practice, the target image is input into the semantic extractor for semantic extraction to obtain a semantic feature map, which has dimensions of C×H×W.
[0048] Step 102: Determine the channel parameter features based on the semantic feature map.
[0049] In practice, the channel parameter characteristics are those related to channel transmission.
[0050] Step 103: Input the channel parameter features into the pre-trained first channel knowledge base to obtain the channel parameter constraint values.
[0051] Step 104: Use the channel parameter limit value to judge the channel parameter features, determine whether the feature points in the semantic feature map corresponding to the channel parameter features are transmitted using indexes, and obtain indexed transmission data.
[0052] In practice, the channel parameter characteristics are compared with the channel parameter limits. If the channel parameter characteristics meet the limits corresponding to the channel parameter limits, it is determined that the data can be transmitted using indexing; otherwise, it is determined that the data cannot be transmitted using indexing and can be transmitted using source-channel joint coding. The results of the determination of whether to use indexing for transmission are combined to obtain the indexed transmission data.
[0053] Step 105: Perform source-channel joint coding on the semantic feature map to obtain a semantic codeword sequence, and transmit the semantic codeword sequence to the receiving end.
[0054] In practice, the semantic feature map is processed by source-channel joint encoder to obtain semantic codeword sequence, which is then sent to the channel for transmission and sent to the receiving end.
[0055] Step 106: The index transmission data is transmitted to the receiving end through error control.
[0056] In practical implementation, the purpose of error control transmission is to ensure that the index transmission data is error-free as much as possible. Error control transmission includes at least one of the following: transmission methods with CRC checksum, transmission methods with retransmission mechanisms, transmission methods with high-code-rate LDPC, and transmission methods with low modulation order. Using error control to transmit index transmission data can achieve lossless transmission of index transmission data while effectively reducing the amount of data transmitted.
[0057] The above scheme allows for the determination of channel parameter features based on the semantic feature map of the target image. These features are then input into a pre-trained first channel knowledge base to obtain corresponding channel parameter constraint values. These constraint values are then used to determine whether the corresponding feature points require indexing for transmission, thus identifying indexed transmission data. The semantic feature map is then jointly encoded using the source and channel to obtain a semantic codeword sequence, which is transmitted to the receiver via a traditional channel, enabling accurate information delivery. The indexed transmission data is transmitted using error control methods. Because the indexed transmission data is relatively small, this reduces the bandwidth required and effectively mitigates the cliff effect in low signal-to-noise ratio transmission.
[0058] In practice, step 102 includes:
[0059] Step 1021: Input the semantic feature map into the entropy model, use the entropy model to determine the information entropy of each feature point in the semantic feature map, and determine the entropy map based on the information entropy.
[0060] In practice, the semantic feature map is fed into an entropy model to calculate the information entropy of each feature point in the semantic feature map. The dimensions of the information entropy are C×H×W. Specifically, the entropy model uses a neural network to estimate the mean and variance of each feature point and to estimate the probability distribution p of each feature point. The information entropy of each feature point is then calculated using -log2p. Simultaneously, the information entropies are summed along the C dimension to obtain the entropy map, which has dimensions H×W.
[0061] Step 1022: For each feature point in the semantic feature map, determine the similarity between the feature point and each feature vector in the source knowledge base, and select the feature vector in the source knowledge base that has the highest similarity to the feature point, as well as the index corresponding to the feature vector with the highest similarity.
[0062] In practice, the source knowledge base stores feature vectors of various dimensions C, and each feature vector has its corresponding index. For each feature point in the semantic feature map, the cosine similarity between the feature point and each feature vector in the source knowledge base is calculated. The feature vector with the highest cosine similarity is selected, and its corresponding index is determined.
[0063] Step 1023: After all feature points of the semantic feature map have been selected, a similarity map is determined based on the similarity value of the feature vector with the highest similarity to all feature points, and an index map is determined based on the index of the feature vector with the highest similarity to all feature points.
[0064] In practice, the highest cosine similarity values corresponding to each feature point are integrated to obtain a similarity map with dimensions H×W. The indices corresponding to each feature point are then integrated to obtain an index map with dimensions H×W.
[0065] Step 1024: Combine the entropy map and the similarity map to form the channel parameter features.
[0066] The above method can accurately determine the entropy map, similarity map, and index map.
[0067] In some embodiments, the channel parameter features include: an entropy map and a similarity map;
[0068] Step 103 includes:
[0069] Step 1031: Obtain channel state information.
[0070] Channel state information includes CSI, SNR, etc.
[0071] Step 1032: Input the channel state information, the entropy map, and the similarity map into the pre-trained first channel knowledge base to obtain the entropy threshold, the similarity threshold, and the index ratio, and use the entropy threshold, the similarity threshold, and the index ratio as the channel parameter limit values.
[0072] In practice, a large number of training samples labeled with "entropy threshold, similarity threshold, and index ratio" are pre-acquired, containing channel state information, entropy maps, and similarity maps. The pre-constructed initial neural network model is then supervisedly trained using these training samples to obtain a first channel knowledge base. This first channel knowledge base can then be used to accurately process the channel state information, entropy maps, and similarity maps, outputting accurate entropy thresholds, similarity thresholds, and index ratios to obtain the channel parameter limits.
[0073] The above method ensures the accuracy of the obtained entropy threshold, similarity threshold, and index ratio.
[0074] In some embodiments, step 104 includes:
[0075] Step 1041: For each feature point in the semantic feature map, in response to determining that the information entropy of the feature point in the entropy map is less than the entropy threshold, and the similarity value in the similarity map is greater than the similarity threshold, determine that the feature point is transmitted using an index;
[0076] Step 1042: For each feature point in the semantic feature map, in response to determining that the information entropy of the feature point in the entropy map is greater than or equal to the entropy threshold, or the similarity value in the similarity map is less than or equal to the similarity threshold, determine that the feature point will not be transmitted using an index.
[0077] Step 1043: Determine that the proportion of all feature points in the semantic feature map that are transmitted using an index is less than or equal to the index ratio, and mark the results of whether or not all feature points in the semantic feature map are transmitted using an index to form a marker map (e.g., a mask map). The positions in the marker map that are transmitted using an index are designated as the first marker (e.g., 0), and the positions that are not transmitted using an index are designated as the second marker (e.g., 1). The marker map has a dimension of H×W.
[0078] Step 1044: Remove the indices in the index map that are not used for transmission according to the marker map to obtain the transmission index.
[0079] In practice, the marker map is applied to the index map to remove positions that are not transmitted using the index, which reduces the amount of data transmitted and yields the transmission index.
[0080] Step 1045: Transmit data using the marker map and the transmission index as the index.
[0081] The above scheme ensures that the amount of index transmission data is as small as possible, thereby reducing the amount of index transmission data and lowering the bandwidth occupied by channel transmission.
[0082] In some embodiments, step 105 includes:
[0083] Step 1051: Input the marker map, the entropy map, and the channel state information into the pre-trained second channel knowledge base to obtain the encoding length corresponding to each feature point in the semantic feature map.
[0084] In practice, a large number of training samples containing label maps, entropy maps, and channel state information, labeled with "encoding length," are pre-acquired. The pre-constructed second initial neural network model is then trained in a supervised manner using these training samples to obtain a second channel knowledge base. This second channel knowledge base can then be used to accurately process the label maps, entropy maps, and channel state information, outputting the encoding length corresponding to each feature point. The encoding length for feature points corresponding to the second label symbol in the label map is a predetermined length (e.g., a predetermined length of 1). Since the feature points corresponding to the second label symbol do not require indexing for transmission, their corresponding encoding lengths are compressed to a minimum, thereby reducing the overall encoding length.
[0085] Step 1052: Input the semantic feature map into the source-channel joint encoder and encode it according to the coding length corresponding to each feature point in the semantic feature map to obtain the initial semantic codeword sequence.
[0086] In practice, based on the encoding length corresponding to each feature point in the semantic feature map obtained above, each feature point can be encoded based on the encoding length to obtain the initial semantic codeword sequence.
[0087] Step 1053: Normalize the initial semantic codeword sequence to obtain the semantic codeword sequence.
[0088] In practice, the initial semantic codeword sequence is normalized to obtain the final semantic codeword sequence, which is then transmitted to the receiving end through the channel.
[0089] The above scheme determines the coding length of each feature point through the second channel knowledge base, compresses the coding length of the feature points corresponding to the second marker symbol, and then uses the source-channel joint encoder to encode according to the coding length, thereby reducing the coding length of the initial semantic codeword sequence. In order to further ensure the efficiency of data transmission, normalization processing is performed to obtain a semantic codeword sequence with higher transmission efficiency.
[0090] Based on the same inventive concept, embodiments of this application propose a knowledge base-enhanced semantic communication method, applied to the receiving end, such as... Figure 2 Combination Figure 3 As shown, the method includes:
[0091] Step 201: Receive the semantic codeword sequence and index transmission data sent by the sending end.
[0092] Step 202: Perform source-channel joint decoding on the semantic codeword sequence to obtain a source-channel joint coded transmission feature map.
[0093] In practice, source-channel joint decoding is a decoding process corresponding to source-channel joint coding at the transmitting end. Therefore, by performing source-channel joint decoding on the semantic codeword sequence, the source-channel joint coded transmission feature map can be obtained.
[0094] Step 203: Determine the corresponding target feature vector based on the index transmission data, and determine the index transmission feature map based on the target feature vector.
[0095] In practice, the index transmission data is used to determine multiple target feature vectors by using a receiver source knowledge base that is aligned with the sender's source knowledge base. These target feature vectors are then fused to obtain the index transmission feature map.
[0096] Step 204: Obtain the signal-to-noise ratio (e.g., SNR) during transmission.
[0097] Step 205: Input the source-channel jointly coded transmission feature map, the indexed transmission feature map, and the signal-to-noise ratio during transmission into the bidirectional gated error correction module to obtain the error-corrected feature map.
[0098] In practice, the bidirectional gated error correction module includes two gated modules, which respectively perform error correction processing on the source-channel joint coded transmission feature map and the indexed transmission feature map, and then combine them to obtain the error-corrected feature map.
[0099] Step 206: Perform semantic recovery on the corrected feature map to obtain the target image.
[0100] In practice, the corrected feature map can contain accurate feature data of the image, and after semantic recovery, an accurate target image can be obtained.
[0101] Through the above scheme, the receiving end can obtain an accurate source-channel joint coded transmission feature map by jointly decoding the received semantic Mazu sequence. It can also determine the index transmission feature map by performing feature correspondence on the index transmission data, and then use a bidirectional gated error correction module to perform error correction. The resulting error-corrected feature map can contain accurate feature data of the image. In this way, after semantic recovery, an accurate target image can be obtained.
[0102] In some embodiments, the index transfer data includes: a marker graph (e.g., a mask graph);
[0103] Step 202 includes:
[0104] Step 2021: Input the semantic codeword sequence into the source-channel joint decoder to perform source-channel joint decoding and obtain the semantic feature map.
[0105] In practice, the source-channel joint decoder decodes the semantic codeword sequence according to the decoding process corresponding to the encoding process to obtain a semantic feature map.
[0106] Step 2022: Multiply the semantic feature map with the label map to obtain the source-channel joint coding transmission feature map.
[0107] Using the above scheme, the semantic feature map obtained after decoding contains features that do not use source-channel joint coding. The label map marks which features are transmitted through the index and which are transmitted through the source channel. Therefore, by multiplying the semantic feature map and the label map, an accurate source-channel joint coding transmission feature map can be obtained.
[0108] In some embodiments, the index transmission data includes: a marker map and a transmission index.
[0109] Step 203 includes:
[0110] Step 2031: Input the transmission index into the information source knowledge base to obtain the feature vector corresponding to the transmission index.
[0111] In practice, the source knowledge base at the receiving end is a database that is aligned with the source knowledge base at the sending end. Therefore, by inputting the transmission index into the source knowledge base, the target feature vector corresponding to the transmission index can be determined, and this target feature vector is consistent with the feature vector corresponding to the index map at the sending end.
[0112] Step 2032: Place the position corresponding to the first marker in the marker map into the corresponding target feature vector, and fill the position corresponding to the second marker in the marker map with empty values to obtain the index transmission feature map.
[0113] In practice, since the placement position of the target feature vector obtained above is uncertain, a marker map is used for assistance. The target feature vector corresponding to the first marker (e.g., 0) is placed in the marker map, and the position corresponding to the second marker (e.g., 1) is filled with a null value (e.g., 0). This can obtain a more accurate index transfer feature map.
[0114] In some embodiments, the index transfer data includes: a marker map.
[0115] like Figure 4As shown, step 205 includes:
[0116] Step 2051: Input the marker map, the source-channel joint coding transmission feature map, and the index transmission feature map into the first conditional error correction module for error correction processing to obtain the source-channel joint coding feature error map.
[0117] The first conditional error correction module uses the indexed transmission feature map as a condition to correct the error of the source-channel joint coding transmission feature map according to the marker map, thereby obtaining an accurate source-channel joint coding feature error map.
[0118] Step 2052: Invert the marker image (e.g., change 0 to 1 and 1 to 0) to obtain the inverted marker image.
[0119] Step 2053: Input the inverted marker map, the source-channel joint coding transmission feature map, and the index transmission feature map into the second conditional error correction module for error correction processing to obtain the index feature error map.
[0120] The second conditional error correction module uses the source-channel jointly coded transmission feature map as a condition, and corrects the error of the index transmission feature map based on the inverted marker map to obtain an accurate index feature error map.
[0121] Step 2054: Input the source-channel joint coding transmission feature map, the index transmission feature map, and the signal-to-noise ratio during transmission into the first gating module to obtain the first gating coefficient.
[0122] Step 2055: Input the source-channel joint coding transmission feature map, the index transmission feature map, and the signal-to-noise ratio during transmission into the second gating module to obtain the second gating coefficient.
[0123] Step 2056: Multiply the first gating coefficient by the source-channel joint coding feature error map and then by the label map to obtain the labeled source-channel joint coding feature error map.
[0124] Step 2057: Multiply the second gating coefficient by the index feature error map and then by the inverted label map to obtain the labeled index feature error map.
[0125] Step 2058: Add the marked source-channel joint coding feature error map to the source-channel joint coding transmission feature map to obtain the corrected source-channel joint coding feature map.
[0126] Step 2059: Add the marked index feature error map to the index transfer feature map to obtain the corrected index feature map.
[0127] Step 20510: Add the corrected source-channel joint coding feature map to the corrected index feature map to obtain the error-corrected feature map.
[0128] The above method ensures that the corrected feature map is more accurate.
[0129] Based on the solutions described in the above embodiments, an effectiveness analysis is performed on the information source knowledge base involved in this application:
[0130] Comparison with reference schemes: (1) JSCC (w / KB) represents a source-channel joint coding transmission scheme with the assistance of a source knowledge base; (2) JSCC (w / o KB) represents a source-channel joint coding transmission scheme without the assistance of a source knowledge base; (3) the traditional communication scheme BPG+2 / 3LDPC+QPSK; (4) the traditional communication scheme BPG+2 / 3LDPC+16QAM.
[0131] like Figure 5 and Figure 6 As shown, quantitative analysis reveals that although JSCC(w / KB) exhibits some loss and gap compared to JSCC(w / o KB) at high signal-to-noise ratios, JSCC(w / KB) can still guarantee higher recovery quality at extremely low signal-to-noise ratios, thus enhancing the robustness of the communication system.
[0132] like Figure 7 and Figure 8 As shown, using qualitative analysis, at a signal-to-noise ratio of 0dB, the image transmitted by JSCC (w / KB) has clearer texture and higher restoration quality compared to the image transmitted by JSCC (w / o KB).
[0133] Combination Figures 5 to 8 It can be seen that the source-channel joint coding transmission scheme assisted by the source knowledge base has stronger noise resistance.
[0134] Based on the scheme described in the above embodiments, an effect analysis is performed on the channel knowledge base (including the first channel knowledge base and the second channel knowledge base) involved in this application:
[0135] For the first channel knowledge base, if the information entropy of the feature vector of a certain feature point is smaller and the matching similarity is higher, the channel knowledge base will transmit the feature vector of that feature point using an index; otherwise, it will transmit it using joint source-channel coding. This means that if the semantic importance of a certain feature point is lower and the loss in representing the feature vector by the source knowledge base at the receiving end is smaller, then the feature vector of that feature point is more likely to be transmitted using a lossy index, thereby reducing the amount of information transmitted. Conversely, if the semantic importance of a certain feature point is higher and the loss in representing the vector by the source knowledge base at the receiving end is greater, then the feature vector of that feature point is more likely to be transmitted using lossless source-channel joint coding, but this will increase the amount of information transmitted.
[0136] For the second channel knowledge base, it can determine the encoding length based on the current channel state, allowing feature vectors with higher information entropy to obtain longer encoding lengths, and vice versa.
[0137] Based on the solutions described in the above embodiments, the effectiveness of the bidirectional gating error correction module involved in this application is analyzed:
[0138] The bidirectional gated error correction module can utilize the semantic correlation between the source-channel jointly coded transmission feature map and the index transmission feature map for error correction. When the SNR is high and the channel quality is good, the source-channel jointly coded transmission feature should be trusted more, thereby using the source-channel jointly coded transmission feature to correct the feature errors existing in the index transmission feature; when the SNR is low and the channel quality is poor, the index transmission feature should be trusted more, thereby using the index transmission feature to correct the feature errors existing in the source-channel jointly coded transmission feature.
[0139] The knowledge base-enhanced semantic communication method based on this application (applied to both the sender and receiver) is analyzed using specific embodiments:
[0140] Example 1: Using the sending end to transmit post-disaster drone images of mountainous areas back to the receiving end.
[0141] Note: After earthquakes or mudslides, roads are blocked and base stations are damaged. Drones must rapidly transmit high-resolution visible light and infrared images back to the rescue command center for disaster assessment and search and rescue of trapped personnel. However, mountainous areas suffer from severe channel fading and limited bandwidth, making it difficult to guarantee the complete transmission of critical images using traditional communication methods. The knowledge-based enhanced semantic communication method proposed in this application can significantly reduce the bit error rate and improve the robustness of system transmission under weak link conditions.
[0142] Specific implementation steps:
[0143] The drone (i.e., the transmitter) first uses each frame of image as the target image, executes the above scheme to calculate the information entropy of each feature point to obtain an entropy map, and then performs cosine similarity matching between the feature vectors corresponding to all feature points and the feature vectors of the information source knowledge base stored in the drone (i.e., the transmitter) to obtain a similarity map and an index map.
[0144] Then, the channel state information (CSI, SNR, etc.) measured in real time is compared with the entropy map and similarity. Figure 1 It is then fed into the first channel knowledge base, and the output includes the proportion of available indexes, entropy threshold, and similarity threshold.
[0145] Feature points whose information entropy is less than the entropy threshold and whose similarity value is greater than the similarity threshold are transmitted using an index, while the remaining points are transmitted using a source-channel joint coding method, thus obtaining a labeled map. The labeled map is applied to the index map to form a transmission index, while the semantic feature map (the semantic feature map obtained from the target image) is sent to the source-channel joint encoder to obtain a semantic codeword sequence.
[0146] Then, the label map, entropy map, and channel state information are input into the second channel knowledge base to determine the coding length of each feature point (where the coding length of the feature points to be indexed is compressed), and a power-normalized semantic codeword sequence is generated.
[0147] This allows for the transmission of small-volume labeled graphs and transmission indexes, while large-volume semantic codeword sequences containing partial semantic features are transmitted directly.
[0148] After receiving the data, the command center (i.e., the receiving end) first recovers the first part of the features (the source-channel jointly encoded transmission feature map) by jointly decoding the semantic codeword sequence. Then, it uses the source knowledge base aligned with the transmission index at the receiving end to determine the second part of the features (the indexed transmission feature map). Both of these features, along with the labels, are then combined. Figure 1 The bidirectional gated error correction module outputs a complete error-corrected feature map, which is then semantically restored by the semantic recovery module to obtain the target image of the disaster area, thus achieving reliable image backhaul under weak channels.
[0149] Example 2: Using the transmitting end to transmit polar scientific research images back to the receiving end.
[0150] Note: The polar monitoring station (i.e., the transmitting end) needs to transmit high-resolution visible light and infrared images of glaciers to an inland research center (i.e., the receiving end) in real time via a low-elevation satellite link for ice movement analysis. The polar environment often experiences extremely low signal-to-noise ratios, frequent fading, and bandwidth jitter, making traditional communication solutions difficult to operate stably. This application achieves reliable image transmission by using differentiated transmission semantic features.
[0151] Specific implementation steps:
[0152] The monitoring station (i.e., the transmitting end) performs the above scheme to calculate the information entropy of each feature point for each frame of image (i.e., the target image) to obtain an entropy map. Then, it performs cosine similarity matching between the feature vectors corresponding to all feature points and the feature vectors of the information source knowledge base stored in the monitoring station (i.e., the transmitting end) to obtain a similarity map and an index map.
[0153] Then, channel state information is obtained by real-time detection of satellite channel CSI and current polar low SNR. This channel state information is then compared with entropy maps and similarity. Figure 1 It is then fed into the first channel knowledge base, and the output includes the proportion of available indexes, entropy threshold, and similarity threshold.
[0154] Feature points whose information entropy is less than the entropy threshold and whose similarity value is greater than the similarity threshold are transmitted using an index, while the remaining points are transmitted using a source-channel joint coding method, thus obtaining a labeled map. The labeled map is applied to the index map to form a transmission index, while the semantic feature map (the semantic feature map obtained from the target image) is sent to the source-channel joint encoder to obtain a semantic codeword sequence.
[0155] Then, the label map, entropy map, and channel state information are input into the second channel knowledge base to determine the coding length of each feature point (where the coding length of the feature points to be indexed is compressed), and a power-normalized semantic codeword sequence is generated.
[0156] This allows for the transmission of small-volume labeled graphs and transmission indexes, while large-volume semantic codeword sequences containing partial semantic features are transmitted directly.
[0157] After receiving the data, the research center (i.e., the receiving end) first recovers the first part of the features (the source-channel jointly encoded transmission feature map) by jointly decoding the semantic codeword sequence. Then, it uses the source knowledge base aligned with the transmission index at the receiving end to determine the second part of the features (the indexed transmission feature map). Both of these features, along with the labels, are then combined. Figure 1 The bidirectional gated error correction module outputs a complete error-corrected feature map, which is then semantically restored by the semantic recovery module to obtain the target image of the glacier cracks, thus achieving stable high-resolution image backhaul under the low signal-to-noise ratio link in the polar region.
[0158] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.
[0159] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0160] Based on the same inventive concept, corresponding to the knowledge base-enhanced semantic communication method applied to any of the above embodiments of the sending end, this application also provides a sending end.
[0161] refer to Figure 9 The transmitting end includes:
[0162] The semantic analysis module 301 is configured to receive a target image and determine the semantic feature map of the target image;
[0163] The channel parameter feature determination module 302 is configured to determine channel parameter features based on the semantic feature map;
[0164] The limit determination module 303 is configured to input the channel parameter features into a pre-trained first channel knowledge base to obtain channel parameter limit values;
[0165] The index transmission data determination module 304 is configured to use the channel parameter limit value to determine whether the feature points in the semantic feature map corresponding to the channel parameter features are transmitted using an index, thereby obtaining index transmission data.
[0166] The joint coding and transmission module 305 is configured to perform source-channel joint coding on the semantic feature map to obtain a semantic codeword sequence, and transmit the semantic codeword sequence to the receiving end;
[0167] The index transmission module 306 is configured to transmit the index transmission data to the receiving end through an error control method.
[0168] In some embodiments, the channel parameter feature determination module 302 is specifically configured as follows:
[0169] The semantic feature map is input into the entropy model, and the information entropy of each feature point in the semantic feature map is determined using the entropy model. The entropy map is then determined based on the information entropy.
[0170] For each feature point in the semantic feature map, determine the similarity between the feature point and each feature vector in the source knowledge base, and select the feature vector in the source knowledge base that has the highest similarity to the feature point, as well as the index corresponding to the feature vector with the highest similarity.
[0171] After all feature points of the semantic feature map have been selected, a similarity map is determined based on the similarity value of the feature vector with the highest similarity to all feature points, and an index map is determined based on the index of the feature vector with the highest similarity to all feature points.
[0172] The entropy map and the similarity map are combined to form the channel parameter features.
[0173] In some embodiments, the channel parameter features include: an entropy map and a similarity map;
[0174] The limit determination module 303 is specifically configured as follows:
[0175] Obtain channel state information;
[0176] The channel state information, the entropy map, and the similarity map are input into a pre-trained first channel knowledge base to obtain an entropy threshold, a similarity threshold, and an index ratio. The entropy threshold, the similarity threshold, and the index ratio are then used as the channel parameter limit values.
[0177] In some embodiments, the index transfer data determination module 304 is specifically configured as follows:
[0178] For each feature point in the semantic feature map,
[0179] In response to determining that the information entropy of the feature point in the entropy map is less than the entropy threshold, and the similarity value in the similarity map is greater than the similarity threshold, it is determined that the feature point is transmitted using an index;
[0180] In response to determining that the information entropy of the feature point in the entropy map is greater than or equal to the entropy threshold, or that the similarity value in the similarity map is less than or equal to the similarity threshold, it is determined that the feature point will not be transmitted using an index;
[0181] Determine that the proportion of all feature points in the semantic feature map that are transmitted using an index is less than or equal to the index ratio, and mark the results of whether or not all feature points in the semantic feature map are transmitted using an index to form a marking map. In the marking map, the position that is transmitted using an index is the first marker, and the position that is not transmitted using an index is the second marker.
[0182] Based on the marker map, remove the indices in the index map that are not used for transmission to obtain the transmission index;
[0183] The marker map and the transmission index are used as the index to transmit data.
[0184] In some embodiments, the joint encoding transmission module 305 is specifically configured as follows:
[0185] The label map, the entropy map, and the channel state information are input into a pre-trained second channel knowledge base to obtain the encoding length corresponding to each feature point in the semantic feature map;
[0186] The semantic feature map is input into the source-channel joint encoder and encoded according to the coding length corresponding to each feature point in the semantic feature map to obtain the initial semantic codeword sequence.
[0187] The initial semantic codeword sequence is normalized to obtain the semantic codeword sequence.
[0188] Based on the same inventive concept, corresponding to the knowledge base-enhanced semantic communication method applied to any of the above embodiments of the receiving end, this application also provides a receiving end.
[0189] refer to Figure 10 The receiving end includes:
[0190] The receiving module 401 is configured to receive semantic codeword sequences and index transmission data sent by the sending end;
[0191] The joint decoding module 402 is configured to perform source-channel joint decoding on the semantic codeword sequence to obtain a source-channel joint coded transmission feature map.
[0192] The index determination module 403 is configured to determine the corresponding target feature vector based on the index transmission data, and to determine the index transmission feature map based on the target feature vector.
[0193] The signal-to-noise ratio acquisition module 404 is configured to acquire the signal-to-noise ratio during transmission.
[0194] The error correction module 405 is configured to input the source channel jointly encoded transmission feature map and the index transmission feature map, as well as the signal-to-noise ratio during transmission, into a bidirectional gated error correction module to obtain an error-corrected feature map.
[0195] The semantic recovery module 406 is configured to perform semantic recovery on the corrected feature map to obtain the target image.
[0196] In some embodiments, the index transfer data includes: a marker map;
[0197] The joint decoding module 402 is specifically configured as follows:
[0198] The semantic codeword sequence is input into the source-channel joint decoder for joint source-channel decoding to obtain a semantic feature map.
[0199] The semantic feature map is multiplied by the label map to obtain the source-channel joint coding transmission feature map.
[0200] In some embodiments, the index transmission data includes: a marker map and a transmission index;
[0201] The index determination module 403 is specifically configured as follows:
[0202] The transmission index is input into the source knowledge base to obtain the target feature vector corresponding to the transmission index;
[0203] The position corresponding to the first marker in the marker map is placed into the corresponding target feature vector, and the position corresponding to the second marker in the marker map is filled with empty values to obtain the indexed transfer feature map.
[0204] In some embodiments, the index transfer data includes: a marker map;
[0205] Error correction module 405 is specifically configured as follows:
[0206] The marker map, the source-channel joint coding transmission feature map, and the index transmission feature map are input into the first conditional error correction module for error correction processing to obtain the source-channel joint coding feature error map.
[0207] The marked image is inverted to obtain the inverted marked image;
[0208] The inverted marker map, the source-channel joint coded transmission feature map, and the index transmission feature map are input into the second conditional error correction module for error correction processing to obtain the index feature error map.
[0209] The source-channel joint coded transmission feature map, the indexed transmission feature map, and the signal-to-noise ratio during transmission are input to the first gating module to obtain the first gating coefficient;
[0210] The source-channel joint coded transmission feature map, the indexed transmission feature map, and the signal-to-noise ratio during transmission are input to the second gating module to obtain the second gating coefficients.
[0211] Multiply the first gating coefficient by the source-channel joint coding feature error map, and then multiply by the label map to obtain the labeled source-channel joint coding feature error map;
[0212] Multiply the second gating coefficient by the index feature error map, and then multiply it by the inverted label map to obtain the labeled index feature error map;
[0213] The marked source-channel joint coding feature error map is added to the source-channel joint coding transmission feature map to obtain the corrected source-channel joint coding feature map;
[0214] The marked index feature error map is added to the index transfer feature map to obtain the corrected index feature map;
[0215] The corrected source-channel joint coding feature map is added to the corrected index feature map to obtain the error-corrected feature map.
[0216] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.
[0217] The apparatus of the above embodiments is used to implement the corresponding method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0218] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the methods described in any of the above embodiments.
[0219] Figure 11 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.
[0220] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.
[0221] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.
[0222] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.
[0223] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).
[0224] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.
[0225] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.
[0226] The electronic devices described above are used to implement the corresponding methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0227] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform the methods described in any of the above embodiments.
[0228] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital video disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.
[0229] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the methods described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0230] Based on the same concept, corresponding to any of the above embodiments, this application also provides a computer program product, including computer program instructions, which, when run on a computer, cause the computer to perform the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0231] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.
[0232] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0233] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.
[0234] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.
Claims
1. A knowledge base-enhanced semantic communication method, characterized in that, Applied to the sending end, the method includes: Receive a target image and determine the semantic feature map of the target image; Determine channel parameter features based on the semantic feature map; The channel parameter features are input into a pre-trained first channel knowledge base to obtain channel parameter constraint values; The channel parameter features are judged using the channel parameter limit values to determine whether the feature points in the semantic feature map corresponding to the channel parameter features are transmitted using indexes, thereby obtaining indexed transmission data; The semantic feature map is jointly encoded by source and channel to obtain a semantic codeword sequence, and the semantic codeword sequence is transmitted to the receiving end; The index transmission data is transmitted to the receiving end through error control.
2. The method according to claim 1, characterized in that, Determining channel parameter features based on the semantic feature map includes: The semantic feature map is input into the entropy model, and the information entropy of each feature point in the semantic feature map is determined using the entropy model. The entropy map is then determined based on the information entropy. For each feature point in the semantic feature map, determine the similarity between the feature point and each feature vector in the source knowledge base, and select the feature vector in the source knowledge base that has the highest similarity to the feature point, as well as the index corresponding to the feature vector with the highest similarity. After all feature points of the semantic feature map have been selected, a similarity map is determined based on the similarity value of the feature vector with the highest similarity to all feature points, and an index map is determined based on the index of the feature vector with the highest similarity to all feature points. The entropy map and the similarity map are combined to form the channel parameter features.
3. The method according to claim 2, characterized in that, The channel parameter features include: entropy map and similarity map; The step of inputting the channel parameter features into a pre-trained first channel knowledge base to obtain channel parameter constraint values includes: Obtain channel state information; The channel state information, the entropy map, and the similarity map are input into a pre-trained first channel knowledge base to obtain an entropy threshold, a similarity threshold, and an index ratio. The entropy threshold, the similarity threshold, and the index ratio are then used as the channel parameter limit values.
4. The method according to claim 3, characterized in that, The step of using the channel parameter limitation value to determine whether the feature points in the semantic feature map corresponding to the channel parameter features are transmitted using an index, and obtaining indexed transmission data, includes: For each feature point in the semantic feature map, In response to determining that the information entropy of the feature point in the entropy map is less than the entropy threshold, and the similarity value in the similarity map is greater than the similarity threshold, it is determined that the feature point is transmitted using an index; In response to determining that the information entropy of the feature point in the entropy map is greater than or equal to the entropy threshold, or the similarity value in the similarity map is less than or equal to the similarity threshold, it is determined that the feature point will not be transmitted using an index; it is determined that the proportion of all feature points in the semantic feature map that are transmitted using an index is less than or equal to the index ratio, and the results of whether or not all feature points in the semantic feature map are transmitted using an index are marked to form a marking map, wherein the position in the marking map that is transmitted using an index is the first marker, and the position that is not transmitted using an index is the second marker; Based on the marker map, remove the indices in the index map that are not used for transmission to obtain the transmission index; The marker map and the transmission index are used as the index to transmit data.
5. The method according to claim 4, characterized in that, The step of performing source-channel joint coding on the semantic feature map to obtain a semantic codeword sequence includes: The label map, the entropy map, and the channel state information are input into a pre-trained second channel knowledge base to obtain the encoding length corresponding to each feature point in the semantic feature map; The semantic feature map is input into the source-channel joint encoder and encoded according to the coding length corresponding to each feature point in the semantic feature map to obtain the initial semantic codeword sequence. The initial semantic codeword sequence is normalized to obtain the semantic codeword sequence.
6. A knowledge base-enhanced semantic communication method, characterized in that, Applied to the receiving end, the method includes: Receive semantic codeword sequences and index transmission data sent by the sending end; The semantic codeword sequence is subjected to source-channel joint decoding to obtain a source-channel joint coded transmission feature map; The corresponding target feature vector is determined based on the index transmission data, and the index transmission feature map is determined based on the target feature vector. Obtain the signal-to-noise ratio during transmission; The source-channel jointly coded transmission feature map and the indexed transmission feature map, along with the signal-to-noise ratio during transmission, are input into a bidirectional gated error correction module to obtain the error-corrected feature map. The semantic recovery of the corrected feature map is performed to obtain the target image.
7. The method according to claim 6, characterized in that, The index transmission data includes: a marker map; The step of performing source-channel joint decoding on the semantic codeword sequence to obtain the source-channel joint coded transmission feature map includes: The semantic codeword sequence is input into the source-channel joint decoder for joint source-channel decoding to obtain a semantic feature map. The semantic feature map is multiplied by the label map to obtain the source-channel joint coding transmission feature map.
8. The method according to claim 6, characterized in that, The index transmission data includes: a marker map and a transmission index; The step of determining the corresponding target feature vector based on the index transmission data and determining the index transmission feature map based on the target feature vector includes: The transmission index is input into the source knowledge base to obtain the target feature vector corresponding to the transmission index; The position corresponding to the first marker in the marker map is placed into the corresponding target feature vector, and the position corresponding to the second marker in the marker map is filled with empty values to obtain the indexed transfer feature map.
9. The method according to claim 6, characterized in that, The index transmission data includes: a marker map; The step of inputting the jointly coded transmission feature map of the source channel, the indexed transmission feature map, and the signal-to-noise ratio during transmission into a bidirectional gated error correction module to obtain the error-corrected feature map includes: The marker map, the source-channel joint coding transmission feature map, and the index transmission feature map are input into the first conditional error correction module for error correction processing to obtain the source-channel joint coding feature error map. The marked image is inverted to obtain the inverted marked image; The inverted marker map, the source-channel joint coded transmission feature map, and the index transmission feature map are input into the second conditional error correction module for error correction processing to obtain the index feature error map. The source-channel joint coded transmission feature map, the indexed transmission feature map, and the signal-to-noise ratio during transmission are input to the first gating module to obtain the first gating coefficient; The source-channel joint coded transmission feature map, the indexed transmission feature map, and the signal-to-noise ratio during transmission are input to the second gating module to obtain the second gating coefficients. Multiply the first gating coefficient by the source-channel joint coding feature error map, and then multiply by the label map to obtain the labeled source-channel joint coding feature error map; Multiply the second gating coefficient by the index feature error map, and then multiply it by the inverted label map to obtain the labeled index feature error map; The marked source-channel joint coding feature error map is added to the source-channel joint coding transmission feature map to obtain the corrected source-channel joint coding feature map; The marked index feature error map is added to the index transfer feature map to obtain the corrected index feature map; The corrected source-channel joint coding feature map is added to the corrected index feature map to obtain the error-corrected feature map.
10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 9.