Attention-based joint source channel method for text

A technology of attention and text, applied in the field of wireless communication, can solve problems such as inability to adapt to complex network environments

CN113300813AActive Publication Date: 2021-08-24CENT SOUTH UNIV
5 Cites 3 Cited by

Patent Information

Authority / Receiving Office
CN · China
Current Assignee / Owner
Publication Date
2021-08-24

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses an attention-based joint source channel method for a text, which uses a generator comprising an encoder, a channel and a decoder to carry out joint source channel coding on the text, and uses a GRU to replace an LSTM, so that the calculation speed is higher. Meanwhile, an attention mechanism is added, so that the decoding efficiency and accuracy are improved. By using the GAN framework, the parameter update of the generated model is not directly from an original data sample but from the back propagation of a discrimination model. According to the method, a one-way text decoding framework based on GRU is used as a generation model, a CNN is used as a discrimination model, and samples generated by a generator are discriminated. Point mutual information is added in beam search, and words are decoded more accurately together with the maximum likelihood probability. At a decoding end, a synchronous bidirectional decoding method is used. Meanwhile, L2R and R2L decoding is carried out, and alignment and information interaction are realized by utilizing SBAtt and bidirectional beam search, that is, past and future information is utilized at the same time, so that the word error rate of decoding is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the field of wireless communication, in particular to an attention-based joint source-channel method for text. Background technique

[0002] With the development of computer technology, modern communication technology and network information processing technology, text has become the main carrier for users to obtain and disseminate information. The encoding and transmission of text data also faces great challenges. On the one hand, in many application occasions, there will be a problem of needing to transmit massive text data information. In order to transmit a large amount of information in a channel with limited bandwidth, high-efficiency compression coding must be performed before transmission. On the other hand, due to the increasing complexity of wireless channels, how to effectively deal with the impact of channels on text transmission is also an urgent problem to be solved.

[0003] Shannon's separation theorem points...

Examples

Embodiment Construction

[0047] This embodiment implements JSCC for text data with a fixed code length based on deep learning and using GRU. This embodiment proposes to include two parts:

[0048] 1) In this embodiment, GAN is used to train the unidirectional JSCC model, and pointwise mutual information (Pointwise Mutual Information, PMI) is added in the process of beam search.

[0049] 2) In order to overcome the deficiency of unidirectional JSCC, this embodiment adopts Synchronous Bidirectional Attention (SBAtt) and synchronous bidirectional beam search to interactively use past and future information to decode text.

[0050] 1) GAN training one-way neural network

[0051] see figure 1 , the SeqGAN model consists of three parts: generator G, discriminator D and sentence-level word error rate (Word Error Rate, WER). The decoding end of the generator is based on the one-way GAN neural network. The whole generator can be regarded as a JSCC framework, the purpose is to decode the input sou...