A communication method, apparatus, and computer program product

Abstract

Describes an apparatus, method and computer program, which includes components for converting one or more inputs into one or more data symbols at a transmitter of a transmission system using a transmitter algorithm, wherein: the transmission system includes a transmitter implementing the transmitter algorithm, a channel and a receiver including a receiver algorithm; the transmitter algorithm includes trainable parameters for converting one or more received data symbols into one or more outputs; and the transmitter algorithm is stochastic.

Description

Technical Field

[0001] This specification relates to transmitter algorithms. Background Art

[0002] A simple communication system includes a transmitter (including a transmitter algorithm), a transmission channel, and a receiver (including a receiver algorithm). Although progress has been made, there is still room for further development in this field. Summary of the Invention

[0003] In a first aspect, this specification describes an apparatus that includes components for converting one or more inputs into one or more data symbols at a transmitter of a transmission system using a transmitter algorithm, where: the transmission system includes a transmitter (e.g., as part of a bit-to-symbol mapper) that implements the transmitter algorithm, a channel, and a receiver that includes a receiver algorithm (e.g., implements a demapper); the transmitter algorithm includes trainable parameters for converting one or more received data symbols into one or more outputs; and the transmitter algorithm is stochastic. Some embodiments also include components for generating updated parameters of the transmission system to maximize the information transfer rate. The transmitter may include a transmitter neural network configured to implement the transmitter algorithm. Alternatively or additionally, the receiver may include a receiver neural network configured to implement the receiver algorithm.

[0004] In a second aspect, this specification describes an apparatus that includes: components for initializing the trainable parameters of a transmission system, where the transmission system includes a transmitter (e.g., as part of a bit-to-symbol mapper), a channel, and a receiver, where the transmitter includes a transmitter algorithm for converting one or more inputs into one or more data symbols (symbols for transmission), and the receiver algorithm includes a receiver algorithm for converting one or more received data symbols into one or more outputs (e.g., implements a demapper), where the transmitter algorithm is stochastic; components for generating updated parameters of the transmission system to maximize the information transfer rate, where generating the updated parameters of the transmission system includes updating the trainable parameters of the transmitter algorithm; and components for repeating the generation of the updated parameters of the transmission system until a first condition is met (e.g., a defined performance level or a defined number of iterations). The components for initializing the trainable parameters of the transmission system may initialize the trainable parameters randomly. The transmitter may include a transmitter neural network configured to implement the transmitter algorithm. Alternatively or additionally, the receiver may include a receiver neural network configured to implement the receiver algorithm.

[0005] In some embodiments, the component for generating updated parameters of a transmission system is configured to generate the parameters to minimize the difference between a distribution function and a target distribution (e.g., using the Kullback-Leibler divergence), where the distribution function defines the probability that an individual data symbol is output by a transmitter algorithm. The distribution function may approximate an expected function.

[0006] Some embodiments further include a component for generating a loss function, where the component for generating updated parameters of a transmission system is configured to minimize the loss function. Additionally, some embodiments further include: a component for generating one or more data symbols for transmission from the transmitter to the receiver; a component for observing a channel output at the receiver in response to transmitting the data symbols; and a component for minimizing the loss function based on the observed channel output.

[0007] In embodiments that include a loss function, the loss function may be (at least in part) based on a first variable minus a second variable. The first variable may include the difference between a distribution function and a target distribution, where the distribution function defines the probability that an individual data symbol is output by a transmitter algorithm, and the second variable includes an information transfer rate.

[0008] The information transfer rate may be based on the sum of the mutual information between one or more inputs and the output of a channel.

[0009] Generating updated parameters of a transmission system may include updating parameters of a transmitter algorithm.

[0010] A receiver algorithm may include trainable parameters. Additionally, generating updated parameters of a transmission system may include updating parameters of both a transmitter algorithm and a receiver algorithm.

[0011] The component for generating updated parameters of a transmission system may use stochastic gradient descent to update the parameters.

[0012] One or more inputs may be the output of a channel encoder of a transmitter. For example, the output of the channel encoder may be a K-bit data vector.

[0013] Data symbols may correspond to constellation positions of a modulation scheme implemented by a modulator of the transmitter. The modulator may convert the data symbols into transmission symbols according to the modulation scheme. Additionally, generating updated parameters of a transmission system may include generating updated parameters of the modulator.

[0014] A receiver algorithm may further include a component for estimating one or more inputs.

[0015] The component may include: at least one processor; and at least one memory including computer program code, the at least one memory and the computer program being configured to cause the execution of the apparatus, together with the at least one processor.

[0016] In a third aspect, this specification describes a method that includes: using a transmitter algorithm at a transmitter of a transmission system to convert one or more inputs into one or more data symbols, where: the transmission system includes a transmitter that implements the transmitter algorithm, a channel, and a receiver that includes a receiver algorithm; the transmitter algorithm includes trainable parameters for converting one or more received data symbols into one or more outputs; and the transmitter algorithm is stochastic. Some embodiments also include generating updated parameters of the transmission system to maximize the information transfer rate. The transmitter may include a transmitter neural network configured to implement the transmitter algorithm. Alternatively or additionally, the receiver may include a receiver neural network configured to implement the receiver algorithm.

[0017] In a fourth aspect, this specification describes a method that includes: initializing trainable parameters of a transmission system, where the transmission system includes a transmitter, a channel, and a receiver, where the transmitter includes a transmitter algorithm for converting one or more inputs into one or more data symbols, and the receiver algorithm includes a receiver algorithm for converting one or more received data symbols into one or more outputs, where the transmitter algorithm is stochastic; generating updated parameters of the transmission system to maximize the information transfer rate, where generating the updated parameters of the transmission system includes updating the trainable parameters of the transmitter algorithm; and repeating generating the updated parameters of the transmission system until a first condition is met. Initializing the trainable parameters of the transmission system may include randomly initializing the trainable parameters. The transmitter may include a transmitter neural network configured to implement the transmitter algorithm. Alternatively or additionally, the receiver may include a receiver neural network configured to implement the receiver algorithm.

[0018] Generating updated parameters of the transmission system may include generating the parameters to minimize the difference between a distribution function and a target distribution, the distribution function defining the probability that an individual data symbol is output by the transmitter algorithm. The distribution function may approximate an expected function.

[0019] Some embodiments also include generating a loss function, where the component for generating updated parameters of the transmission system is configured to minimize the loss function. Additionally, some embodiments also include: generating one or more data symbols for transmission from the transmitter to the receiver; observing a channel output at the receiver in response to transmitting the data symbols; and minimizing the loss function based on the observed channel output. The loss function may be (at least in part) based on a first variable minus a second variable.

[0020] The information transfer rate can be based on the sum of the mutual information between one or more inputs and the output of the channel.

[0021] Generating updated parameters of the transmission system can include updating the parameters of the transmitter algorithm.

[0022] The receiver algorithm can include trainable parameters. Additionally, generating updated parameters of the transmission system can include updating the parameters of both the transmitter algorithm and the receiver algorithm.

[0023] The data symbols can correspond to the constellation positions of the modulation scheme implemented by the modulator of the transmitter. The modulator can convert the data symbols into transmission symbols according to the modulation scheme. Additionally, generating updated parameters of the transmission system can include generating updated parameters of the modulator.

[0024] In a fifth aspect, this specification describes an apparatus configured to perform any of the methods described with reference to the third or fourth aspects.

[0025] In a sixth aspect, this specification describes computer-readable instructions that, when executed by a computing device, cause the computing device to perform any of the methods described with reference to the third or fourth aspects.

[0026] In a seventh aspect, this specification describes a computer-readable medium that includes program instructions stored thereon for performing at least the following operations:

[0027] Using a transmitter algorithm at a transmitter of a transmission system to convert one or more inputs into one or more data symbols, where: the transmission system includes a transmitter that implements the transmitter algorithm, a channel, and a receiver that includes a receiver algorithm. The transmitter algorithm includes trainable parameters for converting one or more received data symbols into one or more outputs; and the transmitter algorithm is stochastic.

[0028] In an eighth aspect, this specification describes a computer-readable medium having program instructions stored thereon for performing at least the following operations: initializing the trainable parameters of a transmission system, where the transmission system includes a transmitter, a channel, and a receiver, where the transmitter includes a transmitter algorithm for converting one or more inputs into one or more data symbols, and the receiver algorithm includes a receiver algorithm for converting one or more received data symbols into one or more outputs, where the transmitter algorithm is stochastic; generating updated parameters of the transmission system to maximize the information transfer rate, where generating the updated parameters of the transmission system includes updating the trainable parameters of the transmitter algorithm; and repeating the generation of the updated parameters of the transmission system until a first condition is met.

[0029] In a ninth aspect, this specification describes a computer program that includes instructions for causing a device to at least perform the following operations: instructions for converting one or more inputs into one or more data symbols at a transmitter of a transmission system using a transmitter algorithm, where: the transmission system includes a transmitter that implements the transmitter algorithm, a channel, and a receiver that includes a receiver algorithm; the transmitter algorithm includes trainable parameters for converting one or more received data symbols into one or more outputs; and the transmitter algorithm is stochastic.

[0030] In a tenth aspect, this specification describes a computer program that includes instructions for causing a device to at least perform the following operations: initializing trainable parameters of a transmission system, where the transmission system includes a transmitter, a channel, and a receiver, where the transmitter includes a transmitter algorithm for converting one or more inputs into one or more data symbols, and the receiver algorithm includes a receiver algorithm for converting one or more received data symbols into one or more outputs, where the transmitter algorithm is stochastic; generating updated parameters of the transmission system to maximize an information transfer rate, where generating the updated parameters of the transmission system includes updating the trainable parameters of the transmitter algorithm; and repeating generating the updated parameters of the transmission system until a first condition is met.

[0031] In an eleventh aspect, this specification describes a device that includes: at least one processor; and at least one memory that includes computer program code that, when executed by the at least one processor, causes the device to: convert one or more inputs into one or more data symbols at a transmitter of a transmission system using a transmitter algorithm, where: the transmission system includes a transmitter that implements the transmitter algorithm, a channel, and a receiver that includes a receiver algorithm; the transmitter algorithm includes trainable parameters for converting one or more received data symbols into one or more outputs; and the transmitter algorithm is stochastic.

[0032] In a twelfth aspect, this specification describes a device that includes: at least one processor; and at least one memory that includes computer program code that, when executed by the at least one processor, causes the device to: initialize trainable parameters of a transmission system, where the transmission system includes a transmitter, a channel, and a receiver, where the transmitter includes a transmitter algorithm for converting one or more inputs into one or more data symbols, and the receiver algorithm includes a receiver algorithm for converting one or more received data symbols into one or more outputs, where the transmitter algorithm is stochastic; generate updated parameters of the transmission system to maximize an information transfer rate, where generating the updated parameters of the transmission system includes updating the trainable parameters of the transmitter algorithm; and repeat generating the updated parameters of the transmission system until a first condition is met.

[0033] In a thirteenth aspect, this specification describes an apparatus that includes: a transmitter control module that includes a transmitter algorithm for converting one or more inputs into one or more data symbols at a transmitter of a transmission system, where: the transmission system includes a transmitter that implements the transmitter algorithm, a channel, and a receiver that includes a receiver algorithm; the transmitter algorithm includes trainable parameters for converting one or more received data symbols into one or more outputs; and the transmitter algorithm is stochastic.

[0034] In a fourteenth aspect, this specification describes an apparatus that includes: an initialization module configured to initialize trainable parameters of a transmission system, where the transmission system includes a transmitter, a channel, and a receiver, where the transmitter includes a transmitter algorithm for converting one or more inputs into one or more data symbols, and the receiver algorithm includes a receiver algorithm for converting one or more received data symbols into one or more outputs, where the transmitter algorithm is stochastic; a parameter update module configured to generate updated parameters of the transmission system to maximize an information transfer rate, where generating the updated parameters of the transmission system includes updating the trainable parameters of the transmitter algorithm; and a control module configured to repeatedly generate the updated parameters of the transmission system until a first condition is met. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Example embodiments will now be described by way of non-limiting example with reference to the following schematic diagrams, in which:

[0036] Figure 1 is a block diagram of an example end-to-end communication system according to an example embodiment;

[0037] Figure 2 is a block diagram of an example transmitter that can be used in a Figure 1 communication system;

[0038] Figure 3 is a block diagram of an example receiver that can be used in a Figure 1 communication system;

[0039] Figure 4 is a representation of a modulation scheme used in an example embodiment;

[0040] Figures 5 to 8 is a flowchart showing an algorithm according to an example embodiment;

[0041] Figure 9 is a block diagram of an example transmitter module according to an example embodiment;

[0042] Figure 10 is a block diagram of an example receiver module according to an example embodiment;

[0043] Figure 11 shows an example neural network that can be used in an example embodiment;

[0044] Figure 12 is a flowchart showing an algorithm according to an example embodiment;

[0045] Figure 13 is a block diagram of components of a system according to an exemplary embodiment; and

[0046] Figure 14A and Figure 14B show tangible media (a removable memory unit and a compact disc (CD), respectively) storing computer-readable code that, when run by a computer, performs operations according to an embodiment. Detailed Description

[0047] The scope of protection sought by the various embodiments of the present invention is defined by the independent claims. Embodiments and features described in the specification that are not within the scope of the independent claims (if any) will be construed as examples useful for understanding the various embodiments of the present invention.

[0048] In the specification and drawings, the same reference numerals always refer to the same elements.

[0049] Figure 1 is a block diagram of an example end-to-end communication system according to an example embodiment, which is generally indicated by reference numeral 1. System 1 includes a transmitter 2, a channel 4, and a receiver 6. From a system perspective, system 1 converts the data received at the input of transmitter 2 into an estimate of that data at the output of receiver 6. More specifically, transmitter 2 converts the input bits ((Data)) into transmitted symbols (x) for transmission through channel 4, and receiver 6 generates output bits based on the symbols (y) received from channel 4 In at least some embodiments, the channel model may not be available.

[0050] Figure 2 is a block diagram of an example transmitter 2 that can be used in the above communication system 1. As shown in FIG. 2, transmitter 2 includes a channel encoder 12, a bit-to-symbol mapper module 14, and a modulator 16.

[0051] The channel encoder 12 receives the data to be transmitted and encodes the data in some way. For example, the channel encoder can be a low-density parity-check code (LDPC) encoder that encodes the received data with parity check bits. Other coding schemes (such as polar coding) can be implemented to replace (or supplement) LDPC coding.

[0052] The bit-to-symbol mapper 14 receives the encoded data bits (b) and converts these bits into data symbols (s) for transmission. For example, the encoded data bits (b) may be in the form of a data stream that is packetized for transmission by the bit-to-symbol mapper 14.

[0053] The modulator 16 converts the data symbols into transmitted symbols (x) according to a modulation scheme. The transmitted symbols are then sent over the channel 4 and received at the receiver 6 as received symbols (y). The modulator 16 maps the symbol s to a constellation point x ∈ C in the complex plane. For example, modulation can be performed using a scheme such as QAM; however, many other schemes can be used, including higher-order modulation schemes, where x ∈ C r and r ≥ 2. As described below, the modulation scheme itself can be trainable.

[0054] Figure 3 is a block diagram of an example receiver 6 that can be used in the communication system 1 described above. As Figure 3 shown, the receiver 6 includes a demodulator 22, a demapper module 24, and a channel decoder 26. The demodulator 22 converts the received symbol (y) into a symbol probability p(s|y), and the demapper module 24 converts the symbol probability into a log-likelihood ratio (LLR), as discussed further below. The channel decoder 26 attempts to reconstruct the original data bits based on the output of the demapper module 24 (e.g., based on LDPC coding, polar coding, and / or other coding schemes).

[0055] A variety of modulation techniques can be used in the implementation of the modulator 16 (and demodulator 22). These include: amplitude-shift keying (ASK), where the amplitude of the carrier signal is modified based on the signal being transmitted; and phase-shift keying (PSK), where the phase of the carrier signal is modified based on the signal being transmitted. For example, quadrature phase-shift keying (QPSK) is a form of phase-shift keying where two bits are modulated at a time to select one of four possible carrier phase shifts (e.g., 0, +90 degrees, 180 degrees, -90 degrees). This carrier phase and amplitude are typically represented as constellation positions in the complex plane.

[0056] For example, Figure 4 is a representation of a quadrature amplitude modulation (QAM) scheme used in an example embodiment, generally indicated by the reference numeral 30. The QAM representation 30 includes 16 points plotted on the in-phase (I) and quadrature (Q) axes. Thus, in the example representation 30, 16 symbols can be modulated in different ways by a modulator (such as the modulator 16 of the transmitter 2 described above). Those skilled in the art will know many other suitable modulation techniques.

[0057] The selection of a modulation scheme for transmitting information, such as communication system 1, may affect the end-to-end performance of such a communication system. Additionally, such a modulation scheme can be optimized. For example, the relative frequencies of different constellation points using a particular modulation scheme can be optimized (e.g., using probability shaping).

[0058] If the probability distribution at which constellation symbols should occur has been identified to maximize the information rate, it remains a challenge to map the bits from an incoming bit stream to constellation symbols such that the constellation symbols occur with the target probability.

[0059] Figure 5 is a flowchart showing an algorithm generally indicated by reference numeral 40 according to an example embodiment. Algorithm 40 can be implemented by the bit-to-symbol mapper module 14 of the transmitter 2 described above.

[0060] Algorithm 40 begins at operation 42, where bits for transmission are received by the bit-to-symbol mapper 14. Module 14 can include a transmitter algorithm (e.g., a trainable transmitter algorithm) for converting one or more received bits into one or more outputs. As described in detail below, the transmitter algorithm implemented by the bit-to-symbol mapper 14 is stochastic.

[0061] At operation 44, data symbols are generated based on the bits received at operation 42. In the example transmitter 2, the generated data symbols are provided to the modulator 16 for transmission over the channel 4.

[0062] Thus, the bit-to-symbol mapper 14 can be provided to map the bits from an incoming bit stream (b) to a sequence of constellation symbols such that the constellation symbols occur with a probability p(s) close to the target distribution p * (s). The incoming bit stream is typically the output of the channel encoder 12, and the generated constellation symbols are typically mapped by the modulator 16 to constellation points in the complex plane according to some constellation scheme (such as QAM, PSK, or ASK), as discussed above. The target distribution p * (s) can be selected such that it maximizes the information rate.

[0063] Operation 44 can implement a fixed-to-fixed length mapping such that each bit vector b of K bits is mapped to a constellation symbol where and M is the modulation order.

[0064] The bit vector b is mapped to a constellation symbol from in a random manner, i.e., the bit vector b is randomly mapped to the symbol according to the distribution p(s|b) The set of conditional distributions can cause the symbol The probability of occurrence is given by the following formula:

[0065]

[0066] where the summation is carried out over all possible binary vectors of length K, and p(s) is close to the target distribution p * (s), which is assumed to maximize the information rate.

[0067] As described below, the cumulative information between the bit b k (k = 1...K) and the channel output y should be high so that demapping and decoding are possible.

[0068] Also as described below, stochastic gradient descent (SGD) can be used to calculate the conditional distribution p(s|b) that satisfies the foregoing conditions.

[0069] Figure 6 is a flowchart showing an algorithm generally indicated by reference numeral 50 according to an exemplary embodiment.

[0070] Algorithm 50 begins at operation 52, where symbols are received from channel 4 at receiver 6. At operation 54, the bit-to-symbol demapper module 24 of receiver 6 is used to generate log-likelihood ratios based on the received data.

[0071] Thus, the demapper module 24 maps the probability distribution over the symbol set conditioned on the received signal y into bit log-likelihood ratios (LLRs), i.e.,

[0072]

[0073] Assuming equally probable independent and identically distributed (iid) data bits, p(b i ) == 1 / 2, the receiver knows the conditional distribution and the demodulator provides p(s|y), so p(b i |y) (i ∈ {1,..., K}) can be calculated as follows:

[0074]

[0075] And

[0076]

[0077] where the calculation of p(s) is as described in (1), and accordingly, the LLR can be calculated according to (2).

[0078] The random bit-to-symbol mapper 14 outputs from symbols randomly drawn. One challenge is to compute the conditional distribution of the 2 K possible input bit vectors

[0079] distribution can be computed by a neural network (NN) with parameters θ from inputs such as channel state information (CSI). The distribution can also be directly optimized, in which case it is assumed that a target distribution p * (s) that seeks to maximize the information rate has been defined, and the set of conditional distributions should be such that:

[0080] · The constellation symbol distribution (1) is close to the target distribution p * (s); and

[0081] · The information about the input bits {b k} k＝1...K carried by the channel output y is high enough to enable demapping.

[0082] These two conditions are represented by the following loss function:

[0083]

[0084] The first term of the loss function (D KL (p(s)||p * (s))) measures the difference between the distribution of the symbols (1) and the target distribution using the Kullback-Leibler (KL) divergence. Other divergence measures for determining the difference between the distribution function and the target distribution are possible, where the distribution function defines the probability that an individual data symbol is output by the transmitter algorithm.

[0085] The second term of the loss function is the sum of the mutual information (MI) between the bits b k and the channel output y.

[0086] In one embodiment, the KL divergence can be computed exactly as follows:

[0087]

[0088] The mutual information (MI) can be estimated by:

[0089]

[0090] where B is the batch size, i.e., the number of samples used to estimate the mutual information, {y (j) , ∼j = 1...B} are samples of the channel output, and

[0091]

[0092] Other mechanisms for determining the information transfer rate for use in the loss function (5) are possible.

[0093] Figure 7 is a flowchart showing an algorithm generally indicated by reference numeral 60 according to an example embodiment. The algorithm 60 relates to the training of a transmission system 1 (e.g., the transmitter 2 of a transmission system). The transmission system includes a transmitter, a channel, and a receiver, where the transmitter includes a random transmitter algorithm for converting one or more inputs into one or more data symbols (i.e., symbols for transmission).

[0094] The algorithm 60 begins at operation 61, where the trainable parameters of the transmission system (such as transmission system 1) are initialized. The initialization of the trainable parameters can be random or can be implemented in some other way (e.g., based on a defined starting point or a previous implementation). The trainable parameters can include the trainable parameters of the transmitter algorithm. Alternatively or additionally, the trainable parameters can include the trainable parameters of one or more of the transmitter's modulator and / or the receiver algorithm.

[0095] At operation 62, updated trainable parameters are generated. For example, the trainable parameters can be updated to maximize the information transfer rate (e.g., as defined by equation (7) above). Alternatively or additionally, the trainable parameters can be updated to minimize the difference between the distribution function and the target distribution, where the distribution function defines the probability that an individual data symbol is output by the transmitter algorithm. The trainable parameters can be updated by minimizing the loss function (e.g., using stochastic gradient descent).

[0096] For example, operation 62 can be implemented by generating one or more data symbols to send from the transmitter 2 of the transmission system 1 to the receiver 6, observing the channel output at the receiver in response to sending the data symbols, and minimizing the above loss function based on the observed channel output.

[0097] At operation 63, the relevant trainable parameters are updated based on the updated parameters generated in operation 62.

[0098] At operation 64, it is determined whether the algorithm 60 is complete. If a first condition is met, the algorithm 60 can be considered complete. The first condition can take various forms, such as a defined performance level, a defined number of iterations, or when it is estimated that the loss function does not decrease by more than a threshold amount during a fixed number of iterations.

[0099] If the algorithm is considered complete, the algorithm terminates at operation 65; otherwise the algorithm returns to operation 62.

[0100] Figure 8is a flowchart showing an algorithm generally indicated by reference numeral 70 according to an example embodiment. The algorithm 70 relates to the training of a transmission system 1 (e.g., a transmitter 2 of a transmission system). The transmission system includes a transmitter, a channel, and a receiver, where the transmitter includes a random transmitter algorithm for converting one or more inputs into one or more data symbols (i.e., symbols for transmission).

[0101] The algorithm 70 begins at operation 71, where the trainable parameter θ is initialized. The initialization of the trainable parameter can be random or can be initialized in some other way (e.g., to a defined starting point or a previous implementation). The trainable parameter can include the trainable parameters of the transmitter algorithm. Alternatively or additionally, the trainable parameter can include the trainable parameters of one or more of the modulator of the transmitter and / or the receiver algorithm.

[0102] At operation 72, B samples of the channel output y (j) (~j = 1...B) are generated as follows.

[0103] · Randomly sample B samples of the bit vector b such that the bits are equiprobable, independent, and identically distributed (iid).

[0104] · For each bit vector b, randomly generate constellation symbols s ~ p(s|b).

[0105] · Transmit the constellation symbols over the channel and observe y.

[0106] At operation 73, the trainable parameter θ is updated by performing one step of stochastic gradient descent (SGD) on the estimated loss:

[0107]

[0108] where p(b k |y) is calculated according to (8), and D KL is calculated according to (6).

[0109] At operation 74, it is determined whether the algorithm 70 is complete. If a first condition is met, the algorithm 70 can be considered complete. The first condition can take various forms, such as a defined performance level, a defined number of iterations, or when the estimated loss function does not decrease by more than a threshold amount during a fixed number of iterations.

[0110] If the algorithm is considered complete, the algorithm terminates at operation 75; otherwise the algorithm returns to operation 72.

[0111] As described above, the generation of the updated parameters in operations 62 and 72 can seek to maximize the information transfer rate and / or minimize the difference between the distribution function and the target distribution (e.g., using the Kullback-Leibler divergence), where the distribution function defines the probability that an individual data symbol is output by the transmitter algorithm. The information transfer rate can be maximized so that the information carried by the channel output y about the input bits { k} k＝1...K is high enough to enable the likelihood of demapping. The difference can be minimized such that the distribution of the constellation symbols output by the transmitter is close to the target distribution p * (s).

[0112] The trainable parameters of algorithms 60 and 70 can be trained by minimizing a loss function (at least in part) based on a first variable minus a second variable, where the first variable includes the difference between the distribution function and the target distribution, the distribution function defines the probability that an individual data symbol is output by the transmitter algorithm, and the second variable includes the information transfer rate (e.g., the sum of the mutual information between one or more inputs and the output of the channel).

[0113] As described above, generating the updated parameters (e.g., operations 62 and 72 of algorithms 60 and 70) can generate updated parameters of the transmitter algorithm. Alternatively or additionally, generating the updated parameters can involve generating updated parameters of the receiver algorithm and / or generating updated parameters of the modulator of the transmitter.

[0114] Algorithms 60 and 70 include operations (operations 64 and 74) in which it is determined whether the relevant algorithm is complete. These operations can be implemented in many different ways. These include stopping after a given number of iterations, stopping when the loss function is below a value defined below, or stopping when the loss function does not increase significantly during a defined number of iterations. Those skilled in the art will be aware of many alternative implementations of operations 64 and 74.

[0115] The transmitter can be implemented using a transmitter neural network configured to implement the transmitter algorithm. Alternatively or additionally, the receiver can be implemented using a receiving neural network configured to implement the receiver algorithm described above.

[0116] Figure 9 is a block diagram of an example transmitter module according to an example embodiment, which is generally indicated by the reference numeral 90. The transmitter 90 implements the transmitter algorithm and can, for example, implement the bit-to-symbol mapper 14 of the transmitter 1 described above. As Figure 9As shown, transmitter 90 includes dense layers of one or more units 91, 92 (e.g., including one or more neural networks), and a normalization module 93. The dense layers 91, 92 may include an embedding module. The modules within transmitter 90 are provided as examples, and modifications are possible. Transmitter module 90 is one of many example configurations that may be provided; those skilled in the art will be aware of many possible variations.

[0117] Figure 10 is a block diagram of an example receiver module according to an example embodiment, which is generally indicated by reference numeral 100. Receiver 100 implements a receiver algorithm and may, for example, implement the demodulator module 22 of receiver 6 described above. Receiver module 100 includes a complex-to-real conversion module 101, a first dense layer 102, and a second dense layer 103 (e.g., including one or more neural networks), and a softmax layer 104. The output of softmax layer 104 is a probability function. Providing two dense layers in system 100 is merely an example; any number of dense layers may be provided. Receiver module 100 is one of many example configurations that may be provided; those skilled in the art will be aware of many possible variations.

[0118] The complex-to-real conversion module 101 converts the received vector y into real values. For example, this may be done by concatenating the real and imaginary parts of the samples to obtain a vector with values in R 2M values.

[0119] Many variations for implementing the receiver are possible. For example, the receiver may include a demapper and a demodulator, which are implemented as a single neural network with trainable parameters that are jointly optimized with θ during training.

[0120] Figure 11 An example neural network 110 that may be used in one or more example embodiments is shown. Neural network 110 includes a plurality of interconnected nodes arranged in multiple layers. A neural network such as network 110 may be trained by adjusting the connections between the nodes and the relative weights of these connections. As described above, the transmitter and receiver algorithms may be implemented using one or more neural networks, such as a neural network in the form of neural network 110.

[0121] As described above, the transmitter may include a bit-to-symbol mapper 14, which is configured to map bits from an incoming bit stream (b) to a constellation symbol sequence such that the constellation symbols occur with a probability p(s) that is close to the target distribution p * (s). The target distribution p * (s) may be selected such that it maximizes the information rate.

[0122] Figure 12is a flowchart showing an algorithm according to an example embodiment, which is generally indicated by reference numeral 120 in the drawings.

[0123] Algorithm 120 begins at operation 121, in which the modulation constellation is optimized. The example constellations were described above with reference to Figure 4 The constellation can be optimized to improve the end-to-end performance of a communication system, such as communication system 1 above. Constellation optimization can involve optimizing the positions of the constellation points (so-called geometric shaping). Alternatively or additionally, constellation optimization can involve optimizing the number of constellation points or selecting the best constellation method. The constellation optimization operation 121 can use machine learning principles, for example, by using stochastic gradient descent or a similar method to optimize the constellation.

[0124] At operation 122, the frequencies at which various constellation points are transmitted are optimized (so-called probability shaping). Probability shaping can be optimized to improve the end-to-end performance of a communication system, such as communication system 1 above. The probability shaping optimization operation 122 can use machine learning principles, such as using stochastic gradient descent or a similar method.

[0125] The output of operation 122 can provide the above-mentioned target distribution p * (s).

[0126] At operation 123, the trainable parameters of communication system 1 are updated, for example, using algorithm 60 or 70 above. Thus, the principles described herein can be applied to a communication system having a modulation scheme optimized in operations 121 and 122.

[0127] At operation 124, it is determined whether algorithm 120 is complete. If a first condition is met, algorithm 120 can be considered complete. The first condition can take various forms, such as a defined performance level, a defined number of iterations, or when the estimated loss function does not decrease by more than a threshold amount during a fixed number of iterations.

[0128] If the algorithm is considered complete, the algorithm terminates at operation 125; otherwise, the algorithm returns to operation 121.

[0129] Many variations of algorithm 120 are possible. For example, one or more of operations 121, 122, and 123 can be omitted, and these operations can be provided in any order. Additionally, two or more (e.g., all) of operations 121, 122, and 123 can be combined to optimize multiple parameters together.

[0130] For completeness, Figure 13 is a schematic diagram of one or more components of the example embodiment(s) described previously, which are collectively referred to as processing system 300 hereinafter. Processing system 300 can be, for example, the apparatus recited in the following claims.

[0131] The processing system 300 can have a processor 302, a memory 304 that is tightly coupled to the processor and includes RAM 314 and ROM 312, and optional user input 310 and display 318. The processing system 300 can include one or more network / device interfaces 308 for connecting to a network / device, such as a modem that can be wired or wireless. The interface 308 can also operate as a connection to other devices, such as a device / apparatus that is not a network-side device. Thus, a direct connection between devices / apparatuses without network participation is possible.

[0132] The processor 302 is connected to each of the other components to control their operation.

[0133] The memory 304 can include non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD). The ROM 312 of the memory 304 stores, in particular, the operating system 315 and can store software applications 316. The RAM 314 of the memory 304 is used by the processor 302 to temporarily store data. The operating system 315 can contain code that, when executed by the processor, implements aspects of the above algorithms 40, 50, 60, 70, and 120. Note that in the case of small devices / apparatuses, the memory is most suitable for small-size use, i.e., an HDD or SSD is not always used.

[0134] The processor 302 can take any suitable form. For example, it can be a microcontroller, multiple microcontrollers, a processor, or multiple processors.

[0135] The processing system 300 can be a stand-alone computer, server, console, or a network thereof. The processing system 300 and the required structural components can all be inside the device / apparatus, such as an Internet of Things device / apparatus, i.e., embedded in a very small size.

[0136] In some example embodiments, the processing system 300 can also be associated with external software applications. These can be applications stored on a remote server device / apparatus and can run partially or exclusively on the remote server device / apparatus. These applications can be referred to as cloud-hosted applications. The processing system 300 can communicate with the remote server device / apparatus to utilize the software applications stored there.

[0137] Figure 14A and Figure 14BIllustrated are tangible media (a removable memory unit 365 and an optical disc (CD) 368, respectively) storing computer-readable code which, when run by a computer, can execute the methods according to the above exemplary embodiments. The removable storage unit 365 can be a memory stick, for example, a USB memory stick, which has an internal memory 366 storing the computer-readable code. The internal memory 366 can be accessed by a computer system via a connector 367. The CD 368 can be a CD-ROM or a DVD, etc. Other forms of tangible storage media can be used. A tangible medium can be any device / apparatus capable of storing data / information which can be exchanged between devices / apparatuses / networks.

[0138] Embodiments of the present invention can be implemented in software, hardware, application logic, or a combination of software, hardware, and application logic. The software, application logic, and / or hardware can reside in a memory or any computer medium. In an exemplary embodiment, the application logic, software, or instruction set is maintained on any of a variety of conventional computer-readable media. In the context of this document, "memory" or "computer-readable medium" can be any non-transitory medium or device that can contain, store, transmit, propagate, or transport instructions for use by or in connection with an instruction execution system, apparatus, or device such as a computer.

[0139] In relevant cases, references to "computer-readable medium", "computer program product", "tangibly embodied computer program", etc. or "processor" or "processing circuitry", etc. should be understood to include not only computers having different architectures (such as single / multi-processor architectures and sequencer / parallel architectures), but also dedicated circuits such as field programmable gate arrays FPGAs, application specific integrated circuits ASICs, signal processing devices / apparatuses, and other devices / apparatuses. References to computer programs, instructions, code, etc. should be understood to express software for programmable processor firmware (such as programmable content of a hardware device / apparatus), instructions for a processor, or configuration or configuration settings for fixed function devices / apparatuses, gate arrays, programmable logic devices / apparatuses, etc.

[0140] If desired, the different functions discussed herein can be performed in a different order and / or simultaneously with each other. Additionally, if desired, one or more of the above functions can be optional or can be combined. Similarly, it should also be understood that Figure 5 、 Figure 6 、 Figure 7 、 Figure 8 and Figure 12 the flowcharts of

[0141] It should be understood that the above exemplary embodiments are purely illustrative and do not limit the scope of the present invention. Other variations and modifications will be apparent to those skilled in the art upon reading this specification.

[0142] Furthermore, the disclosure of the present application should be understood to include any novel feature or any novel combination of features or any generalization thereof that is explicitly or implicitly disclosed herein, and during the implementation of the present application or any application derived therefrom, new claims may be formulated to cover any such feature and / or combination of such features.

[0143] Although various aspects of the present invention are set forth in the independent claims, other aspects of the present invention include other combinations of features from the described exemplary embodiments and / or dependent claims with the features of the independent claims, rather than only the combinations explicitly given in the claims.

[0144] It should also be noted herein that although the above describes various examples, these descriptions should not be regarded as restrictive. On the contrary, various changes and modifications can be made without departing from the scope of the present invention as defined by the appended claims.

Claims

1. A device for communication, comprising components for converting one or more inputs into one or more data symbols at a transmitter of a transmission system using a transmitter algorithm, wherein: the transmission system comprises the transmitter implementing the transmitter algorithm, a channel, and a receiver comprising a receiver algorithm; the transmitter algorithm comprises trainable parameters for converting one or more received data symbols into one or more outputs; and the transmitter algorithm is stochastic, wherein the device further comprises components for generating updated parameters of the transmission system to maximize an information transfer rate and to minimize a difference between a distribution function and a target distribution, the distribution function defining a probability that an individual data symbol is output by the transmitter algorithm.

2. A device for communication, comprising: components for initializing trainable parameters of a transmission system, wherein the transmission system comprises a transmitter, a channel, and a receiver, wherein the transmitter comprises a transmitter algorithm for converting one or more inputs into one or more data symbols, and the receiver algorithm comprises a receiver algorithm for converting one or more received data symbols into one or more outputs, wherein the transmitter algorithm is stochastic; components for generating updated parameters of the transmission system to maximize an information transfer rate, wherein generating the updated parameters of the transmission system comprises updating the trainable parameters of the transmitter algorithm; and components for repeating generating the updated parameters of the transmission system until a first condition is met, wherein the components for generating the updated parameters of the transmission system are configured to generate the parameters to minimize a difference between a distribution function and a target distribution, the distribution function defining a probability that an individual data symbol is output by the transmitter algorithm.

3. The device according to claim 2, wherein the components for initializing the trainable parameters of the transmission system initialize the trainable parameters randomly.

4. The device according to any one of claims 1 to 3, further comprising components for generating a loss function, wherein the components for generating the updated parameters of the transmission system are configured to minimize the loss function.

5. The device according to claim 4, further comprising: components for generating one or more data symbols for transmission from the transmitter to the receiver; components for observing a channel output at the receiver in response to transmitting the data symbols; and components for minimizing the loss function based on the observed channel output.

6. The device according to claim 4, wherein the loss function is based on a first variable minus a second variable, wherein the first variable comprises a difference between a distribution function and a target distribution, the distribution function defining a probability that an individual data symbol is output by the transmitter algorithm, and the second variable comprises the information transfer rate.

7. The device according to any one of claims 1 to 3, wherein the information transfer rate is based on a sum of mutual information between the one or more inputs and an output of the channel.

8. The apparatus according to any one of claims 1 to 3, wherein generating the updated parameters of the transmission system includes updating the parameters of the transmitter algorithm.

9. The apparatus according to any one of claims 1 to 3, wherein the receiver algorithm includes trainable parameters.

10. The apparatus according to claim 9, wherein generating the updated parameters of the transmission system includes updating the parameters of both the transmitter algorithm and the receiver algorithm.

11. The apparatus according to any one of claims 1 to 3, wherein the component for generating the updated parameters of the transmission system uses stochastic gradient descent to update the parameters.

12. The apparatus according to any one of claims 1 to 3, wherein the one or more inputs are the output of the channel encoder of the transmitter.

13. The apparatus according to any one of claims 1 to 3, wherein the data symbols correspond to the constellation positions of the modulation scheme implemented by the modulator of the transmitter.

14. The apparatus according to claim 13, wherein generating the updated parameters of the transmission system includes generating the updated parameters of the modulator.

15. The apparatus according to any one of claims 1 to 3, wherein the receiver algorithm further includes a component for estimating the one or more inputs.

16. The apparatus according to any one of claims 1 to 3, wherein the transmitter includes a transmitter neural network configured to implement the transmitter algorithm.

17. The apparatus according to any one of claims 1 to 3, wherein the receiver includes a receiver neural network configured to implement the receiver algorithm.

18. The apparatus according to any one of claims 1 to 3, wherein the component includes: at least one processor; and at least one memory including computer program code, the at least one memory and the computer program being configured to, together with the at least one processor, cause the execution of the apparatus.

19. A method of communication, including converting one or more inputs into one or more data symbols at a transmitter of a transmission system using a transmitter algorithm, wherein: the transmission system includes the transmitter implementing the transmitter algorithm, a channel, and a receiver including a receiver algorithm; the transmitter algorithm includes trainable parameters for converting one or more received data symbols into one or more outputs; and the transmitter algorithm is stochastic, wherein the method further includes generating updated parameters of the transmission system to maximize the information transfer rate and minimize the difference between the distribution function and the target distribution, the distribution function defining the probability that an individual data symbol is output by the transmitter algorithm.

20. A method of communication, including: Initialize the trainable parameters of a transmission system, where the transmission system includes a transmitter, a channel, and a receiver, where the transmitter includes a transmitter algorithm for converting one or more inputs into one or more data symbols, and the receiver algorithm includes a receiver algorithm for converting one or more received data symbols into one or more outputs, where the transmitter algorithm is random; Generate updated parameters of the transmission system to maximize the information transfer rate and to minimize the difference between a distribution function and a target distribution, the distribution function defining the probability that an individual data symbol is output by the transmitter algorithm, where generating the updated parameters of the transmission system includes updating the trainable parameters of the transmitter algorithm; and Repeat generating the updated parameters of the transmission system until a first condition is met.

21. A computer program product comprising instructions for causing an apparatus to use a transmitter algorithm to convert one or more inputs into one or more data symbols at a transmitter of a transmission system, where: the transmission system includes the transmitter implementing the transmitter algorithm, a channel, and a receiver including a receiver algorithm; the transmitter algorithm includes trainable parameters for converting one or more received data symbols into one or more outputs; and the transmitter algorithm is random, where the computer program product further includes instructions for generating updated parameters of the transmission system to maximize the information transfer rate and to minimize the difference between a distribution function and a target distribution, the distribution function defining the probability that an individual data symbol is output by the transmitter algorithm.

22. A computer program product comprising instructions for causing an apparatus to at least perform the following operations: Initialize the trainable parameters of a transmission system, where the transmission system includes a transmitter, a channel, and a receiver, where the transmitter includes a transmitter algorithm for converting one or more inputs into one or more data symbols, and the receiver algorithm includes a receiver algorithm for converting one or more received data symbols into one or more outputs, where the transmitter algorithm is random; Generate updated parameters of the transmission system to maximize the information transfer rate and to minimize the difference between a distribution function and a target distribution, the distribution function defining the probability that an individual data symbol is output by the transmitter algorithm, where generating the updated parameters of the transmission system includes updating the trainable parameters of the transmitter algorithm; and Repeat generating the updated parameters of the transmission system until a first condition is met.

Images