Method and system for compression

Abstract

A method and system for compression is provided herein. According to one embodiment, a method includes: training a conditional autoencoder using Lagrange multipliers; and training a neural network including the conditional autoencoder with mixed quantization bin sizes.

Description

[0001] This application is based on and claims the benefit of priority of U.S. Provisional Patent Application filed in the U.S. Patent and Trademark Office on March 18, 2019 and assigned Serial No. 62 / 820,148, U.S. Provisional Patent Application filed in the U.S. Patent and Trademark Office on June 12, 2019 and assigned Serial No. 62 / 860,522, and U.S. Patent Application filed in the U.S. Patent and Trademark Office on September 19, 2019 and assigned Serial No. 16 / 576,166, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present disclosure relates generally to data compression. In particular, the present disclosure relates to methods and apparatus for compression using conditional autoencoders. Background Art

[0003] Variable ratio compression can utilize neural network functions and can be applied in various fields, including but not limited to image compression and radio frequency integrated circuit (RFIC) output compression.

[0004] Image compression is the application of data compression to digital images to reduce the storage and / or transmission requirements of digital images. Transform coding can be used to generate image compression algorithms, such as the Joint Photographic Experts Group (JPEG) and JPEG2000. Such transform coding converts the input to a potential representation in the transform domain, where lossy compression (usually a combination of quantization and lossless source coding) is more compliant and more effective. For example, JPEG utilizes the discrete cosine transform (DCT) to convert an image to a sparse frequency domain representation. JPEG2000 replaces the DCT with an enhanced discrete wavelet transform.

[0005] Deep learning can provide improved performance in various computer vision tasks. Along with this revolutionary progress in deep learning, learning-based image compression has also gained attention. In particular, nonlinear transform coding designed with deep neural networks has advanced to outperform classical image compression codecs (e.g., Better Portable Graphics (BPG) as a still image version of the High Efficiency Video Codec (HEVC) standard) that are maturely designed and optimized by domain experts. Typical learning-based image compression schemes provide BPG codecs for peak signal-to-noise ratio (PSNR) as a metric based on mean square error (MSE).

[0006] Typical autoencoder-based image compression frameworks use a Lagrange multiplier approach to optimize a weighted combination of ratio-distortion (RD) pairs. In order to adapt the quality (or ratio) of the compressed image, multiple models are trained separately for different values ​​of the Lagrange multiplier. That is, typical systems train and deploy multiple models for ratio adaptation, or keep the model while changing the Lagrange multiplier, which may not be achievable when it is desired to cover a wide range of RD curves with high resolution and the size of each network is large. Summary of the invention

[0007] According to one embodiment, a method includes: training a conditional autoencoder using Lagrange multipliers; and training a neural network including the conditional autoencoder with mixed quantization bin sizes.

[0008] According to one embodiment, a system includes a neural network including a conditional autoencoder and a processor configured to train the conditional autoencoder using Lagrangian multipliers and train the neural network using mixed quantization bin sizes. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0010] Figure 1A A diagram illustrating a variable ratio image compression model according to one embodiment;

[0011] Figure 1B A diagram illustrating a variable ratio image compression model according to one embodiment;

[0012] Figure 2 A flow chart showing a method for compression according to one embodiment;

[0013] Figure 3 A diagram illustrating a conditional autoencoder according to one embodiment;

[0014] Figure 4 A diagram showing an improved variable ratio compression model according to one embodiment;

[0015] Figure 5 A diagram illustrating a system for radio frequency integrated circuit (RFIC) analog-to-digital output compression according to one embodiment;

[0016] Figure 6 A diagram showing a conditional encoder according to one embodiment;

[0017] Figure 7 A diagram showing a conditional encoder according to one embodiment;

[0018] Figure 8 A diagram illustrating a conditional entropy model according to one embodiment;

[0019] Fig. 9 A diagram illustrating a conditional entropy model according to one embodiment;

[0020] Fig.10 A diagram showing a conditional decoder according to one embodiment;

[0021] Fig.11 A diagram showing a conditional decoder according to one embodiment;

[0022] Fig.12 A block diagram illustrating an electronic device in a network environment according to one embodiment. DETAILED DESCRIPTION

[0023] Hereinafter, embodiments of the present disclosure are described in detail with reference to the accompanying drawings. It should be noted that although the same elements are shown in different drawings, the same elements will be represented by the same reference numerals. In the following description, specific details (such as, detailed configuration and components) are only provided to help the overall understanding of the embodiments of the present disclosure. Therefore, it should be clear to those skilled in the art that various changes and modifications of the embodiments described herein can be made without departing from the scope of the present disclosure. In addition, for clarity and brevity, the description of well-known functions and configurations is omitted. The terms described below are defined in consideration of the functions in the present disclosure, and may be different according to the user, the user's intention or custom. Therefore, the definition of the term should be determined based on the content throughout this specification.

[0024] The present disclosure may have various modifications and various embodiments, and the embodiments thereof are described in detail below with reference to the accompanying drawings. However, it should be understood that the present disclosure is not limited to these embodiments, but includes all modifications, equivalents and substitutes within the scope of the present disclosure.

[0025] Although the term including ordinal numbers (such as, first, second, etc.) can be used to describe various elements, structural elements are not limited by the term. The term is only used to distinguish one element from another element. For example, without departing from the scope of the present disclosure, the first structural element may be referred to as the second structural element. Similarly, the second structural element may also be referred to as the first structural element. As used herein, the term "and / or" includes any combination and all combinations of one or more related items.

[0026] The terms used herein are only used to describe various embodiments of the present disclosure and are not intended to limit the present disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In the present disclosure, it should be understood that the term "including" or "having" indicates the presence of features, quantities, steps, operations, structural elements, parts or combinations thereof, and does not exclude the presence or possible addition of one or more other features, quantities, steps, operations, structural elements, parts or combinations thereof.

[0027] Unless defined differently, all terms used herein have the same meaning as understood by a person skilled in the art to which the present disclosure belongs. Unless clearly defined in the present disclosure, terms (such as those defined in a general dictionary) will be interpreted as having the same meaning as the contextual meaning in the relevant technical field and will not be interpreted as having an idealized or overly formal meaning.

[0028] The electronic device according to one embodiment may be one of various types of electronic devices. For example, the electronic device may include a portable communication device (e.g., a smart phone), a computer, a portable multimedia device, a portable medical device, a camera, a wearable device, or a household appliance. According to one disclosed embodiment, the electronic device is not limited to the electronic device described above.

[0029] The terms used in this disclosure are not intended to limit the disclosure, but are intended to include various variations, equivalents or alternatives of corresponding embodiments. With regard to the description of the accompanying drawings, similar reference numerals may be used to represent similar or related elements. Unless the relevant context clearly indicates otherwise, the singular form of the noun corresponding to the project may include one or more things. As used herein, each of the phrases such as "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C" and "at least one of A, B or C" may include all possible combinations of the project listed together with the corresponding one in the phrase. As used herein, terms such as "the 1st", "the 2nd", "the first" and "the second" may be used to distinguish a corresponding component from another component, and are not intended to limit the component in other aspects (e.g., importance or order). It is intended that if an element (e.g., a first element) is referred to as being "coupled with" another element (e.g., a second element), "coupled to" another element, "connected with" another element, or "connected to" another element, with or without the term "operably" or "communicatively", it indicates that the element can be coupled with the other element directly (e.g., wired), wirelessly, or via a third element.

[0030] As used herein, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms (e.g., "logic," "logic block," "component," or "circuit"). A module may be a single integrated component or its smallest unit or component adapted to perform one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application specific integrated circuit (ASIC).

[0031] Variable rate compression can exploit neural network functionality and can be applied to a variety of fields including, but not limited to, image compression and radio frequency integrated circuit (RFIC) output compression.

[0032] According to one embodiment, the present system and method provide a single variable rate image compression network that performs rate adaptation. In particular, the network includes a conditional autoencoder that is conditioned on Lagrangian multipliers (i.e., the network takes Lagrangian multipliers as input and produces a latent representation whose compression ratio (e.g., bits per pixel (BPP) in image compression) depends on the input values). In addition, the network is trained with mixed quantization bin sizes that allow adaptation of the compression rate by adjusting the bin size of the quantization applied to the latent representation. Coarse compression rate adaptation to a target is achieved by changing the Lagrangian multipliers in the conditional model, while fine compression rate adaptation is performed by adjusting the quantization bin size.

[0033] Figure 1A A diagram of a variable rate image compression model 100 according to one embodiment is shown. The model 100 includes a conditional autoencoder 102, a conditional decoder 104, and a conditional entropy model 106. The autoencoder 102 is conditioned on a Lagrangian multiplier λ 108, which adapts the compression ratio and optimizes the rate-distortion Lagrangian operator for various λ values ​​in a conditional model. The model 100 is trained for a mixture of quantization bin size Δ 110 and a universal quantization (or universal quantization function) 111, so that the compression ratio can be varied by changing Δ 110. The multiplier 108 is utilized in both the conditional entropy model 106 and the conditional decoder 104. The model 100 utilizes entropy encoding 112 and entropy decoding 114.

[0034] The conditional autoencoder 102 receives the hyperparameters of the optimization problem (i.e., the Lagrange multiplier λ 108) as its conditioning variables. The present system and method uses one conditional network to solve multiple objectives, rather than using separate unconditional networks to solve the objectives separately. The model 100 is based on a conditional autoencoder that does not require multiple iterations, and the quality is controlled by the conditioning variables λ 108 and Δ 110 of the model 100.

[0035] Figure 1B A diagram of a variable ratio image compression model 150 according to one embodiment is shown. The model 150 includes a first conditional autoencoder 152, a second conditional autoencoder 154, a first conditional entropy model 156, a second conditional entropy model 158, a first conditional autodecoder 160, and a second conditional autodecoder 162. The model 150 also includes a first general quantization function 164 and a second general quantization function 166, as well as a first arithmetic encoding (AE) / arithmetic decoding (AD) function 168 and a second AE / AD function 170. As described below, a hyperparameter 172 is utilized at the general quantization function 164 and the general quantization function 166.

[0036] The autoencoder architecture consists of an encoder f φ (x) and decoder g θ (z), where x is the input range and z = round Δ (f φ (x)) is the quantized latent representation encoded from the input x with quantization bin size Δ, where round() is the rounding function and round Δ (x) = Δ round (x / Δ). In the following equation, Δ is fixed to 1. Lossless entropy source coding is used to generate a compressed bitstream from a quantized representation z. Taking the entropy source coding for the quantized latent variable z and obtaining its entropy ratio, the compression ratio R and the distortion D (i.e., L2 loss or MSE loss) are given by equation (1):

[0037]

[0038] Among them, φ and θ are encoder f φ (x) and decoder g θ (z) and as in equation (2):

[0039] P φ (x) = ∫p(x)δ(z-round Δ (f φ (x)))dx (2)

[0040] p(x) is the probability density function of all natural images, and P φ(z) is the value from the encoder f φ (x) and round Δ The derived probability mass function of z satisfies equation (2), where δ represents the Dirac delta function. Using the Lagrange multiplier method, for a fixed Lagrange multiplier λ>0, the RD optimization problem is expressed as equation (3):

[0041]

[0042] The compression ratio and distortion provided in equation (1) are due to P φ (z) and round Δ However, the network parameter φ is not differentiable, and therefore equation (3) cannot be optimized directly.

[0043] According to one embodiment, the system uses universal quantization. As in equation (4), universal quantization dithers the latent representation f with a common uniform random variable φ For each element in (x):

[0044] z=round Δ (f φ (x)+u)-u,u=[U,U,...,U] (4)

[0045] where the jitter vector u consists of multiple repetitions of a single uniform random variable U with support [-Δ / 2, Δ / 2]. In the back-propagation of network training, universal quantization is approximated as a linear function of unit slope. Although the noise derived from universal quantization depends across dimensions, in each dimension, universal quantization is effectively equivalent to adding uniform noise in the distribution.

[0046] Under relaxation with universal quantization, similar to equation (1), the compression ratio and distortion can be expressed as equation (5):

[0047]

[0048] Among them, p φ (z) = ∫p(x)p φ (z|x)dx. The compression ratio is now a differential entropy, which is differentiable, but it is still difficult to calculate p in general. φ (z). In order to calculate p φ (z), a tractable density function q θ (z) is introduced and equation (6) is derived as:

[0049]

[0050] Where KL represents the Kullback-Leibler divergence. φ (z) = q θ (z), the equality holds. For a fixed λ>0, let R φ Approaching its upper bound R φ，θ , the RD optimization in equation (3) is simplified to equation (7):

[0051]

[0052] By optimizing the network for different values ​​of λ, the quality can be traded off against the compression ratio. With a smaller Lagrangian multiplier λ, the trained network produces higher quality images while using a larger ratio.

[0053] In order to adapt the quality and compression ratio of the compressed image, the system needs to optimize the RD Lagrangian function in equation (7) for the changing value of the Lagrangian multiplier λ. A variable ratio model is disclosed herein, which can be deployed once and can be used to generate compressed images of varying quality with different ratios according to the user's needs without the need for retraining.

[0054] Figure 2 A flow chart 200 of a method for compression according to one embodiment is shown. At 202, the system trains a conditional autoencoder. The system can train a conditional autoencoder conditioned on a Lagrangian multiplier λ. The network receives λ as a conditional input parameter and an input image, and produces a compressed image with a varying compression ratio and distortion according to the conditional value λ. Therefore, for λ∈Λ, where Λ is a finite set of predefined Lagrangian multiplier values, the compression ratio and distortion terms in equations (5) and (6) are changed to equations (8) and (9):

[0055]

[0056]

[0057] Then, the combined objective function of equation (10) is minimized.

[0058]

[0059] Figure 3A diagram of a conditional autoencoder 300 according to one embodiment is shown. To implement the conditional autoencoder, a conditional convolution conditioned on a Lagrangian multiplier is developed. The system sends input 302 through a convolutional layer 304, while the system sends a multiplier λ 306 through a one-hot encoding layer 308, the output of which is then split into the inputs of a first fully connected layer 310 and a second fully connected layer 312. The system sends the output of the first fully connected layer 310 to a SoftPlus function block 314, the output of which is multiplied at 316 with the output of the convolutional layer 304. The system then accumulates the multiplied output 318 with the output from the fully connected layer 312 using an adder 319 to produce an output 320. The convolution is conditioned on the Lagrangian multiplier λ 306, which produces different outputs depending on the Lagrangian multiplier λ 306 of the input.

[0060] Specifically, refer to Figure 3 , X i is the two-dimensional (2-D) input of channel i, Y j is the 2-D output of channel j, and W i，j is a 2-D convolution kernel for input channel i and output channel j. The conditional convolution is given by equation (11):

[0061]

[0062] Where * denotes 2-D convolution. As in Equation (12), the channel-wise scaling factor s j (λ) and the additive bias term b j (λ) depends on λ:

[0063]

[0064] in, and is the fully connected layer weight vector of length |Λ| for output channel j, T denotes the transpose, and onehot(λ) is the one-hot encoding of λ within Λ.

[0065] Reference Figure 2, at 204, the system trains the network with mixed quantization bin sizes. Since λ is selected from a predetermined finite set Λ, the above conditional autoencoder can only obtain a finite number of discrete points in the RD curve. In order to extend the coverage to the entire continuous range of the RD curve, another control aspect in the network (i.e., quantization bin size) is included to control the compression ratio. In the RD formula in equation (1), the quantization bin size is fixed to Δ=1 (i.e., round is used for quantization). In actual inference, the bin size can be changed to adapt to the compression ratio, so that the larger the bin size, the lower the compression ratio. However, performance naturally suffers from mismatched bin sizes in training and inference, so the system trains the network with mixed bin sizes.

[0066] During training, the system extracts uniform noise for various noise levels (for random Δ). The range of Δ and the distribution of mixtures within that range are design choices. In one example, Δ=2 b is utilized, where b is uniformly drawn from [-1, 1], so Δ∈[0.5, 2] is covered. Although performance may degrade, the larger the range of b, the wider the range of RD curves that can optimize the network. As in equation (13), the optimization is solved:

[0067]

[0068] where p(Δ) is the predefined mixing density of Δ and as in equation (14):

[0069]

[0070] Reference Figure 2 At 206, the system performs extrapolation. With respect to compression ratio adaptation, the compression ratio increases as the Lagrangian multiplier and / or the quantization bin size decreases. Given a user's target compression ratio, the system implements a wide range of discrete compression ratio adaptation by varying the Lagrangian multiplier, while for a fixed Lagrangian multiplier, the system can perform fine-grained continuous compression ratio adaptation by adjusting the quantization bin size.

[0071] Regarding compression, after choosing λ∈Λ, the system performs one-hot encoding of the Lagrange multipliers and uses one-hot encoding in all conditional convolutional layers to encode the latent representation of the input. The system performs conventional deterministic quantization on the encoded representation with the selected quantization bin size. The system then encodes the quantized latent representation into a compressed bitstream using entropy coding while storing the values ​​of the conditioning variables used in the encoding.

[0072] On decompression, the system decodes the compressed bitstream and obtains the conditioning variables used in encoding from the compressed bitstream. The system recovers the quantized latent representation from the decoder integer values ​​by multiplying the decoder integer values ​​by the quantization bin size. The system then feeds the recovered latent representation to the decoder to reconstruct the image. The values ​​of the Lagrange multipliers used in encoding are reused in all deconvolution layers for conditional generation.

[0073] Figure 4 A diagram showing an improved variable ratio compression model according to one embodiment. Figure 4 , an encoder 400 and a decoder 402 are depicted. The system improves the probability model by introducing a secondary latent variable w 404 that depends on x 406 and z 408. The ratio and distortion terms given in equation (14) are then given by equation (15).

[0074]

[0075] For compression, the system encodes z 408 from x 406, and further encodes w 404 from z(408) and x(406). The encoding represents z 408 and w 404, respectively, based on q φ (w|λ,Δ) and q φ (z|w,λ,Δ) is entropy coded. For decompression, given q φ (w|λ,Δ), the system decodes w 404 and then uses w 404 to calculate q φ (z|w, λ, Δ) and decode z 408. As shown in equation (16), this model introduces q φ (w|λ,Δ) and q φ The autoregressive model of (z|w, λ, Δ) is further improved.

[0076]

[0077] The system replaces the lossless source coded portion of the encoded representation with lossy compression to adapt the ratio. It should be noted that any type of lossy compression codec can be used, including but not limited to Better Portable Graphics (BPG). By using lossy compression instead of lossless compression of the encoded feature map, the system can fine-tune the decoder for lossy compressed output to improve accuracy.

[0078] For BPG lossy compression, the system may rearrange the encoded feature map into a grayscale image. The system further compresses the rearranged grayscale image into BPG lossy compression. For decompression, the system decompresses the feature map from BPG and then passes the decompressed feature map to the decoder to obtain the decompressed image.

[0079] According to another embodiment, the system and method may be applied in the field of wireless communication chipsets.

[0080] Figure 5 A diagram of a system 500 for RFIC ADC output compression according to one embodiment is shown. The system includes an RFIC 502, a variable ratio compression network 504, a baseband modem 506, and a lookup table 508. The system adapts the compression ratio (e.g., bit / subframe, bit / frame, bit / time slot, etc.) based on a modulation coding scheme (MCS) 510 determined by an operating signal-to-noise ratio (SNR). The system may select conditional variables λ, Δ as a function of the MCS that determines the compression ratio and distortion. Here, the distortion level may be below the operating SNR to avoid any performance degradation due to compression. The system may include a lookup table 508 that maps the MCS to λ, Δ.

[0081] The system can apply the present variable ratio compression scheme to compress the analog-to-digital converter (ADC) 502 output of a radio frequency integrated circuit (RFIC). The system can compress the ADC 502 output using a compression network 504 and pass the compressed ADC output to a baseband (BB) modem 506 to reduce the communication cost (i.e., bandwidth and / or number of channels) between the RFIC and the BB. Compression can be performed per time slot, per subframe, or per frame, depending on the bandwidth and latency that can be tolerated.

[0082] Figure 6 FIG. 6 shows a first conditional encoder 600 according to one embodiment. The conditional encoder 600 may be similar to Figure 1A The conditional encoder 102 and Figure 1B 6. An input (e.g., image, ADC output, etc.) 602 is provided and the system processes the input 602 through a first 192×5×5 / 2 conditional convolution layer (CConv) 604, a first generalized divisive normalization (GDN) layer 606, a second 192×5×5 / 2 conditional convolution layer 608, a second GDN layer 610, a third 192×5×5 / 2 conditional convolution layer 612, a third GDN layer 614, a 384×5×5 / 2 conditional convolution layer 616, and a universal quantization (UnivQuant) layer 618 that produces an output 620. The third GDN layer 614 produces an output 622, which is sent to a 192×5×5 / 2 conditional convolution layer 624 and then to a leaky rectified linear unit (ReLU) activation function 626, which produces an output 628.

[0083] Figure 7 The second conditional encoder 700 according to one embodiment is shown. The second conditional encoder 700 may be similar to Figure 1BThe conditional encoder 154 depicted in FIG. The concatenation (Concat) layer 702 receives the output 620 and the output 628, and the system produces an output processed by a 192×3×3 / 1 conditional convolution layer 704, a first leaky ReLU activation function 706, a first 192×5×5 / 2 conditional convolution layer 708, a second leaky ReLU activation function 710, a second 192×5×5 / 2 conditional convolution layer 712, and a UnivQuant layer 714 that produces an output 716.

[0084] Figure 8 FIG. 8 shows a conditional entropy model 800 according to one embodiment. The entropy model 800 may be used with Figure 1B The entropy model 800 receives the output 716 processed by the AE 802 and the AD 804. The entropy model 800 processes the output 716 through a masked 384×5×5 / 1 conditional convolution layer 806, a 640×1×1 / 1 conditional convolution layer 808, a first leaky ReLU activation function 810, a 512×1×1 / 1 conditional convolution layer 812, and a second leaky ReLU activation function 814. The output of the ReLU function 814 is processed by a first 192×1×1 / 1 conditional convolution layer 816 and a second 192×1×1 / 1 conditional convolution layer 818. The output of the second conditional convolution layer 818 is processed by a third ReLU activation function 820, and the outputs of the first conditional convolution layer 816 and the ReLU function 820 are processed to produce a Gaussian model 822. Gaussian model 822 is utilized with AE 802 and AD 804 to produce output 826 .

[0085] Fig. 9 FIG. 9 shows a conditional entropy model 900 according to one embodiment. The conditional entropy model 900 may be similar to Figure 1A The conditional entropy model depicted 106 and Figure 1B8. The masked 768×5×5 / 1 conditional convolution layer 914 receives input 826 and produces output 916. In addition, the system receives output 826 and processes output 826 through 384×5×5 / 2 conditional convolution layer 902, first leaky ReLU activation function 904, 576×5×5 / 2 conditional convolution layer 906, second leaky ReLU activation function 908, and 768×3×3 / 1 conditional convolution layer 610. The system processes the output of layer 910 and output 916 through a Concat function, which generates an output that is processed through a 1280×1×1 / 1 conditional convolution layer 918, a third leaky ReLU activation function 920, a 1024×1×1 / 1 conditional convolution layer 922, and a fourth leaky ReLU activation function 924 to produce output 926. The system then processes the output 926 through a first 384×1×1 / 1 conditional convolution layer 928 and a second 384×1×1 / 1 conditional convolution layer 930. The output of the first 384×1×1 / 1 conditional convolution layer 928 is processed through a ReLU activation function 932, and the output 931 of the ReLU activation function and the output 933 of the second 384×1×1 / 1 conditional convolution layer 930 are processed through a Gaussian density model 934.

[0086] The system processes the output 936 of the Gaussian density model 934 and the output 620 from the conditional encoder 600 with an autoencoder (AE) 938, which generates a signal processed by a quantization table 939. The system processes the output of the quantization table 939 and the output 936 of the Gaussian density model 934 with a decoder 940 to generate an output 942.

[0087] Fig.10 FIG. 1 shows a conditional decoder 1000 according to one embodiment. The conditional decoder 1000 may be similar to Figure 1B 1. The conditional decoder 160 depicted in FIG. The decoder 1000 receives the output 826 and processes the output 826 through a first 192×5×5 / 2 conditional convolutional layer 1002, a first leaky ReLU activation function 1004, a second 192×5×5 / 2 conditional convolutional layer 1006, a second leaky ReLU activation function 1008, a 192×3×3 / 1 conditional convolutional layer 1010, and a third leaky ReLU activation function 1012 to produce an output 1014.

[0088] Fig.11 FIG. 1 shows a conditional decoder 1100 according to one embodiment. The conditional decoder 1100 may be similar to Figure 1A The decoder 104 and Figure 1B1014 and 940 and processes them at a Concat function 1102. The system processes the output of the Concat function 1102 to a first 192×5×5 / 2 conditional convolution layer 1104, a first inverse generalized divisive normalization (IGDN) layer 1106, a second 192×5×5 / 2 conditional convolution layer 1108, a second IGDN layer 1110, a third 192×5×5 / 2 conditional convolution layer 1112, a third IGDN layer 1114, and a 3×5×5 / 2 conditional convolution layer 1116 to produce a reconstructed input 1118.

[0089] Fig.12 An electronic device 1201 is shown in a network environment 1200 according to one embodiment. Fig.12 , the electronic device 1201 in the network environment 1200 may communicate with the electronic device 1202 via a first network 1298 (e.g., a short-range wireless communication network), or communicate with the electronic device 1204 or the server 1208 via a second network 1299 (e.g., a long-range wireless communication network). The electronic device 1201 may communicate with the electronic device 1204 via the server 1208. The electronic device 1201 may include a processor 1220, a memory 1230, an input device 1250, a sound output device 1255, a display device 1260, an audio module 1270, a sensor module 1276, an interface 1277, a haptic module 1279, a camera module 1280, a power management module 1288, a battery 1289, a communication module 1290, a user identification module (SIM) 1296, and / or an antenna module 1297. In one embodiment, at least one of the components (e.g., display device 1260 or camera module 1280) may be omitted from electronic device 1201, or one or more other components may be added to electronic device 1201. In one embodiment, some of the components may be implemented as a single integrated circuit (IC). For example, sensor module 1276 (e.g., fingerprint sensor, iris sensor, or illumination sensor) may be embedded in display device 1260 (e.g., display).

[0090] The processor 1220 may execute, for example, software (e.g., program 1240) to control at least one other component (e.g., hardware component or software component) of the electronic device 1201 connected to the processor 1220, and may perform various data processing or calculations. As at least part of the data processing or calculation, the processor 1220 may load a command or data received from another component (e.g., sensor module 1276 or communication module 1290) into the volatile memory 1232, process the command or data stored in the volatile memory 1232, and store the result data in the non-volatile memory 1234. The processor 1220 may include a main processor 1221 (e.g., a central processing unit (CPU) or an application processor (AP)) and an auxiliary processor 1223 (e.g., a graphics processor (GPU), an image signal processor (ISP), a sensor hub processor, or a communication processor (CP)), and the auxiliary processor 1223 may operate independently of the main processor 1221, or operate together with the main processor 1221. Additionally or alternatively, the auxiliary processor 1223 may be adapted to consume less power than the main processor 1221, or to perform a specific function. The auxiliary processor 1223 may be implemented separately from the main processor 1221, or as part of the main processor 1221.

[0091] The auxiliary processor 1223 may replace the main processor 1221 when the main processor 1221 is in an inactive (e.g., sleeping) state, or control at least some of the functions or states related to at least one of the components of the electronic device 1201 (e.g., the display device 1260, the sensor module 1276, or the communication module 1290) together with the main processor 1221 when the main processor 1221 is in an active state (e.g., executing an application). According to one embodiment, the auxiliary processor 1223 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., a camera module 1280 or a communication module 1290) that is functionally related to the auxiliary processor 1223.

[0092] The memory 1230 may store various data used by at least one component (e.g., the processor 1220 or the sensor module 1276) in the electronic device 1201. The various data may include, for example, input data or output data of software (e.g., the program 1240) and commands related thereto. The memory 1230 may include a volatile memory 1232 or a nonvolatile memory 1234.

[0093] The program 1240 may be stored as software in the memory 1230 , and may include, for example, an operating system (OS) 1242 , middleware 1244 , or an application 1246 .

[0094] The input device 1250 may receive commands or data to be used by other components (eg, the processor 1220) of the electronic device 1201 from outside the electronic device 1201 (eg, a user). The input device 1250 may include, for example, a microphone, a mouse, or a keyboard.

[0095] The sound output device 1255 can output a sound signal to the outside of the electronic device 1201. The sound output device 1255 may include, for example, a speaker or a receiver. The speaker can be used for general purposes (such as playing multimedia or recording), and the receiver can be used to receive an incoming call. According to one embodiment, the receiver can be implemented as a separate speaker or as part of the speaker.

[0096] The display device 1260 may visually provide information to the outside of the electronic device 1201 (e.g., a user). The display device 1260 may include, for example, a display, a hologram device, or a projector, and a control circuit for controlling a corresponding one of the display, the hologram device, and the projector. According to one embodiment, the display device 1260 may include a touch circuit adapted to detect a touch, or a sensor circuit (e.g., a pressure sensor) adapted to measure the strength of a force caused by a touch.

[0097] The audio module 1270 may convert sound into an electrical signal, and vice versa. According to one embodiment, the audio module 1270 may obtain sound via the input device 1250, or output sound via the sound output device 1255 or an earphone of an external electronic device 1202 directly (e.g., wired) or wirelessly connected to the electronic device 1201.

[0098] The sensor module 1276 may detect an operating state (e.g., power or temperature) of the electronic device 1201 or an environmental state (e.g., a user's state) outside the electronic device 1201, and then generate an electrical signal or data value corresponding to the detected state. The sensor module 1276 may include, for example, a posture sensor, a gyro sensor, an atmospheric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illumination sensor.

[0099] The interface 1277 may support one or more specific protocols for direct (e.g., wired) or wireless connection of the electronic device 1201 with the external electronic device 1202. According to one embodiment, the interface 1277 may include, for example, a high-definition multimedia interface (HDMI), a universal serial bus (USB) interface, a secure digital (SD) card interface, or an audio interface.

[0100] The connection terminal 1278 may include a connector via which the electronic device 1201 may be physically connected to the external electronic device 1202. According to one embodiment, the connection terminal 1278 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

[0101] The haptic module 1279 may convert an electrical signal into mechanical stimulation (eg, vibration or movement) or electrical stimulation, which may be recognized by the user via tactile or kinesthetic sense. According to one embodiment, the haptic module 1279 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0102] The camera module 1280 may capture still images or moving images. According to one embodiment, the camera module 1280 may include one or more lenses, image sensors, image signal processors, or flashes.

[0103] The power management module 1288 may manage power supplied to the electronic device 1201. The power management module 1288 may be implemented as, for example, at least a portion of a power management integrated circuit (PMIC).

[0104] The battery 1289 may supply power to at least one component of the electronic device 1201. According to one embodiment, the battery 1289 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0105] The communication module 1290 may support establishing a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device 1201 and an external electronic device (e.g., electronic device 1202, electronic device 1204, or server 1208), and support communication via the established communication channel. The communication module 1290 may include one or more communication processors that can operate independently of the processor 1220 (e.g., AP) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module 1290 may include a wireless communication module 1292 (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 1294 (e.g., a local area network (LAN) communication module or a power line communication (PLC) module). A corresponding one of these communication modules may communicate via a first network 1298 (e.g., a short-range communication network (such as Bluetooth TM), Wireless Fidelity (Wi-Fi) Direct, or Infrared Data Association (IrDA) standards) or a second network 1299 (e.g., a long-distance communication network such as a cellular network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN))))) to communicate with an external electronic device. These various types of communication modules may be implemented as a single component (e.g., a single IC), or may be implemented as multiple components separated from each other (e.g., multiple ICs). The wireless communication module 1292 may use user information (e.g., an International Mobile Subscriber Identity (IMSI)) stored in the user identification module 1296 to identify and authenticate the electronic device 1201 in a communication network (such as the first network 1298 or the second network 1299).

[0106] The antenna module 1297 may transmit a signal or power to the outside of the electronic device 1201 (e.g., an external electronic device), or receive a signal or power from the outside of the electronic device 1201 (e.g., an external electronic device). According to one embodiment, the antenna module 1297 may include one or more antennas, for example, the communication module 1290 (e.g., the wireless communication module 1292) may select at least one antenna suitable for a communication scheme used in a communication network (such as the first network 1298 or the second network 1299) from the one or more antennas. Then, a signal or power may be transmitted or received between the communication module 1290 and the external electronic device via the selected at least one antenna.

[0107] At least some of the above components may be connected to each other via an inter-peripheral communication scheme (e.g., a bus, a general purpose input output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI)) and transfer signals (e.g., commands or data) therebetween.

[0108] According to one embodiment, a command or data may be sent or received between the electronic device 1201 and the external electronic device 1204 via the server 1208 connected to the second network 1299. Each of the electronic devices 1202 and 1204 may be a device of the same type or a different type as the electronic device 1201. All or some of the operations to be performed at the electronic device 1201 may be performed at one or more of the external electronic device 1202, the external electronic device 1204, or the external electronic device 1208. For example, if the electronic device 1201 should perform a function or service automatically or in response to a request from a user or another device, the electronic device 1201 may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service, or in addition to performing the function or service, the electronic device 1201 may also request one or more external electronic devices to perform at least part of the function or service. The one or more external electronic devices that receive the request may perform at least part of the requested function or service or an additional function or additional service related to the request, and transmit the result of the execution to the electronic device 1201. The electronic device 1201 may provide the result, with or without further processing of the result, as at least part of a reply to the request. To this end, for example, cloud computing, distributed computing, or client-server computing techniques may be used.

[0109] One embodiment may be implemented as software (e.g., program 1240), which includes one or more instructions stored in a storage medium (e.g., internal memory 1236 or external memory 1238) that can be read by a machine (e.g., electronic device 1201). For example, the processor of the electronic device 1201 may call at least one of the one or more instructions stored in the storage medium, and use or do not use one or more other components under the control of the processor to execute the at least one instruction. Thus, the machine may be operated to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. The term "non-transitory" indicates that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but this term does not distinguish between a case where data is semi-permanently stored in a storage medium and a case where data is temporarily stored in a storage medium.

[0110] According to one embodiment, the disclosed method may be included and provided in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or via an application store (e.g., Play Store TM) online distribution (e.g., downloading or uploading), or directly between two user devices (e.g., smartphones). If distributed online, at least part of the computer program product may be temporarily generated or at least temporarily stored in a machine-readable storage medium (such as a memory of a manufacturer's server, an application store's server, or a relay server).

[0111] According to one embodiment, each of the above-mentioned components (e.g., a module or a program) may include a single entity or multiple entities. One or more of the above-mentioned components may be omitted, or one or more other components may be added. Alternatively or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, the integrated component may still perform one or more functions of each of the multiple components in the same or similar manner as one or more functions were performed by corresponding components in the multiple components before integration. The operations performed by a module, a program or another component may be performed sequentially, in parallel, repeatedly or heuristically, or one or more of the operations may be performed in a different order or one or more of the operations may be omitted, or one or more other operations may be added.

[0112] Although specific embodiments of the present disclosure have been described in the specific embodiments of the present disclosure, the present disclosure may be modified in various forms without departing from the scope of the present disclosure. Therefore, the scope of the present disclosure should not be determined based solely on the described embodiments, but should be determined based on the attached claims and their equivalents.

Claims

1. A method for compression, comprising: Use Lagrange multipliers to train conditional autoencoders; Training neural networks including conditional autoencoders with mixed quantization bin sizes; receiving data including an analog-to-digital converter output of an image or radio frequency integrated circuit; and The received data is compressed using the trained neural network.

2. The method according to claim 1, wherein: The values ​​of the Lagrange multipliers are selected from a predetermined set of values.

3. The method according to claim 1, wherein: The neural network produces a compressed output having a compression ratio and distortion that varies according to the conditional values ​​of the Lagrange multipliers.

4. The method according to claim 1, wherein: Use conditional convolutions to train conditional autoencoders.

5. The method according to claim 1, wherein: The step of training a neural network with a mixed quantization bin size also includes: adjusting the quantization bin size.

6. The method according to claim 1, wherein: The step of training the neural network with the mixed quantization bin sizes further includes extracting uniform noise for various noise levels for the random quantization bin sizes.

7. The method according to claim 1, wherein: The neural network is configured to: Perform better Portable Graphics (BPG) compression.

8. The method according to claim 1, wherein: The neural network is configured to perform compression of an analog-to-digital converter output of the radio frequency integrated circuit.

9. The method according to claim 8, further comprising: The Lagrange multipliers and quantization bin size are chosen as a function of the modulation coding scheme.

10. The method according to claim 9, wherein: The neural network also includes a lookup table that maps modulation coding schemes to Lagrangian multipliers and quantization bin sizes.

11. A system for compression, comprising: Neural networks, including conditional autoencoders; and The processor is configured as: Use Lagrange multipliers to train conditional autoencoders; Training neural networks using mixed quantization bin sizes; receiving data including an image or an analog-to-digital converter output of a radio frequency integrated circuit; and The received data is compressed using the trained neural network.

12. The system according to claim 11, wherein: The values ​​of the Lagrange multipliers are selected from a predetermined set of values.

13. The system according to claim 11, wherein: The neural network produces a compressed output having a compression ratio and distortion that varies according to the conditional values ​​of the Lagrange multipliers.

14. The system according to claim 11, wherein: The processor is configured to train a conditional autoencoder using conditional convolution.

15. The system according to claim 11, wherein: The processor is further configured to train a neural network with the mixed quantization bin size by adjusting the quantization bin size.

16. The system according to claim 11, wherein: The processor is further configured to train a neural network with mixed quantization bin sizes by extracting uniform noise for various noise levels for random quantization bin sizes.

17. The system of claim 11, wherein: The neural network is configured to: Perform better Portable Graphics (BPG) compression.

18. The system of claim 11, wherein: The neural network is configured to perform compression of an analog-to-digital converter output of the radio frequency integrated circuit.

19. The system of claim 18, the processor further configured to select the Lagrangian multiplier and the quantization bin size as a function of a modulation coding scheme.

20. The system of claim 19, wherein: The neural network also includes a lookup table that maps modulation coding schemes to Lagrangian multipliers and quantization bin sizes.

Images