Methods, apparatuses, and media for end-to-end task-oriented latent image compression using deep reinforcement learning

Abstract

The present disclosure provides an end-to-end task-oriented latent compression method, device and medium using deep reinforcement learning. The method is executed by at least one processor and comprises: generating a plurality of latent representations of an input image using a first neural network, wherein the plurality of latent representations are a sequence of latent signals; encoding the plurality of latent signals using a second neural network; generating a set of quantization keys based on a set of previous quantization states using a third neural network, wherein each quantization key in the set of quantization keys and each previous quantization state in the set of previous quantization states correspond to each latent signal in the plurality of latent signals; generating a set of dequantized numbers representing dequantized representations of the encoded plurality of latent signals based on the set of quantization keys using a fourth neural network; generating a reconstructed output based on the set of dequantized numbers; and performing a target task based on the reconstructed output using a fifth neural network.

Description

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to U.S. Provisional Patent Application No. 63 / 133,696, filed January 4, 2021, and U.S. Patent Application No. 17 / 478,089, filed September 17, 2021, the entire contents of which are incorporated herein by reference. Background Technology

[0003] International standardization organizations ISO / IEC / IEEE are actively searching for AI-based video coding and decoding technologies, with a particular focus on technologies based on deep neural networks (DNNs). Various AhGs have been formed to study neural network compression (NNR), video coding for machines (VCM), and neural network-based video coding (NNVC), among others. The Artificial Intelligence Industry Technology Innovation Strategic Alliance (AITISA) and the Audio Video Coding Standard (AVS) in China have also established corresponding expert groups to study the standardization of similar technologies.

[0004] The process of end-to-end latent representation compression (E2ELRC) can be described as follows: Given an input image or video sequence x, the DNN latent generator first computes a latent representation f, f is then passed to the DNN encoder to compute a compact representation y, and y is quantized into a discrete-valued quantized representation. This discrete value represents Lossless entropy encoding can be performed for easy storage and transmission. At the decoder end, discrete values ​​are represented... It can be recovered from lossless entropy decoding and used as input to a DNN decoder to compute the reconstructed latent representation. Then, the DNN task executor is based on the reconstructed latent representation. To perform target tasks such as detection, recognition, and segmentation. In other words, without encoding and decoding processes (from latent representation f to reconstructed latent representation). In the case of [missing information], the original DNN latent generator will compute the latent representation f, and the DNN task executor will directly use the latent representation f to perform the target task. Therefore, the reconstructed latent representation [missing information]. It can be viewed as a modified version of the latent representation f. The goal of E2ELRC is to find an efficient encoder-decoder mechanism that enables compact representations. It is efficient for storage and transmission, and recovers the potential representation of the reconstruction. It can maintain the original task performance.

[0005] Quantization is a core process in all compression standards and production of images, videos, and latent features. Quantization is also a major source of compression quality loss, and improving quantization efficiency can bring significant performance gains in image and video compression tasks. Summary of the Invention

[0006] According to an embodiment, an end-to-end task-oriented latent image compression method using deep reinforcement learning is executed by at least one processor and includes: generating a plurality of latent representations of an input image using a first neural network, wherein the plurality of latent representations include sequences of latent signals; encoding the plurality of latent representations using a second neural network; generating a set of quantization keys based on a set of previous quantization states using a third neural network, wherein each quantization key in the set of quantization keys and each previous quantization state in the set of previous quantization states correspond to the plurality of latent representations; generating a set of dequantized numbers based on the set of quantization keys using a fourth neural network, the set of dequantized numbers representing dequantized representations of the encoded plurality of latent representations; generating a reconstructed output based on the set of dequantized numbers; and performing a target task using a fifth neural network based on the reconstructed output.

[0007] According to an embodiment, an end-to-end task-oriented latent image compression apparatus using deep reinforcement learning includes: at least one memory for storing program code; and at least one processor for reading the program code and operating according to the instructions of the program code. The program code includes: first generation code for causing the at least one processor to use a first neural network to generate a plurality of latent representations of an input image, wherein the plurality of latent representations include a sequence of latent signals; encoding code for causing the at least one processor to use a second neural network to encode the plurality of latent representations; second generation code for causing the at least one processor to use a third neural network to generate a set of quantization keys based on a set of previous quantization states, wherein each quantization key in the set of quantization keys and each previous quantization state in the set of previous quantization states corresponds to the plurality of latent representations; third generation code for causing the at least one processor to use a fourth neural network to generate a set of dequantized numbers based on the set of quantization keys, the set of dequantized numbers representing dequantized representations of the encoded plurality of latent representations; decoding code for causing the at least one processor to generate a reconstructed output based on the set of dequantized numbers; and execution code for causing the at least one processor to use a fifth neural network to perform a target task based on the reconstructed output.

[0008] According to an embodiment, a non-volatile computer-readable medium storing instructions is provided, wherein when at least one processor executes the instructions to perform end-to-end task-oriented latent image compression using deep reinforcement learning, the at least one processor: generates a plurality of latent representations of an input image using a first neural network, wherein the plurality of latent representations include a sequence of latent signals; encodes the plurality of latent representations using a second neural network; generates a set of quantization keys using a third neural network based on a set of previous quantization states, wherein each quantization key in the set of quantization keys and each previous quantization state in the set of previous quantization states corresponds to the plurality of latent representations; generates a set of dequantized numbers using a fourth neural network based on the set of quantization keys, the set of dequantized numbers representing dequantized representations of the encoded plurality of latent representations; generates a reconstructed output based on the set of dequantized numbers; and performs a target task using a fifth neural network based on the reconstructed output. Attached Figure Description

[0009] Figure 1 The diagram illustrates an environment in which the methods, apparatus, and systems described herein can be implemented according to embodiments.

[0010] Figure 2 yes Figure 1 A block diagram of example components of one or more devices.

[0011] Figure 3 This is a diagram illustrating the DQ mechanism that uses two quantizers in dependent quantization (DQ) design.

[0012] Figure 4(a) is a state diagram of a hand-designed state machine, illustrating the switching between two quantizers in a DQ design.

[0013] Figure 4(b) is a state table representing the state diagram of the manually designed state machine in Figure 4(a).

[0014] Figure 5 This is a block diagram of the general process of a Latent Representation Compression (LRC) system.

[0015] Figure 6 This is a block diagram of an end-to-end potential representation compression (E2ELRC) device during the testing phase according to an embodiment.

[0016] Figure 7 During the testing phase according to the embodiments Figure 6 A detailed block diagram of the Deep Reinforcement Learning (DRL) quantization module in the test phase device.

[0017] Figure 8 During the testing phase according to the embodiments Figure 6 Detailed block diagram of the DRL inverse quantization module in the test phase device.

[0018] Figure 9 This describes the workflow of the Deep Reinforcement Learning DRL quantization module and DRL dequantization module during the training phase, according to the embodiment.

[0019] Figure 10 This describes the detailed workflow of the MemoryReplay & Weight Update module during the training phase, according to an embodiment.

[0020] Figure 11 This is a flowchart of an end-to-end latent representation compression (E2ELRC) method using deep reinforcement learning (DRL) according to an embodiment.

[0021] Figure 12 This is a block diagram of an apparatus for end-to-end latent representation compression (E2ELRC) using deep reinforcement learning (DRL) according to an embodiment. Detailed Implementation

[0022] Examples may involve a framework for end-to-end latent representation compression (E2ELRC) using deep reinforcement learning (DRL). This method considers both task performance and compression efficiency, and jointly optimizes the system.

[0023] Encoding and transmitting the latent representation of the original input instead of the raw image / video can offer benefits such as reduced transmission costs and improved privacy. For example, a surveillance system designed to detect anomalous vehicles does not need to view the raw video stream, but only the latent features required for the detection task. VCM and DCM (Machine Data Encoding) standards have been developed to study latent feature encoding and decoding techniques to generate encoded latent features that can be efficiently stored and transmitted, and can effectively perform machine vision or human vision tasks.

[0024] Traditional image and video codec standards use dependent quantization (DQ) or trellis-coded quantization with hand-designed quantization rules. DQ consists of two quantizers, Q0 and Q1, and a procedure for switching between them. Figure 3 An example illustration of a DQ mechanism using quantizers Q0 and Q1 in a DQ design is given. The labels above the circles indicate the associated states, and the labels below the circles indicate the associated quantization keys. At the decoder, the reconstructed number x′ is determined by multiplying the integer key k by the quantization step size Δ of either quantizer Q0 or Q1. The switching between quantizers Q0 and Q1 can be represented by a state machine, where the state machine has M=2. K There are 2 DQ states, K ≥ 2 (therefore M ≥ 4), where each DQ state is associated with either quantizer Q0 or Q1. The current DQ state consists of the previous DQ state and the current quantization key k. i The value is uniquely determined. To encode the input streams x1, x2..., the potential transition between quantizers Q0 and Q1 can be determined by having 2... K This can be illustrated using a grid with DQ states. Therefore, choosing the optimal sequence of quantization bonds k1, k2, ... is equivalent to finding the grid path with the minimum rate-distortion (RD) cost. This problem can be solved using the Viterbi algorithm.

[0025] Traditionally, state machines are designed manually based on experience. Figure 4 shows an example of a manually designed state machine used in the VVC standard with four states. Specifically, Figure 4(a) is the state diagram of the manually designed state machine. Figure 4(b) is the state table representing the state diagram of the manually designed state machine.

[0026] Traditional DQ methods have three main limitations. First, they use only two quantizers. Increasing the number of quantizers could reduce the bit consumption for encoding the numbers. Second, manually designing the state machine is not optimal and is too expensive to accommodate a large number of DQ states. Increasing the number of quantizers would require increasing the number of DQ states, which could improve quantization efficiency, but would make the state machine too complex to be designed manually. Finally, the methods for key generation and number reconstruction are manually heuristically designed, which is also not optimal. Searching for better methods requires domain expertise and may be too expensive to design manually.

[0027] Accordingly, embodiments of this disclosure may relate to learning-based quantization learned by a DRL mechanism. Embodiments can flexibly support various types of quantization methods (e.g., uniform quantization, codebook-based quantization, or deep learning-based quantization) and learn the optimal quantization in a data-driven manner. Furthermore, embodiments may jointly relate to the entire E2ELRC process, wherein the DNN encoder, DNN decoder, learning-based quantization method, DNN latent generator, and DNN task executor can be jointly optimized to provide improved data adaptive compression results.

[0028] Figure 1 This is a schematic diagram of an environment 100 in which the methods, apparatus and systems described herein can be implemented, according to an embodiment.

[0029] like Figure 1 As shown, environment 100 may include user equipment 110, platform 120, and network 130. The devices in environment 100 can be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0030] User equipment 110 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information related to platform 120. For example, user equipment 110 may include computing devices (e.g., desktop computers, laptop computers, tablet computers, handheld computers, smart speakers, servers, etc.), mobile phones (e.g., smartphones, cordless phones, etc.), wearable devices (e.g., smart glasses or smartwatches), or similar devices. In some embodiments, user equipment 110 may receive information from and / or send information to platform 120.

[0031] Platform 120 includes one or more devices as described elsewhere herein. In some embodiments, platform 120 may include a cloud server or a group of cloud servers. In some embodiments, platform 120 may be designed to be modular, allowing software components to be swapped in or out. This allows platform 120 to be easily and / or quickly reconfigured for different uses.

[0032] In some implementations, as shown in the figures, platform 120 may be hosted in a cloud computing environment 122. It is worth noting that while the implementations described herein describe platform 120 as hosted in a cloud computing environment 122, in some implementations, platform 120 may not be cloud-based (i.e., may be implemented outside of a cloud computing environment) or may be partially cloud-based.

[0033] The cloud computing environment 122 includes the environment of the hosting platform 120. The cloud computing environment 122 can provide services such as computing, software, data access, and storage, without requiring end users (e.g., user equipment 110) to know the physical location and configuration of the systems and / or devices of the hosting platform 120. As shown in the figure, the cloud computing environment 122 may include a set of computing resources 124 (collectively referred to as "computing resources 124" and individually as "computing resource 124").

[0034] Computing resource 124 includes one or more personal computers, workstations, server devices, or other types of computing and / or communication devices. In some embodiments, computing resource 124 may host platform 120. Cloud resources may include computing instances executing in computing resource 124, storage devices provided in computing resource 124, data transmission devices provided by computing resource 124, etc. In some embodiments, computing resource 124 may communicate with other computing resources 124 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0035] Further as Figure 1 As shown, computing resources 124 include a set of cloud resources, such as one or more applications (“APP”) 124-1, one or more virtual machines (“VM”) 124-2, virtualized storage (“VS”) 124-3, one or more hypervisors (“HYP”) 124-4, etc.

[0036] Application 124-1 includes one or more software applications that can be provided to, or accessed by, user device 110 and / or platform 120. Application 124-1 does not require the installation and execution of any software applications on user device 110. For example, application 124-1 may include software associated with platform 120, and / or any other software available through cloud computing environment 122. In some implementations, an application 124-1 may send / receive information to or from one or more other applications 124-1 via virtual machine 124-2.

[0037] Virtual machine 124-2 includes a software implementation of a machine (e.g., a computer) that executes programs, similar to a physical machine. Virtual machine 124-2 can be a system virtual machine or a process virtual machine, depending on the extent to which virtual machine 124-2 uses and corresponds to any real machine. A system virtual machine can provide a complete system platform that supports the execution of a complete operating system (“OS”). A process virtual machine can execute a single program and can support a single process. In some implementations, virtual machine 124-2 can execute on behalf of a user (e.g., user device 110) and can manage the infrastructure of cloud computing environment 122, such as data management, synchronization, or long-term data transfer.

[0038] Virtualized storage 124-3 includes one or more storage systems and / or one or more devices that utilize virtualization technology within the storage systems or devices of computing resource 124. In some implementations, the type of virtualization within the context of the storage system may include block virtualization and file virtualization. Block virtualization may refer to the abstraction (or separation) of logical storage from physical storage so that the storage system can be accessed without regard to physical storage or heterogeneous architecture. Separation allows storage system administrators to flexibly manage end-user storage. File virtualization can eliminate the dependency between data accessed at the file level and the location of physical storage files. This can optimize storage usage, server consolidation, and / or performance for non-disruptive file migration.

[0039] Hypervisor 124-4 provides hardware virtualization technology that allows multiple operating systems (e.g., "guest operating systems") to run simultaneously on a host computer such as computing resource 124. Hypervisor 124-4 can provide a virtual operating platform to the guest operating systems and manage their execution. Multiple instances of various operating systems can share virtualized hardware resources.

[0040] Network 130 includes one or more wired and / or wireless networks. For example, network 130 may include cellular networks (e.g., fifth-generation (5G) networks, Long-Term Evolution (LTE) networks, third-generation (3G) networks, Code Division Multiple Access (CDMA) networks, etc.), Public Land Mobile Networks (PLMNs), Local Area Networks (LANs), Wide Area Networks (WANs), Metropolitan Area Networks (MANs), telephone networks (e.g., Public Switched Telephone Networks (PSTNs)), private networks, self-organizing networks, intranets, the Internet, fiber-optic networks, etc., and / or combinations of these or other types of networks.

[0041] Figure 1 The number and arrangement of devices and networks shown are provided as an example. In reality, with... Figure 1 Compared to the devices and / or networks shown, there can be more devices and / or networks, fewer devices and / or networks, different devices and / or networks, or devices and / or networks arranged differently. Furthermore, Figure 1 The two or more devices shown can be implemented within a single device, or Figure 1 The single device shown can be implemented as multiple distributed devices. Alternatively, a group of devices in environment 100 (e.g., one or more devices) can perform one or more functions described as being performed by another group of devices in environment 100.

[0042] Figure 2 yes Figure 1 A block diagram of example components for one or more devices. Device 200 may correspond to user device 110 and / or platform 120. Figure 2 As shown, device 200 may include bus 210, processor 220, memory 230, storage component 240, input component 250, output component 260 and communication interface 270.

[0043] Bus 210 includes components that allow communication between components of device 200. Processor 220 is implemented in hardware, firmware, or a combination of hardware and software. Processor 220 is a central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), microprocessor, microcontroller, digital signal processor (DSP), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), or another type of processing component. In some embodiments, processor 220 includes one or more processors that can be programmed to perform functions. Memory 230 includes random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic storage, and / or optical storage) that stores information and / or instructions for use by processor 220.

[0044] Storage component 240 stores information and / or software related to the operation and use of device 200. For example, storage component 240 may include hard disks (e.g., magnetic disks, optical disks, magneto-optical disks, and / or solid-state disks), optical disks (CDs), digital versatile disks (DVDs), floppy disks, cassette tapes, magnetic tapes, and / or other types of non-volatile computer-readable media, and corresponding drives.

[0045] Input component 250 includes components that allow device 200 to receive information, such as a touchscreen display, keyboard, keypad, mouse, buttons, switches, and / or microphone. Alternatively, input component 250 may include sensors for sensing information (e.g., a Global Positioning System (GPS) component, accelerometer, gyroscope, and / or actuator). Output component 260 includes components that provide output information from device 200, such as a display, speaker, and / or one or more light-emitting diodes (LEDs).

[0046] Communication interface 270 includes transceiver-like components (e.g., a transceiver and / or separate receiver and transmitter) that enable device 200 to communicate with other devices, for example, via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communication interface 270 may allow device 200 to receive information from and / or provide information to another device. For example, communication interface 270 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, etc.

[0047] Device 200 can perform one or more processes described herein. Device 200 can perform these processes in response to processor 220 executing software instructions stored in a non-volatile computer-readable medium (e.g., memory 230 and / or storage component 240). Computer-readable medium is defined herein as a non-volatile memory device. A memory device includes storage space within a single physical storage device or storage space distributed across multiple physical storage devices.

[0048] Software instructions can be read into memory 230 and / or storage component 240 from another computer-readable medium or from another device via communication interface 270. When executed, the software instructions stored in memory 230 and / or storage component 240 can cause processor 220 to perform one or more processes described herein. Alternatively or additionally, hardware wiring circuitry may be used in place of or in combination with the software instructions to perform one or more processes described herein. Therefore, the embodiments described herein are not limited to any particular combination of hardware circuitry and software.

[0049] Figure 2 The number and arrangement of components shown are provided as an example. In fact, with... Figure 2 Compared to the components shown, device 200 may include more components, fewer components, different components, or components arranged differently. Alternatively, a set of components of device 200 (e.g., one or more components) may perform one or more functions described as being performed by another set of components of device 200.

[0050] Now, reference will be made to the embodiments. Figure 5 A method and apparatus for a general process used in a latent representation compression (LRC) system are described in detail.

[0051] Figure 5 This is a block diagram of a device for a general process of a potential representation compression (LRC) system.

[0052] like Figure 5 As shown, the apparatus for a general process includes a DNN latent generation module 510, a DNN encoding module 520, a quantization module 530, an entropy encoding module 540, an entropy decoding module 550, an inverse quantization module 560, and a DNN decoding module 570.

[0053] Let X represent the input (image, video, audio, or other types of data). The DNN latent generation module 510 generates a latent representation F using a DNN latent generator. The latent representation F can be serialized into a sequence of encoded signals, F = f1, f2, ..., where the signal f tThis can typically be represented as a 4D tensor of size (h, w, c, d). For each signal f t The DNN encoding module 520 uses a DNN encoder based on the signal f t Compute the DNN encoded representation y t Then, the quantization module 530 uses a quantizer to quantize based on the encoded representation y. t Generate quantized representation Following this, the entropy coding module 540 uses an entropy encoder to quantize the representation. Encode into a compact representation that is easy to store and transmit. Then, at the decoder end, upon receiving the compact representation... Subsequently, the entropy decoding module 550 uses an entropy decoder based on a compact representation. To recover the decoded representation Lossless entropy encoding methods can be used by both entropy encoders and entropy decoders, and result in decoded representations. Equal to quantization (Right now, Then, the dequantization module 560 uses a dequantizer to perform dequantization based on the decoded representation. To calculate the inverse quantization representation y′ t The DNN decoding module 570 then uses the DNN decoder to determine the inverse quantization representation y′. t To generate the latent representation of the reconstruction Finally, the DNN task module 580 executes the latent representation based on the recovered data by using the DNN task executor. To carry out the target task.

[0054] The overall goal of the LRC system is to minimize joint losses. Minimization, which considers two aspects: minimizing the rate-distortion (RD) loss and minimizing the quantization representation. It will have very little bit consumption (due to rate loss). (reflection), and the potential representation of reconstruction Approximate to the original f t (due to distortion loss) (Reflection); and enabling task prediction loss. Minimize, so that the reconstructed latent representation It can perform the original objective task well. The joint loss can be calculated according to the following equation.

[0055]

[0056] Distortion loss Reconstruction errors, such as PSNR and / or SSIM metrics, were measured. Rate loss. With quantization The bit rate is related. The hyperparameters β and λ balance the importance of different loss terms.

[0057] Since quantization / dequantization operations are typically non-differentiable, the quantizer / dequantizer is optimized separately from the DNN encoder, DNN decoder, DNN latent generator, and DNN task executor. For example, previous methods assumed linear quantization and approximated differentiable rate loss through entropy estimation. Thus, backpropagation can be used to learn the DNN encoder, DNN decoder, DNN latent generator, and DNN task executor.

[0058] The proposed implementation presents the E2ELRC method, in which a DNN encoder, DNN decoder, DNN latent generator, and DNN task executor, along with quantizers and dequantizers, are jointly learned. Specifically, deep reinforcement learning (DRL) is used to combine the optimizations of the DNN encoder, DNN decoder, DNN latent generator, DNN task executor, and quantizers and dequantizers. The proposed E2ELRC framework is general and broad, adaptable to different types of quantization methods and different types of DNN encoder, DNN decoder, DNN latent generator, and DNN task executor network architectures.

[0059] The methods and apparatus for an end-to-end latent representation compression (E2ELRC) system using deep reinforcement learning (DRL) will now be described in detail.

[0060] Figure 6 This is a block diagram of the E2ELRC device during the testing phase according to an embodiment.

[0061] like Figure 6 As shown, the E2ELRC test apparatus includes a DNN latent generation module 610, a DNN encoding module 620, a DRL quantization module 630, an entropy encoding module 640, an entropy decoding module 650, a DRL inverse quantization module 660, a DNN decoding module 670, and a DNN task execution module 680.

[0062] As part of the encoding process, given an input signal X, the DNN latent generation module 610 generates a latent representation F using a DNN latent generator. The latent representation F is then serialized into a sequence of encoded signals, F = f1, f2, ..., where each signal f... t It is a 4D tensor of size (h, w, c, d). The DNN encoding module 620 uses a DNN encoder based on this signal f. t To compute the DNN-encoded representation yt The DNN has encoded the representation of y. t It can be viewed as a digital stream, y t =y t，1 y t，2 … For a batch of m numbers Y t，i =...,y t，i-1 y t，i The DRL quantization module 630 calculates a batch of quantization keys (QK) using the DRL quantizer. t，i =…,k t，i-1 k t，i Each QK k t，l Corresponding to the encoded representation y t，l Each number in the batch is processed individually. For a batch of size 1 (m = 1), the numbers are processed one by one. When m > 1, the numbers are quantized in an organized manner. The numbers can also be organized in different orders. For example, the numbers can be organized in blocks to preserve relative position information. Then, the system will process QKK. t，i Send it to the decoding process and continue processing the next batch of numbers Y t，i+1 Optionally, the entropy coding module 640 further compresses (preferably in a lossless manner) QKK. t，i To facilitate storage and transmission.

[0063] As part of the decoding process, upon receiving QK K t，i Subsequently, if the received QKs are entropy-encoded, then the entropy decoding module 650 is applied to obtain the entropy-decoded QKs. Then, the DRL dequantization module 660 recovers a batch of dequantized numbers Y′ using the DRL dequantizer. t，i =…,y′ t，i-1 y′ t，i The dequantized number is the dequantized representation y′ t A batch from the entire stream. Then, the DNN decoding module 670, using the DNN decoder, based on the inverse quantization representation y′ t Generate reconstructed output Finally, the DNN task module 680 executes the DNN task executor based on the recovered reconstruction output. To perform the target task. Note that the entropy encoding module 640 and the entropy decoding module 650 are optional, and... Figure 6 The text is marked with a dashed line. In the example embodiment, when using the entropy encoding module 640 and the entropy decoding module 650, this embodiment employs a lossless entropy encoding method, and therefore, the QK obtained by entropy decoding is the same as the QK calculated by the DRL quantization module 630 (i.e., ...). Therefore, the QK calculated in the subsequent encoding and decoding processes will use the same symbol (K).t，i ).

[0064] Figure 6 The DRL quantizer and DRL dequantizer in the code use a learning-based quantization method. Figure 7 and Figure 8 The detailed workflows of the DRL quantization module 630 and the DRL dequantization module 660 are described respectively.

[0065] like Figure 7 As shown, the DRL quantization module 630 includes a key calculation module 710 and a state prediction module 720.

[0066] As part of the encoding process, given a set of m numbers Y t，i =...,y t，i-1 y t，i Based on a batch of previous quantization states (QSs) S t，i-1 =...,s t，i-2 s t，i-1 Each QS s t，l-1 Corresponding to the encoded representation y t，l For each of these, the key calculation module 710 calculates QK using a key generator. t，i =…,k t，i-1 k t，i Each QKk t，l Corresponding to the encoded representation y t，l Each of them. Then, the state prediction module 720 calculates the current QS using the state predictor. t，i =…,s t，i-1 s t，i .

[0067] Given the previous QS S t，i-1 The bond generator uses a quantization method to compute QK. This quantization method can be a predefined rule-based method, such as uniform quantization with a fixed step size, where QK = k. t，i It is an integer that can optimally reconstruct the corresponding encoded representation y. t，i As QK k t，i The product of Q and the quantization step size. This quantization method can also be a statistical model similar to k-means, where Q = Kk. t，i It is an index of a cluster whose centroid can optimally reconstruct the encoded representation y. t，i This disclosure does not impose any restrictions on the specific quantization methods used as key generators.

[0068] Given the previous QS S t，i-1 and the current QK K t，iThe state prediction module 720 calculates the current QS s t，i In the example embodiment, the state prediction module 720 uses only the latest QSs. t，i-1 The state prediction module 720 is connected to each of the m QKs to form a pair, and all m pairs are stacked together to form an input matrix of size (m, 2). In another example embodiment, each QK and its corresponding QS form a pair (k... t，l s t，l-1 The m pairs are stacked together to form an input matrix of size (m, 2). The state prediction module 720 calculates the current QS based on the state predictor. t，i The state predictor uses a learning-based model to support transitions between any number of possible states that the QS can take. In this embodiment, the learning-based model is trained using a Deep Q-Learning (DQN) algorithm, which will be described in detail later.

[0069] like Figure 8 As shown, the DRL inverse quantization module 660 includes a state prediction module 720 and a reconstruction module 810.

[0070] As part of the decoding process, upon receiving QK K t，i =…,k t，i-1 k t，i Subsequently, the state prediction module 720, based on the input QK K t，i And the previous QS S t，i-1 =…,s t，i-2 s t，i-1 The current QSs is calculated by using the encoding process. t，i The same method is used to calculate the current QS s using a state predictor. t，i Then, the reconstruction module 810 uses a QKK-based reconstructor. t，i and QS S t，i-1 To calculate the number Y′ of this batch of dequantized data. t，i =…,y′ t，i-1 y′ t，i The reconstructor uses a dequantization method corresponding to the quantization method used in the key generator. For example, when the quantization method is a predetermined rule-based method, such as uniform quantization with a fixed step size, the dequantization method is also predetermined rule-based, such as dequantizing the dequantized number y′. t，i Calculated as QK k t，i The product of Q and the quantization step size. When the quantization method is a statistical model such as k-means, the inverse quantization method can be QK k. t，i The centroid of the index. This disclosure does not impose any restrictions on the specific dequantization methods used as reconstructors.

[0071] Given QK K t，i and QS S t，i-1 The state predictor is action a. j and the output Q value v associated with this action j The action-value mapping function f(a) between (j = 1, ..., J) j v j |K t，i S t，i-1 (Assume we have a total of J possible actions). Each action a j Corresponding to QS s t，i Possible states that can be taken. Given the current QK K t，i and QS S t，i-1 The state predictor calculates all possible actions a j Q value v j And select the one with the best Q value. Best move With best action The corresponding state is the QS s selected by the system. i The Q-value is designed to measure the target compression performance associated with a sequence of actions. Therefore, selecting the optimal action yields the best target compression performance.

[0072] Deep Q-learning mechanisms, particularly the DQN algorithm, are used as the training method in this embodiment. DQN is an off-policy DRL method that learns an action-value mapping function to assign reward Q-values ​​to actions, thereby finding the optimal action selection policy for any given finite Markov decision process. A policy is a rule that the system follows when selecting actions. Given a current state, the learning agent can choose from a set of candidate actions, resulting in different reward values. By experiencing various states and trying various actions in those states, the learning agent learns over time to optimize rewards so that it can perform optimally in any future given state.

[0073] Specifically, the DNN is used as a state predictor, acting as a function approximator to estimate the action-value mapping function f(a). j v j |K t，i S t，i-1 State predictor DNNs typically consist of a set of convolutional layers, followed by one or more fully connected layers. This disclosure does not impose any limitations on the specific network architecture of the state predictor.

[0074] The training process of the DRL quantization module 630 and the DRL dequantization module 660 according to the embodiments will now be described. Figure 9The overall workflow of the training process is shown.

[0075] like Figure 9 As shown, the E2ELRC system training device includes a DNN latent generation module 610, a DNN encoding module 620, a DNN decoding module 670, a task execution module 680, a key calculation module 710, a state prediction module 720, a reconstruction module 810, a distortion calculation module 910, a rate calculation module 920, a reward calculation module 930, a memory replay and weight update module 940, an LRC distortion calculation module 950, an LRC rate calculation module 960, and an LRC weight update module 970.

[0076] Let State(t) s -1) is the current state predictor, let Key(t) k -1) represents the current key generator, let Recon(t) r -1) is the current reconstructor, let Enc(t) e -1) is the current DNN encoder, let Dec(t) d -1) is the current DNN decoder, let Latent(t) l -1) represents the current DNN potential generator, and let Task(t) be the current DNN potential generator. t -1) represents the current DNN task executor. s t k t r t e t l and t t They can be different, allowing the state predictor, key generator, reconstructor, DNN encoder, DNN decoder, DNN latent generator, and DNN task executor to update at different times and at different update frequencies.

[0077] Given training input X, the DNN latent generator module 610 uses the current DNN latent generator Latent(t) l -1) Calculate the potential signal sequence F = f1, f2, ... For each signal f t The DNN encoding module 620 uses the current DNN encoder Enc(t) e -1) to compute the DNN-encoded representation y t =y t，1 y t，2 … For this batch of m numbers Y t，i =...,y t，i-1 y t，i According to previous QSS t，i-1 =…,s t，i-2 s t，i-1 The key calculation module 710 uses the current key generator Key(t)k -1) to calculate QK K t，i =…,k t，i-1 k t，i The batch size and number organization are the same as in the testing phase. Then, the state prediction module 720 is based on the previous QS S. t，i-1 and the current QK K t，i Using the current state predictor State(t) s -1) to calculate the current QS s t，i The input to the state prediction module 720 is also the same as that in the testing phase. Then, the reconstruction module 810 uses the current reconstructor Recon(t) r -1), based on QKK t，i and QS S t，i-1 To calculate the number Y′ of this batch of dequantized data. t，i =…,y′ t，i-1 y′ t，i Finally, the DNN decoding module 670 uses the current DNN decoder Dec(t) d -1) Based on inverse quantization, the digital y′ t Generate reconstructed z t .

[0078] During training, the state predictor uses an ∈-greedy method to select the best action. Specifically, in the current state predictor State(t) s -1) Calculate all possible actions a using probabilities ∈ (numbers between 0 and 1). j Q value v j Then, a random action will be selected as the best action. And using the probability (1-∈), the option with the best Q value will be selected. Best move

[0079] The distortion calculation module 910 calculates the distortion loss D(Y). t，i ,Y′ t，i ), to measure the original DNN already encoded representation of Y t，i With the decoded representation Y′ t，i The difference between them. For example, distortion loss D(Y) t，i ,Y′ t，i ) can be an encoded representation of Y t，i With the decoded representation Y′ t，i The average of the Lk norms of the differences between corresponding elements in the matrix (e.g., the L1 norm as mean absolute error and the L2 norm as mean squared error):

[0080]

[0081] Meanwhile, the computation rate module 920 suffers a computation rate loss R(K). t，i (), to measure the bit consumption of the quantized representation, i.e., the calculated QK sent from the encoder to the decoder. t，i There are several methods to calculate the rate loss. For example, any lossless entropy coding method can be used to compress QK, and the actual bit count of the compressed bitstream can be obtained as the rate loss.

[0082] For adjacent batches Y t，i and Y t，i+1 Based on distortion D(Y) t，i ,Y′ t，i ) and D(Y t，i+1 ,Y′ t，i+1 ) and rate loss R(K t，i ) and R(K t，i+1 The reward calculation module 930 calculates the reward φ(Y). t，i+1 K t，i+1 ,Y′ t，i+1 Given the current QKK t，i and QS S t，i-1 In this case, the reward φ(Y) t，i+1 K t，i+1 ,Y′ t，i+1 By taking the best action according to the following equation To measure the reward that the state predictor can obtain:

[0083] φ(Y t，i+1 K t，i+1 ,Y′ t，i+1 )=D(Y t，i+1 ,Y′ t，i+1 )+αR(K t，i+1 (3)

[0084] Here, α is a hyperparameter that balances rate loss and distortion in the reward calculation. (Experience...) Add to the replay memory. Among them, experience... Based on QK K t，i and QS S t，i-1 Choose the one with the associated Q value. action Then receive the reward φ(Y) t，i+1 K t，i+1 ,Y′ t，i+1 Replay memory typically has a maximum storage limit, and once that limit is reached, the oldest experience is replaced by the most recent one.

[0085] When it's time to update the state predictor, key generator, and reconstructor, the system samples a batch of experience from the replay memory and uses this sampled experience to update the model parameters in the memory replay and weight update module 940. Figure 10 This describes the detailed workflow of the memory replay and weight update module 940 during the training phase.

[0086] like Figure 10 As shown, the memory replay and weight update module 940 includes a key calculation module 710, a state prediction module 720, a reconstruction module 810, a distortion calculation module 910, a rate calculation module 920, a reward calculation module 930, a sampling experience module 1001, a loss calculation module 1002, and a weight update module 1003.

[0087] During the training phase, the target state predictor State T Key generator T and target reconstructor Recon T They are preserved and have the exact same model structure as the state predictor, key generator, and reconstructor, respectively. The only difference is the model parameters, such as the DNN weight coefficients of the state predictor, or the k-means model parameters of the key generator when using k-means quantization, or the DNN weight coefficients of the key generator when using deep clustering-based quantization. In each T... s T k and T r The parameter update cycle clones these model parameters from the corresponding state predictor, key generator, and reconstructor.

[0088] In each parameter update cycle, the sampling experience module 1001 samples data from the replay memory. A set of experiences is sampled. For each experience... The state prediction module 720 uses the target state predictor State. T Based on experience with QK Y t，l and QS S t，l-1 Predicting targets Based on the goal Target Key Generator T The target key is calculated in the key calculation module 710. Based on target key and target Target Recon T A batch of target dequantized numbers can be calculated in reconstruction module 810. Then, the distortion calculation module 910 calculates the original representation Y in the experience. t，l+1 With decoded representation Distortion of targets between The computation rate module 920 is based on the target key. Calculate target rate loss Then, in the reward calculation module 930, the target reward is... The calculation is as follows:

[0089]

[0090] Then, the loss calculation module 1002 will calculate the target reward. The calculation is as follows:

[0091]

[0092] in, It is the target state predictor State T For a given and action The predicted Q-value. The hyperparameter γ is the discount rate, which takes values ​​between 0 and 1, and determines the importance of the system's trade-off between long-term and short-term rewards. The smaller the discount rate, the less weight the system places on long-term rewards, and the more it focuses on short-term rewards. Then, based on the target reward... and Q value derived from experience (e.g., the Lk norm of the difference between two rewards) to calculate the target loss.

[0093]

[0094] Then, the weight update module 1003 calculates the gradient of the target loss, which is backpropagated to update the weight parameters of the DNN of the state predictor to State(t). s The gradient of the objective loss can also be combined with the optimization objective of the learning-based key generator and reconstructor to update the key generator Key(t). k ) and Recon(t r For example, when the key generator and reconstructor use a deep clustering-based quantization method, the weight parameters of the DNNs of the key generator and reconstructor are updated via backpropagation. When other learning-based methods are used for quantization, the model parameters are learned by optimizing the objective function, and the objective loss is... These can be weighted and added as additional regularization terms to the optimization objective function to update the model parameters. As mentioned earlier, the state predictor, key generator, and reconstructor can be updated at different timestamps.

[0095] For each T s T k and T rIn the iteration, the weight parameters of the state predictor, key generator, and reconstructor will be cloned into the target state predictor, State. T Key generator T and target reconstructor Recon T .

[0096] The implementation uses a replay memory, a target state predictor, a target key generator, and a target reconstructor to stabilize the training process. The replay memory can contain only one latest experience, which is equivalent to having no replay memory at all. Furthermore, T... s T k and T r All of them can be equal to 1, so that the target state predictor, target key generator, and target reconstructor will be updated for each iteration, which is equivalent to not having another set of target state predictors, target key generators, and target reconstructors.

[0097] For the entire E2ELRC system for each input X (in Figure 9 (As described in the text), the DNN latent generation module 610 uses the current DNN latent generator Latent(t) l -1) to calculate the potential signal sequence F = f1, f2, ... For each signal f t The DNN encoding module 620 uses the current DNN encoder Enc(t) e -1) to compute the DNN-encoded representation y t =y t，1 y t，2 ... The dequantized representation y′ was generated using the DRL quantization module 630 and the DRL dequantization module 660. t =y′ t，1 y′ t，2 Then, the DNN decoding module 670 uses the current DNN decoder Dec(t) d -1), based on the inverse quantization representation y′ t To generate the latent representation of the reconstruction Finally, the DNN task module 680 executes by using the current DNN task executor Task(t) t -1), based on the reconstructed latent representation To perform the target task, and to calculate the task prediction loss based on the training labels. (For example, classification or regression loss of the original task).

[0098] Then, the LRC distortion module 950 calculates the latent representation distortion loss. To measure errors introduced by the latent representation compression process, such as PSNR and / or SSIM-related metrics. The LRC rate module 960 calculates the LRC rate, for example, based on a quantization representation with uniform or normal density. (That is, QKk stored and transmitted to the decoding process) t，1 k t，2 Nonparametric density estimation of (, ...) is used to calculate the potential compressibility loss. Then, the overall joint loss. It can be calculated as follows:

[0099]

[0100] The hyperparameters β and λ balance the importance of different loss terms.

[0101] Then, the LRC weight update module 970 calculates the gradient of the joint loss (e.g., by summing the gradients of the joint loss over several input data), which can update the weight parameters of the DNN encoder, DNN decoder, DNN latent generator, and DNN task executor to Enc(t) via backpropagation. e Dec(t) d ), Latent(t l ) and Task(t t ).

[0102] In this embodiment, the DNN latent generator and the DNN task executor are pre-trained (represented by latent(0) and task(0) respectively) by omitting the encoding / decoding process. During this pre-training process, given a pre-training input X, the DNN latent generator module 610 computes a latent representation F, which is directly used by the DNN task execution module 680. The task prediction loss T can then be calculated. LRC (f t Task prediction loss T LRC (f t The gradients of the DNN are backpropagated to learn the DNN latent generator and the DNN task executor.

[0103] Furthermore, in the embodiments, the potential compression ratio loss is estimated by assuming a uniform quantization method and by using an entropy estimation model. The DNN encoder and DNN decoder are pre-trained (denoted by Enc(0) and Dec(0) respectively). During this pre-training process, given the pre-training latent signal f... t The DNN encoder computes the representation y t The entropy estimation model further uses this representation y t To calculate potential compression ratio loss Then, the DNN decoder is based on this representation y t Computational output (reconstructed latent representation) Then, the potential distortion loss can be calculated. Furthermore, the obtained RD loss is:

[0104]

[0105] Its gradient can be used to update the DNN encoder and DNN decoder through backpropagation.

[0106] When deploying pre-trained DNN encoders, DNN decoders, DNN latent generators, and DNN task executors, Figure 9 and Figure 10 The training process described in the embodiments trains the DRL quantizer and DRL dequantizer to work in conjunction with the DNN encoder, DNN decoder, DNN latent generator, and DNN task executor to improve quantization performance. The described training process can also update the DNN encoder, DNN decoder, DNN latent generator, and DNN task executor based on the current training data, allowing the entire latent compression system to adaptively improve overall compression performance and task performance. Updates to the DNN encoder, DNN decoder, DNN latent generator, and DNN task executor can occur offline or online and can be permanent or temporary and data-dependent.

[0107] Similarly, after deployment, the state predictor, key generator, and reconstructor in the DRL quantizer and DRL dequantizer can be updated offline or online, and can be permanent or temporary data-dependent. For example, in the case of video-based tasks, some or all of the DNN encoder, DNN decoder, DNN latent generator, DNN task executor state predictor, key generator, and reconstructor can be updated based on the previous few frames. However, these updates are not recorded to avoid affecting the computation of future videos. Such updates can also accumulate to a certain number, based on which modules can be permanently updated for application to future videos. Regarding parameter updates, a portion of the DNN model parameters can be frozen, and only the remaining parameters are updated. This disclosure makes no restrictions on which DNN models or which part of the weight parameters in the DNN model are updated.

[0108] Figure 11 This is a flowchart of an end-to-end latent representation compression method using deep reinforcement learning according to an embodiment.

[0109] In some implementations, Figure 11 One or more process blocks can be executed by platform 120. In some implementations, Figure 11One or more process blocks can be executed by another device or group of devices (e.g., user equipment 110) that is separate from or includes platform 120.

[0110] like Figure 11 As shown, in operation 1101, the method includes generating multiple latent representations of the input using a first neural network. The multiple latent representations may be a sequence of latent signals.

[0111] In operation 1102, the method includes encoding multiple latent representations using a second neural network.

[0112] In operation 1103, the method includes using a third neural network to generate a set of quantization keys based on a set of previous quantization states, wherein each quantization key in the set of quantization keys and each previous quantization state in the set of previous quantization states correspond to multiple latent representations. A set of encoded quantization keys can also be generated by entropy encoding the set of quantization keys.

[0113] A third neural network is trained to generate a set of current quantized states based on a set of previous quantized states and a set of quantized keys. The third neural network is trained by: calculating the q-values ​​of all possible actions; randomly selecting an action as the best action with the optimal q-value; generating a reward for the selected best action; sampling a set of selected best actions; and updating the weight parameters of the third neural network to minimize distortion loss.

[0114] In operation 1104, the method includes using a fourth neural network to generate a set of dequantized numbers based on the set of quantization keys, the set of dequantized numbers representing the dequantized representations of multiple encoded latent representations. If the set of encoded quantization keys is generated, a set of decoded quantization keys can also be generated by entropy decoding of the set of encoded quantization keys, and the set of dequantized numbers can be generated based on the set of decoded quantization keys instead.

[0115] Using block-by-block quantization / dequantization, individual quantization / dequantization, or static quantization / dequantization model methods, the set of quantized keys generated in operation 1103 and the set of dequantized numbers generated in operation 1104 are quantized and dequantized, respectively. Furthermore, the quantization method for the set of quantized keys and the dequantization method for the set of dequantized numbers are the same.

[0116] In operation 1105, the method includes generating a reconstructed output based on the set of dequantized numbers.

[0117] In operation 1106, the method includes using a fifth neural network to perform a target task based on the reconstructed output.

[0118] Conversely, the target task can be performed based on multiple generated latent representations. Furthermore, based on the target task, a task prediction loss can be computed, where the gradient of the task prediction loss is backpropagated, and the weight parameters of the first and fifth neural networks are updated, thus training the first and fifth neural networks.

[0119] Although Figure 11 An example box for this method is shown, but in some implementations, the method may include... Figure 11 The boxes depicted are compared to additional boxes, fewer boxes, different boxes, or boxes arranged in a different way. Alternatively, the method can be performed on two or more boxes in parallel.

[0120] Figure 12 This is a block diagram of an apparatus for end-to-end latent representation compression using deep reinforcement learning according to an embodiment.

[0121] like Figure 12 As shown, the device includes a first generation code 1201, an encoding code 1202, a second generation code 1203, a third generation code 1204, a decoding code 1205, and an execution code 1206.

[0122] The first generation code 1201 is configured to cause at least one processor to use a first neural network to generate multiple latent representations of the input, wherein the multiple latent representations include a sequence of latent signals.

[0123] Encoding code 1202 is configured to enable at least one processor to use a second neural network to encode multiple potential representations.

[0124] The second generation code 1203 is configured to cause at least one processor to use a third neural network to generate a set of quantization keys based on a set of previous quantization states, wherein each quantization key in the set of quantization keys and each previous quantization state in the set of previous quantization states correspond to multiple potential representations.

[0125] Furthermore, the operation of the device may also include state generation code configured to cause at least one processor to generate a set of current quantized states based on the set of previous quantized states and the set of quantized keys by training a third neural network. The third neural network is trained by: calculating the q-values ​​of all possible actions, randomly selecting an action as the best action with the best q-value, generating a reward for the selected best action, sampling the set of selected best actions, and updating the weight parameters of the third neural network to minimize distortion loss.

[0126] The third generation code 1204 is configured to enable at least one processor to use a fourth neural network to generate a set of dequantized numbers based on the set of quantization keys, the set of dequantized numbers representing the dequantized representations of multiple encoded latent representations.

[0127] The block-by-block quantization / dequantization method, the individual quantization / dequantization method, or the static quantization / dequantization model method can be used to quantize and dequantize the set of quantized keys generated by the second generation code 1203 and the set of dequantized numbers generated by the third generation code 1204, respectively. Furthermore, the quantization method for the set of quantized keys and the dequantization method for the set of dequantized numbers are the same.

[0128] Decoding code 1205 is configured to cause at least one processor to decode the reconstructed output based on the set of dequantized numbers.

[0129] Execution code 1206 is configured to cause at least one processor to use a fifth neural network to perform the target task based on the reconstructed output.

[0130] Conversely, the target task can be performed based on multiple generated potential representations. Figure 12 The apparatus may also include computational code configured to cause at least one processor to compute a task prediction loss based on the target task, wherein the gradient of the task prediction loss is backpropagated, and the weight parameters of the first neural network and the fifth neural network are updated, and the first neural network and the fifth neural network are trained.

[0131] Although Figure 12 An example box of the device is shown, but in some implementations, the device may include [other components]. Figure 12 The boxes depicted are compared to additional boxes, fewer boxes, different boxes, or boxes arranged in a different manner. Additionally or alternatively, two or more boxes of the device may be combined.

[0132] The embodiments relate to end-to-end latent representation compression (E2ELRC), which improves compression performance by optimizing latent representation compression to perform the target task as a whole system. This method provides the flexibility to adjust learning-based quantization and encoding methods online or offline based on current data, and supports different types of learning-based quantization methods, including DNN-based methods or methods based on conventional models. The described method also provides a flexible and general framework to adapt to different DNN architectures and tasks.

[0133] The proposed methods can be used individually or in any combination in any order. Furthermore, the methods (or embodiments) can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-volatile computer-readable medium.

[0134] This disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit implementations to the precise forms disclosed. Modifications and variations are possible with respect to this disclosure, or may be derived from the practice of implementation.

[0135] As used herein, the term component is intended to be interpreted broadly as hardware, firmware, or a combination of hardware and software.

[0136] Clearly, the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not limited to these implementations. Therefore, this document describes the operation and behavior of the systems and / or methods without referring to any specific software code—it should be understood that software and hardware can be designed to implement the systems and / or methods based on the descriptions herein.

[0137] Although combinations of features are referenced in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways not specifically referenced in the claims and / or disclosed in the specification. While each dependent claim listed below may be directly subordinated to only one claim, the disclosure of possible implementations includes every dependent claim combined with all other claims in the claim set.

[0138] The elements, actions, or instructions used herein should not be construed as critical or necessary unless explicitly stated otherwise. Furthermore, as used herein, the articles “a” and “an” are intended to include one or more items and are interchangeable with “one or more.” Additionally, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, etc.) and is interchangeable with “one or more.” The term “a” or similar language is used when intended to refer to only one item. Furthermore, as used herein, the terms “has,” “have,” “having,” etc., are intended to be open-ended terms. Additionally, the phrase “based on” is intended to mean “at least partially based on” unless explicitly stated otherwise.

Claims

1. An end-to-end task-oriented latent image compression method using deep reinforcement learning, characterized in that, The method includes: A first neural network is used to generate multiple latent representations of an input image, wherein the multiple latent representations include a sequence of latent signals; The multiple latent representations are encoded using a second neural network; A third neural network is used to generate a set of quantization keys based on a set of previous quantization states, wherein each quantization key in the set of quantization keys and each previous quantization state in the set of previous quantization states correspond to the plurality of latent representations; By performing lossless entropy encoding on the set of quantization keys, a set of encoded quantization keys is generated for storage and transmission. A set of decoded quantized keys is generated by performing lossless entropy decoding on the set of encoded quantized keys. A fourth neural network is used to generate a set of dequantized numbers based on the set of decoded quantization keys, the set of dequantized numbers representing the dequantized representations of multiple encoded latent representations; The latent representation reconstructed based on the set of inversely quantized numbers; and, The target task is performed using a fifth neural network based on the reconstructed latent representation; The task prediction loss is calculated based on the target task, wherein the gradient of the task prediction loss is backpropagated and the weight parameters of the first neural network and the fifth neural network are updated to train the first neural network and the fifth neural network.

2. The method according to claim 1, characterized in that, Further includes: The target task is performed based on the generated multiple potential representations.

3. The method according to claim 1, characterized in that, Further includes: The set of quantization bonds is generated using at least one of the block-by-block quantization method, the individual quantization method, and the static quantization model method. as well as, The set of dequantized numbers is generated using at least one of the block-by-block dequantization method, the individual dequantization method, and the static dequantization model method.

4. The method according to claim 3, characterized in that, The quantization method of the set of quantization keys is the same as the dequantization method of the set of dequantized numbers; Wherein, based on the set of quantization keys, the block-by-block quantization method is used as the quantization method, and the set of dequantized numbers is dequantized using the block-by-block dequantization method as the dequantization method; Wherein, based on the set of quantization keys, the individual quantization method is used as the quantization method, and the set of dequantized numbers uses the individual dequantization method as the dequantization method; Specifically, based on the set of quantization keys, the static quantization model method is used as the quantization method, and the set of dequantized numbers uses the static dequantization model method as the dequantization method.

5. The method according to any one of claims 1-4, characterized in that, Further includes: By training the third neural network, a set of current quantization states is generated based on the set of previous quantization states and the set of quantization keys. The third neural network is trained by: calculating the q-value of all possible actions, randomly selecting an action as the best action with the best q-value, generating a reward for the selected best action, sampling a set of selected best actions, and updating the weight parameters of the third neural network to minimize distortion loss.

6. An apparatus for end-to-end task-oriented latent image compression using deep reinforcement learning, characterized in that, The device includes: A first generation module is configured to generate multiple latent representations of an input image using a first neural network, wherein the multiple latent representations include a sequence of latent signals; An encoding module is used to encode the plurality of latent representations using a second neural network; The second generation module is used to generate a set of quantization keys based on a set of previous quantization states using a third neural network, wherein each quantization key in the set of quantization keys and each previous quantization state in the set of previous quantization states correspond to the plurality of potential representations; The encoding key module is used to generate a set of encoded quantization keys for storage and transmission by performing lossless entropy encoding on the set of quantization keys. The decoding key module is used to generate a set of decoded quantized keys by performing lossless entropy decoding on the set of encoded quantized keys. The third generation module is used to generate a set of dequantized numbers based on the set of decoded quantization keys using the fourth neural network. The set of dequantized numbers represents the dequantized representation of multiple encoded latent representations. A decoding module is configured to generate a reconstructed latent representation based on the set of inversely quantized numbers; and, An execution module is used to perform a target task based on the reconstructed latent representation using a fifth neural network; The calculation module is used to calculate the task prediction loss based on the target task, wherein the gradient of the task prediction loss is backpropagated and the weight parameters of the first neural network and the fifth neural network are updated to train the first neural network and the fifth neural network.

7. The apparatus according to claim 6, characterized in that, The execution module is further configured to: perform the target task based on the generated plurality of potential representations.

8. The apparatus according to claim 6, characterized in that, The device further includes: The fourth generation module is configured to generate the set of quantization bonds using at least one of the block-by-block quantization method, individual quantization method, and static quantization model method; and, The fifth generation module is used to generate the set of dequantized numbers using at least one of the block-by-block dequantization method, the individual dequantization method, and the static dequantization model method.

9. The apparatus according to claim 8, characterized in that, The quantization method of the set of quantization keys is the same as the dequantization method of the set of dequantized numbers; Wherein, based on the set of quantization keys, the fourth generation module is used to use the block-by-block quantization method as the quantization method, and the set of dequantized numbers uses the block-by-block dequantization method as the dequantization method; Specifically, based on the set of quantization keys, the fourth generation module is used to use the individual quantization method as the quantization method, and the set of dequantized numbers uses the individual dequantization method as the dequantization method; and... Specifically, based on the set of quantization keys, the fourth generation module is used to use the static quantization model method as the quantization method, and the set of dequantized numbers uses the static dequantization model method as the dequantization method.

10. The apparatus according to claim 6, characterized in that, The system further includes a state generation module for generating a set of current quantized states based on the set of previous quantized states and the set of quantized keys by training the third neural network. The third neural network is trained by: calculating the q-value of all possible actions, randomly selecting an action as the best action with the best q-value, generating a reward for the selected best action, sampling a set of selected best actions, and updating the weight parameters of the third neural network to minimize distortion loss.

11. An apparatus for end-to-end task-oriented latent image compression using deep reinforcement learning, characterized in that, include: One or more computer-readable non-volatile storage media for storing computer program code; as well as, One or more computer processors are configured to access the computer program code and operate in accordance with the instructions of the computer program code to perform the method of any one of claims 1 to 5.

12. A non-volatile computer-readable medium storing instructions, wherein, When at least one processor executes the instructions to perform end-to-end task-oriented latent image compression using deep reinforcement learning, the at least one processor performs the method according to any one of claims 1 to 5.

Images