Method and apparatus for adaptive image compression, computer readable medium

Abstract

A method and device for adaptive neural image compression, and a computer device, the method comprising: generating statistical features based on an input image and hyperparameters; generating first shared features and estimated adaptive coding parameters; encoding the input image based on the generated first shared features and the generated estimated adaptive coding parameters to obtain a signal coded image; generating second shared features and estimated adaptive hyper-coding parameters; generating hyper-features based on the signal coded image, the generated second shared features and the generated estimated adaptive hyper-coding parameters; and compressing the obtained signal coded image, the generated statistical features and the generated hyper-features.

Description

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to U.S. Provisional Patent Application No. 63 / 161,216, filed March 15, 2021, and U.S. Patent Application No. 17 / 362,039, filed June 29, 2021, the disclosures of which are incorporated herein by reference in their entirety. Background Technology

[0003] Standards bodies and companies have been actively seeking out potential needs for standardization in future video encoding and decoding technologies. These bodies and companies are focusing on end-to-end neural image compression (NIC) based on artificial intelligence (AI) using deep neural networks (DNNs). The success of this approach has generated increasing industrial interest in advanced neural image and video compression methods.

[0004] Typically, pre-trained NIC model instances are computed using a set of training data, assuming the training data covers the entire data distribution of all natural images, and that a generic model instance with pre-trained, fixed model parameters can be obtained to handle all natural images. This assumption does not hold true in practice. Real-world natural images have diverse data distributions, and pre-trained NIC models only function correctly on subsets of images. There is a strong need for NIC models to adaptively select their model parameters to suit different input images. Summary of the Invention

[0005] According to an embodiment, a method for adaptive neural image compression using a super-prior model through meta-learning includes: generating statistical features based on an input image and hyperparameters; generating a first shared feature and estimated adaptive coding parameters; encoding the input image based on the generated first shared feature and the generated estimated adaptive coding parameters to obtain a signal-coded image; generating a second shared feature and estimated adaptive super-coding parameters; generating super-features based on the obtained signal-coded image, the generated second shared feature, and the generated estimated adaptive super-coding parameters; and compressing the obtained signal-coded image, the generated statistical features, and the generated super-features. The method further includes: decoding the compressed signal-coded image to obtain a recovered image; decoding the compressed statistical features to obtain recovered statistical features; decoding the compressed super-features to obtain recovered super-features; generating a third shared feature and estimated adaptive super-decoding parameters; generating super-prior features based on the recovered statistical features, the generated third shared feature, and the estimated adaptive super-decoding parameters; and generating a reconstructed image based on the generated super-prior features and the obtained recovered image.

[0006] According to an embodiment, an apparatus for adaptive neural image compression using a super-prior model through meta-learning includes: at least one memory configured to store program code; and at least one processor configured to read the program code and operate according to the instructions of the program code to perform the method described in the above embodiment.

[0007] According to an embodiment, an apparatus for adaptive neural image compression using a super-prior model through meta-learning includes: a statistical feature generation module configured to generate statistical features based on an input image and hyperparameters; a first shared feature generation module configured to generate first shared features; an adaptive encoding module configured to generate estimated adaptive encoding parameters; an encoding module configured to encode the input image based on the first shared features and the estimated adaptive encoding parameters to obtain a signal-encoded image; a second shared feature generation module configured to generate second shared features; an adaptive super-encoding module configured to generate estimated adaptive super-encoding parameters; a super-feature generation module configured to generate super-features based on the obtained signal-encoded image, the second shared features, and the estimated adaptive super-encoding parameters; and a compression module configured to compress the obtained signal-encoded image, the generated statistical features, and the generated super-features. The apparatus further includes: a decoding module configured to decode the compressed image to obtain a restored image, decode the compressed statistical features to obtain restored statistical features, and decode the compressed super features to obtain restored super features; a third shared feature generation module configured to generate a third shared feature; an adaptive super-decoding module configured to generate estimated adaptive super-decoding parameters; a super-prior feature generation module configured to generate super-prior features based on the restored statistical features, the third shared features, and the estimated adaptive super-decoding parameters; and a reconstruction module configured to generate a reconstructed image based on the generated super-prior features and the restored image.

[0008] According to an embodiment, a non-volatile computer-readable medium is used to store instructions that, when executed by at least one processor, generate statistical features based on an input image and hyperparameters; generate a first shared feature; generate estimated adaptive coding parameters; encode the input image based on the first shared feature and the estimated adaptive coding parameters to obtain a signal-coded image; generate a second shared feature; generate estimated adaptive supercoding parameters; generate a super feature based on the obtained signal-coded image, the second shared feature, and the estimated adaptive supercoding parameters; compress the obtained signal-coded image, the generated statistical features, and the generated super features; decode the compressed signal-coded image to obtain a recovered image, decode the compressed statistical features to obtain recovered statistical features, decode the compressed super features to obtain recovered super features; generate a third shared feature; generate estimated adaptive superdecoding parameters; generate a super-prior feature based on the recovered statistical features, the generated third shared feature, and the generated estimated adaptive superdecoding parameters; and generate a reconstructed image based on the generated super-prior feature and the recovered image. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of an environment in which the methods, apparatus and systems described herein can be implemented, according to embodiments.

[0010] Figure 2 yes Figure 1 A block diagram of example components for one or more devices.

[0011] Figure 3 This is a block diagram of a test apparatus for multi-rate neural image compression during the testing phase using microstructured nested masks and weight unification, according to an embodiment.

[0012] Figure 4(a) is a flowchart of the model parameters of the underlying NIC encoder and decoder modules, which are separated into a single neural network (NN).

[0013] Figure 4(b) is a schematic diagram of parameter partitioning in the NN layer.

[0014] Figure 5 This is a block diagram of the inference workflow of the meta-signal encoding module.

[0015] Figure 6 This is a block diagram of the inference workflow of the meta-supercoding NN module.

[0016] Figure 7 This is a block diagram of the inference workflow of the Meta-Decoding NN module.

[0017] Figure 8This is a block diagram of the inference workflow for the meta-signal decoding NN in the meta-super decoding module.

[0018] Figure 9 This is the workflow of the Ada-NIC model meta-training process.

[0019] Figure 10 This is a flowchart of a method for automatic adaptive neural image compression using flexible hyperprior parameters through meta-learning.

[0020] Figure 11 This is a block diagram of a device for automatic adaptive neural image compression using flexible hyperprior parameters through meta-learning. Detailed Implementation

[0021] This disclosure describes a method and apparatus for an adaptive neural image compression (Ada-NIC) framework that automatically and adaptively selects optimal model parameters for image compression based on the characteristics of a single input image. A meta-learning mechanism is used to automatically compute adaptive weight parameters of the underlying NIC model based on the current input image.

[0022] Figure 1 This is a schematic diagram of an environment 100 in which the methods, apparatus and systems described herein can be implemented according to an embodiment.

[0023] like Figure 1 As shown, environment 100 may include user equipment 110, platform 120, and network 130. The devices in environment 100 can be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0024] User equipment 110 includes one or more devices capable of receiving, generating, storing, processing, and / or providing information related to platform 120. For example, user equipment 110 may include computing devices (e.g., desktop computers, laptop computers, tablet computers, handheld computers, smart speakers, servers, etc.), mobile phones (e.g., smartphones, cordless phones, etc.), wearable devices (e.g., smart glasses or smartwatches), or similar devices. In some embodiments, user equipment 110 may receive information from and / or send information to platform 120.

[0025] Platform 120 includes one or more devices as described elsewhere herein. In some embodiments, platform 120 may include a cloud server or a group of cloud servers. In some embodiments, platform 120 may be designed to be modular, allowing software components to be swapped in or out. This allows platform 120 to be easily and / or quickly reconfigured for different uses.

[0026] In some implementations, as shown in the figures, platform 120 may be hosted in a cloud computing environment 122. It is worth noting that while the implementations described herein depict platform 120 as hosted in a cloud computing environment 122, in some implementations, platform 120 may not be cloud-based (i.e., it may be implemented outside of a cloud computing environment) or may be partially cloud-based.

[0027] The cloud computing environment 122 includes the environment of the hosting platform 120. The cloud computing environment 122 can provide services such as computing, software, data access, and storage, without requiring end users (e.g., user equipment 110) to know the physical location and configuration of the systems and / or devices of the hosting platform 120. As shown in the figure, the cloud computing environment 122 may include a set of computing resources 124 (collectively referred to as "computing resources 124" and individually as "computing resource 124").

[0028] Computing resource 124 includes one or more personal computers, workstations, server devices, or other types of computing and / or communication devices. In some embodiments, computing resource 124 may host platform 120. Cloud resources may include computing instances executing in computing resource 124, storage devices provided in computing resource 124, data transmission devices provided by computing resource 124, etc. In some embodiments, computing resource 124 may communicate with other computing resources 124 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0029] like Figure 1 As further shown, computing resources 124 include a set of cloud resources, such as one or more applications (“APP”) 124-1, one or more virtual machines (“VM”) 124-2, virtualized storage (“VS”) 124-3, one or more hypervisors (“HYP”) 124-4, etc.

[0030] Application 124-1 includes one or more software applications that can be provided to or accessed by user device 110 and / or platform 120. Application 124-1 can eliminate the need to install and execute software applications on user device 110. For example, application 124-1 may include software associated with platform 120 and / or any other software that can be provided through cloud computing environment 122. In some implementations, an application 124-1 can send / receive information to / from one or more other applications 124-1 via virtual machine 124-2.

[0031] Virtual machine 124-2 includes a software implementation of a machine (e.g., a computer) that executes programs, similar to a physical machine. Virtual machine 124-2 can be a system virtual machine or a process virtual machine, depending on the extent to which virtual machine 124-2 uses and corresponds to any real machine. A system virtual machine can provide a complete system platform that supports the execution of a complete operating system (“OS”). A process virtual machine can execute a single program and can support a single process. In some implementations, virtual machine 124-2 can execute on behalf of a user (e.g., user device 110) and can manage the infrastructure of cloud computing environment 122, such as data management, synchronization, or long-term data transfer.

[0032] Virtualized storage 124-3 includes one or more storage systems and / or one or more devices that utilize virtualization technology within the storage systems or devices of computing resource 124. In some implementations, the type of virtualization within the context of the storage system may include block virtualization and file virtualization. Block virtualization may refer to the abstraction (or separation) of logical storage from physical storage so that the storage system can be accessed without regard to physical storage or heterogeneous architecture. Separation allows storage system administrators to flexibly manage end-user storage. File virtualization can eliminate the dependency between data accessed at the file level and the location of physical storage files. This can optimize storage usage, server consolidation, and / or performance for non-disruptive file migration.

[0033] Hypervisor 124-4 provides hardware virtualization technology that allows multiple operating systems (e.g., "guest operating systems") to run simultaneously on a host computer such as computing resource 124. Hypervisor 124-4 can provide a virtual operating platform to the guest operating systems and manage their execution. Multiple instances of various operating systems can share virtualized hardware resources.

[0034] Network 130 includes one or more wired and / or wireless networks. For example, network 130 may include cellular networks (e.g., fifth-generation (5G) networks, long-term evolution (LTE) networks, third-generation (3G) networks, code division multiple access (CDMA) networks, etc.), public land mobile networks (PLMN), local area networks (LAN), wide area networks (WAN), metropolitan area networks (MAN), telephone networks (e.g., public switched telephone network (PSTN)), private networks, self-organizing networks, intranets, the Internet, fiber-optic based networks, etc., and / or combinations of these or other types of networks.

[0035] Figure 1 The number and arrangement of devices and networks shown are provided as an example. In reality, with... Figure 1Compared to the devices and / or networks shown, there can be more devices and / or networks, fewer devices and / or networks, different devices and / or networks, or different arrangements of devices and / or networks. Furthermore, Figure 1 The two or more devices shown can be implemented within a single device, or Figure 1 The single device shown can be implemented as multiple distributed devices. Alternatively, a group of devices in environment 100 (e.g., one or more devices) can perform one or more functions described as being performed by another group of devices in environment 100.

[0036] Figure 2 yes Figure 1 A block diagram of example components for one or more devices.

[0037] Device 200 may correspond to user device 110 and / or platform 120. For example... Figure 2 As shown, device 200 may include bus 210, processor 220, memory 230, storage component 240, input component 250, output component 260 and communication interface 270.

[0038] Bus 210 includes components that allow communication between components of device 200. Processor 220 is implemented in hardware, firmware, or a combination of hardware and software. Processor 220 is a central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), microprocessor, microcontroller, digital signal processor (DSP), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), or another type of processing component. In some embodiments, processor 220 includes one or more processors that can be programmed to perform functions. Memory 230 includes random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic storage, and / or optical storage) that stores information and / or instructions for use by processor 220.

[0039] Storage component 240 stores information and / or software related to the operation and use of device 200. For example, storage component 240 may include hard disks (e.g., magnetic disks, optical disks, magneto-optical disks, and / or solid-state disks), optical disks (CDs), digital versatile disks (DVDs), floppy disks, cassette tapes, magnetic tapes, and / or other types of non-volatile computer-readable media, and corresponding drives.

[0040] Input component 250 includes components that allow device 200 to receive information, such as a touchscreen display, keyboard, keypad, mouse, buttons, switches, and / or microphone. Alternatively, input component 250 may include sensors for sensing information (e.g., a Global Positioning System (GPS) component, accelerometer, gyroscope, and / or actuator). Output component 260 includes components that provide output information from device 200, such as a display, speaker, and / or one or more light-emitting diodes (LEDs).

[0041] Communication interface 270 includes transceiver-like components (e.g., a transceiver and / or separate receiver and transmitter) that enable device 200 to communicate with other devices, for example, via a wired connection, a wireless connection, or a combination of wired and wireless connections. Communication interface 270 may allow device 200 to receive information from and / or provide information to another device. For example, communication interface 270 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, etc.

[0042] Device 200 can perform one or more processes described herein. Device 200 can perform these processes in response to processor 220 executing software instructions stored in a non-volatile computer-readable medium (e.g., memory 230 and / or storage component 240). Computer-readable medium is defined herein as a non-volatile memory device. A memory device includes storage space within a single physical storage device or storage space distributed across multiple physical storage devices.

[0043] Software instructions can be read into memory 230 and / or storage component 240 from another computer-readable medium or from another device via communication interface 270. When executed, the software instructions stored in memory 230 and / or storage component 240 can cause processor 220 to perform one or more processes described herein. Alternatively or additionally, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more processes described herein. Therefore, the embodiments described herein are not limited to any particular combination of hardware circuitry and software.

[0044] Figure 2 The number and arrangement of components shown are provided as an example. In fact, with... Figure 2 Compared to the components shown, device 200 may include more components, fewer components, different components, or components arranged differently. Alternatively, a set of components of device 200 (e.g., one or more components) may perform one or more functions described as being performed by another set of components of device 200.

[0045] A method and apparatus for adaptive neural image compression (Ada-NIC) using adaptive model parameter selection through meta-learning will now be described in detail.

[0046] This disclosure proposes an Ada-NIC framework that supports adaptive model parameter selection. A meta-learning mechanism is used to automatically compute adaptive weight parameters of the underlying NIC model based on the current input image, enabling the Ada-NIC model to improve image compression.

[0047] Figure 3 This is a block diagram of a test apparatus 300 for using a flexible advanced prior model through meta-learning during the testing phase of an Ada-NIC, according to an embodiment.

[0048] like Figure 3 As shown, the test device 300 includes a primary signal encoding module 310, a primary super encoding module 320, a signal Q&AE module 330, a super Q&AE module 331, a primary Q&AE module 332, a signal AD&DQ module 340, a super AD&DQ module 341, a primary AD&DQ module 342, a primary super decoding module 350, and a primary signal decoding module 360.

[0049] Given an input image x of size (h, w, c), where h, w, and c are the height, width, and number of channels, respectively, the objective of the testing phase of the NIC workflow can be described as follows: Compute a compact compressed representation for storage and transmission. Then, based on this compressed representation... Reconstructed output image And the reconstructed output image It can be similar to the original input x. Distortion loss Used to measure reconstruction errors, such as PSNR or SSIM. Calculate the rate loss R to measure the compressed representation. Bit consumption. The hyperparameter λ is used to form the joint rate distortion (RD) loss:

[0050]

[0051] Training with a larger hyperparameter λ results in less distortion in the compressed model, but more bit consumption, and vice versa.

[0052] Figure 4(a) is a flowchart of the model parameters 401, 402, 411, 412, 421, 422, 431, and 432 of the bottom-level NIC encoder modules 400 and 410 and decoder modules 420 and 430, which are separated into individual NNs.

[0053] The model parameters of the underlying NIC encoder and decoder are divided into 8 parts. These represent Shared Signal Encoding Parameters (SSEP) 401, Adaptive Signal Encoding Parameters (ASEP) 402, Shared Signal Decoding Parameters (SSDP) 432, Adaptive Signal Decoding Parameters (ASDP) 431, Shared Super Coding Parameters (SHEP) 411, Adaptive Super Coding Parameters (AHEP) 412, Shared Super Decoding Parameters (SHDP) 422, and Adaptive Super Decoding Parameters (AHDP) 421, respectively. In an embodiment of the NIC network architecture, SSEP 401, ASEP 402, SHEP 411, AHEP 412, AHDP 421, SHDP 422, ASDP 431, and SSDP 432 are separate individual NN modules, each comprising one or more NN layers. These individual modules are sequentially interconnected for network forward computation. Figure 4(a) illustrates the preferred order of connecting these individual NN modules. Other orders may also be used.

[0054] Parameter partitioning can also exist within the NNN layer. Let Let SSEP and ASEP represent the i-th layer of the NIC signal encoder 400. Let represent the SHEP and AHEP of the i-th layer of the NIC super encoder 410. Let Let AHDP and SHDP represent the j-th layer of the NIC super decoder 420. Let Let ASDP and SSDP represent the j-th layer of the NIC signal decoder 430. The network computes inference outputs based on the corresponding inputs of SSEP and ASEP, or SHEP and AHEP, or AHDP and SHDP, or ASDP and SSDP, respectively. These outputs are combined (e.g., through addition, concatenation, multiplication, etc.) and then sent to the next layer of the NIC module. The output of one layer is the input of the next layer. Figure 4(b) is an example with parameter partitioning in the NN layers of the NIC signal encoder 400, super encoder 410, super decoder 420, and signal decoder 430. The i-th layer SSEP 402-1 and ASEP 402-1 is combined and input into the next layer SSEP. 402-2 and ASEP 402-2, and so on. The last few layers of the signal encoder 400 are combined and sent to the super encoder 410. The i-th layer is SHEP 411-1 and 412-1 is combined and input into the next layer SHEP. 411-2 and AHEP 412-2, and so on. The last few layers of the super encoder 410 are combined and sent to the super decoder 420. The j-th layer AHDP... 421-1 and SHDP 422-1 is combined and input into the next layer AHDP. 421-2 and SHDP 422-2, and so on. The last few layers of the super decoder 420 are combined and sent to the signal decoder 430. The j-th layer ASDP... 431-1 and SSDP 432-1 is combined and input into the next layer ASDP. 431-2 and SSDP 432-2, and so on.

[0055] The flowchart in Figure 4(a) can be viewed as a special case of Figure 4(b), where the shared signal encoding NN... The layers in 402-1 are empty; adaptive signal encoding NN The layers in 401-1 are empty, sharing the supercode NN. The layers in 412-1 are empty; adaptive supercoding NN The layers in 411-1 are empty, sharing the signal decoding NN. The layers in 431-1 are empty; adaptive signal decoding NN The layers in 432-1 are empty, sharing the super-decoding NN. The layers in 421-1 are empty; adaptive super-decoding NN The layer in 422-1 is empty. Therefore, in other embodiments, the networks shown in Figures 4(a) and 4(b) can be combined, wherein the NIC architecture includes both pure shared encoding / decoding layers and / or pure adaptive encoding / decoding layers, as well as hybrid layers with partially shared encoding / decoding parameters and partially adaptive encoding / decoding parameters.

[0056] Figure 3 The general notation is shown because for a completely shared layer, and / or and / or and / or It's empty. For a fully adaptive layer, and / or and / or It is empty. Therefore, Figure 3 The symbols shown can be used in the two embodiments of Figure 4.

[0057] Specifically, refer to Figure 3 Given an input image x and hyperparameters λ, the meta-signal coding module 310 computes the latent representation y and statistical features μ. Figure 5This is a block diagram of the inference workflow of the meta-signal encoding module 310. In this embodiment, the meta-signal encoding module 310 uses a statistical analysis neural network 500 and a meta-signal encoding neural network 510. The input image x is passed through the statistical analysis module 500 to generate statistical features μ, which captures the statistical properties of the input image x. The statistical analysis module 500 can use a neural network (e.g., including convolutional layers and fully connected layers) to compute the statistical features μ. The statistical analysis module 500 can also use conventional methods (e.g., color and texture extraction methods) to compute the statistical features μ. Let f(i) and f(i+1) represent the input and output tensors of the i-th layer of the meta-signal encoding neural network 510. Based on the input f(i) and SSEP SSEP Inference Module 511 Based on Shared Inference Functions To compute the shared feature g(i) (i.e., the function is modeled using the forward computation of SSEP in the i-th layer). Based on the input f(i), shared feature g(i), and ASEP... The ASEP prediction module 512 calculates the estimated ASEP for the i-th layer of the metadata encoding NN 510 using statistical features μ and hyperparameter λ.

[0058] The ASEP prediction module 512 is a neural network, for example, including convolutional layers and fully connected layers, which is based on the original ASEP. The current input, statistical features μ, and hyperparameters λ are used to predict the updated ASEP. In some embodiments, the input f(i) is used as the input to the ASEP prediction module 512. In some other embodiments, the shared feature g(i) instead of the input f(i) is used as the input to the ASEP prediction module 512. In other embodiments, the SSEP loss can be calculated based on the shared feature g(i). Figure 5 (Not shown), and the gradient of the loss is used as the input to the ASEP prediction module 512. Based on the estimated ASEP And sharing features g(i), the ASEP inference module 513 is based on the ASEP inference function The output tensor f(i+1) is computed (i.e., the function is modeled by using the forward computation of the estimated ASEP in the i-th layer).

[0059] It should be noted that Figure 5 The workflow described in [the document] is a general notation. For ASEP... An empty, fully shared layer can omit modules related to ASEP, and f(i+1) = g(i). For SSEP An empty, fully adaptive layer can omit the modules related to SSEP, and g(i) = f(i).

[0060] Assume the original signal encoding NN 510 has a total of N S The output of the last layer is the latent representation y. Then, the latent representation y is passed through the meta-supercoding module 320 to compute the superfeature z. Figure 6 This is a block diagram of the inference workflow of the meta-super-encoder NN 600 in the meta-super-encoder module 320. Let h(i) and h(i+1) represent the input and output tensors of the i-th layer of the meta-super-encoder NN 600. Based on the input h(i) and SHEP... SHEP Inference Module 610 is based on shared inference functions To compute the shared feature p(i) (i.e., the function is modeled using the forward computation of SHEP in the i-th layer). Based on the input h(i), shared feature p(i), and AHEP Statistical features μ and hyperparameter λ are used to compute the estimated AHEP of the i-th layer of the meta-super-encoded NN 600 using the AHEP prediction module 620.

[0061] The AHEP prediction module 620 is a neural network, including convolutional and fully connected layers, based on the original AHEP. The current input, statistical features μ, and hyperparameters λ are used to predict the updated AHEP. In some embodiments, h(i) is used as input to the AHEP prediction module 620. In some other embodiments, shared features p(i) instead of input h(i) are used as input to the AHEP prediction module 620. In other embodiments, the SHEP loss can be calculated based on shared features p(i). Figure 5 (Not shown), and the gradient of the loss is used as input to the ASEP prediction module 620. Based on the estimated AHEP And sharing features p(i), the AHEP inference module 630 is based on the AHEP inference function. To compute the output tensor h(i+1) (i.e., this function is modeled using the forward computation of the estimated AHEP in the i-th layer). Similarly, Figure 6 The workflows described in [the document] are general symbols. For AHEP... An empty, fully shared layer, omitting modules related to AHEP, and h(i+1) = p(i). For SHEP The layer is an empty, fully adaptive layer, omitting the SHEP-related modules, and p(i) = h(i).

[0062] Assume that the meta-super-encoder NN600 has a total of N HThe output of the last layer is the super-feature z. Then, the latent representation y is passed through the signal Q&AE module 330 to be further compressed (through quantization and arithmetic entropy encoding) into a compact representation y′. The statistical feature μ is passed through the meta-Q&AE module 332 to generate the compact meta-feature μ′. The super-feature z is passed through the super-Q&AE module 331 to generate the compact super-feature z′. Then, the compact representation y′, the compact meta-feature μ′, and the compact super-feature z′, along with the hyperparameter λ, are sent to the decoder (e.g., ...). Figure 3 (As shown). In this embodiment, the latent representation y passes through the signal Q&AE module 330, then the statistical feature μ passes through the meta-Q&AE module 332, and then the super-feature z passes through the super-Q&AE module 331. However, this disclosure is not limited to this order.

[0063] On the decoder side, the received compact representation y′ is passed through the signal AD&DQ module 340 to calculate the recovered latent representation. (Through arithmetic entropy decoding and dequantization processes). Similarly, the compact meta-feature μ′ is processed by the meta-AD&DQ module 342 to compute the recovered statistical features. The compact super-feature z′ is computed via the super AD&DQ module 341 to recover the super-feature. (like Figure 3 (As shown).

[0064] Then, hyperfeatures The meta-super-decoding module 350 is used to compute the super-prior features. Figure 7 This is a block diagram of the inference workflow of the meta-superdecoding NN 700 within the meta-superdecoding module 350. Let h(j) and h(j+1) represent the input and output tensors of the j-th layer of the meta-superdecoding NN 700. Based on the input h(j) and SHDP Inference Module 710 is based on shared inference functions To compute the shared feature p(j) (i.e., the function is modeled using the forward computation of SHDP in the j-th layer). Based on the input h(j), shared feature p(j), and AHDP Statistical characteristics of recovery And with the hyperparameter λ, the AHDP prediction module 720 calculates the estimated AHDP for the j-th layer. The AHDP prediction module 720 is a neural network, including convolutional and fully connected layers, based on the original AHDP. Current input and recovery statistical characteristics And hyperparameter λ to predict updated AHDP In some embodiments, input h(j) is used as input to the AHDP prediction module 720. In some other embodiments, shared features are used. Instead of using the input h(j) as the input to the AHDP prediction module 720, in other embodiments, the SHDP loss can be calculated based on the shared feature p(j). Figure 7 (Not shown), and the gradient of the loss is used as input to the ASDP prediction module 720. Based on the estimated AHDP And sharing features p(j), the AHDP inference module 730 is based on the AHDP inference function. To compute the output tensor h(j+1) (i.e., this function is modeled using the forward computation of the estimated AHDP in the j-th layer). With Figure 5 and Figure 6 The encoder workflow described in [the document] is similar. Figure 7 The workflows described are based on generic symbols. For AHDP... An empty, fully shared layer, omitting AHDP-related modules, and h(j+1) = p(j). For SHDP The layer is an empty, fully adaptive layer, omitting the SHDP-related modules, and p(j) = h(j).

[0065] Assume that the total number of N700 in the meta-decoding NN is M. H If the layer is layered, then the output of the last layer is the super-prior feature. Then, the super-prior features and potential representations of recovery The reconstructed image is calculated using the meta-signal decoding module 360. Figure 8 This is a block diagram of the inference workflow of the meta-signal decoding NN 800 in the meta-signal decoding module 360. Let f(j) and f(j+1) represent the input and output tensors of the j-th layer of the meta-signal decoding NN800. Based on the input f(j) and SSDP SSDP Inference Module 810 is based on shared inference functions To compute the shared feature g(j) (i.e., the function is modeled using the forward computation of SSDP in the j-th layer). Based on the input f(j), shared feature g(j), and ASDP... Statistical characteristics of recovery And with the hyperparameter λ, the ASDP prediction module 820 calculates the estimated ASDP of the j-th layer. The ASDP prediction module 820 is a neural network, including convolutional and fully connected layers, based on the original ASDP. Current input and recovery statistical characteristics And hyperparameter λ to predict updated ASDP In some embodiments, the input f(j) is used as the input to the ASDP prediction module 820. In some other embodiments, the shared feature g(j) instead of the input f(j) is used as the input to the ASDP prediction module 820. In other embodiments, the SSDP loss can be calculated based on the shared feature g(j). Figure 8 (Not shown), and the gradient of the loss is used as input to the ASDP prediction module 820. Based on the estimated ASDP And sharing features g(j), the ASDP inference module 830 is based on the ASDP inference function To compute the output tensor f(j+1) (i.e., the function is modeled by using the forward computation of the estimated ASDP in the j-th layer). Figure 8 The workflows described in the documentation are general symbols. For ASDP... An empty, fully shared layer, omitting modules related to ASDP, and f(j+1) = g(j). For SSDP... The layer is an empty, fully adaptive layer, omitting the modules related to SSDP, and g(j) = f(j).

[0066] Assume that the total number of elementary signal decoders NN 800 is M. S If the layers are continuous, then the output of the last layer is the reconstructed image. Note that the proposed Ada-NIC framework supports adaptive image compression on arbitrarily smooth image data distributions. As a special case, the ASEP prediction module 512 can perform predictions by selecting from a set of pre-trained ASEPs, each corresponding to a training dataset with the target data distribution, rather than predicting arbitrary model parameters. Similarly, the AHEP prediction module 620, AHDP prediction module 720, and / or ASDP prediction module 820 can select model parameters from a set of pre-trained AHEPs, AHDPs, and / or ASDPs. In this case, the Ada-NIC method simplifies to an adaptive NIC model selection method, where the best pre-trained model instance is automatically selected based on the current input image.

[0067] The training process aims to learn the SSEP of the meta-signal encoding NN510. and ASEP i = 1, ..., N S SHEP of NN 600 supercode and AHEP i = 1, ..., N H SHDP of Yuanchao Decoding NN 700 and AHDP j = 1, ..., M H SSDP of NN 800 for decoding meta-signals and ASDP j = 1, ..., M S And ASEP predicts NN (represented as Model parameters), AHEP prediction NN (represented as Model parameters), ASDP prediction NN (represented as The model parameters of ) and the AHDP prediction NN (represented as (Model parameters).

[0068] In this embodiment, the Model-Agnostic Meta-Learning (MAML) mechanism is used for training purposes. Figure 9 This is the workflow of the Ada-NIC model meta-training process 900. Of course, other meta-training algorithms can also be used here.

[0069] Specifically, for training, there exists a set of training data D. tr (π i ), i = 1, ..., K, where each D tr (π i ) corresponds to a training data distribution π i There are a total of K training data distributions. Note that this is a general notation because each training data point can be considered a separate distribution, and K will be the same size as the entire training set. Additionally, there exists a set of validation data D. val (π j ), j = 1, ..., P, where each D val (π j ) corresponds to a validation data distribution π j The validation data distribution includes the data distribution in the training set. The validation data distribution may also include data distributions not included in the training set.

[0070] The goal of the training process is to learn the Ada-NIC model so that NIC tasks with target data distributions are learned from the task distribution P(π). j Under the assumptions extracted from [the model], this model can be broadly applied to all data distributions (including training and future unseen data). To achieve this, the loss of learning the Ada-NIC model is minimized across all training datasets across all training data distributions.

[0071] make Includes all shared model parameters, and lets Includes all adaptive model parameters. Let Includes all prediction model parameters on the encoder side (400, 410), and lets This includes all predicted model parameters on the decoder side (420, 430). The MAML training process 900 includes an outer loop and an inner loop for gradient-based parameter updates. For each outer loop iteration, in the task sampling module 910, a set of K′ training data distributions (K′≤K) is first sampled. Then, for each sampled training data distribution π i From D tr (π i Sample a set of training data A set of validation data distributions P′ (P′≤P) is also sampled. For each sampled validation π j From D val (π j Sample a set of validation data Then, for each sampled data Based on the current parameter Θ s Θ a Φ e and Φ d Perform Ada-NIC forward computation. Then, in the inner loop loss calculation module 920, calculate the accumulated inner loop loss according to equation (2).

[0072]

[0073] Loss function L(x, Θ) s Θ a Φ e , Φ d , π i This includes the RD loss described in equation (1) and other conventional losses (e.g., auxiliary losses that distinguish intermediate network outputs for different tradeoffs). Then, based on... Given step size α si and α ai As π i The hyperparameters are calculated in the inner loop update module 930 according to equations (3) and (4) to update the specific task parameters:

[0074]

[0075]

[0076] Accumulated internal circulation losses gradient and Used to calculate adaptive parameters respectively and The updated version.

[0077] Then, the external meta-target can be calculated in the meta-loss calculation module 940 according to equations (5) and (6) for the distribution of all sampled validation data:

[0078]

[0079] in, Based on Ada-NIC forward computation parameters Φ s , Φ a The loss calculated for input x.

[0080] Given step size β aj and β sj As π j The hyperparameters are updated in the meta-update module 950 according to equations (7) and (8):

[0081]

[0082]

[0083] In some embodiments, Θ may not be updated in the inner loop. s That is, α si =0, This can help stabilize the training process.

[0084] Regarding the parameters Φ of ASEP predictive NN, AHEP predictive NN, ASDP predictive NN 820, and AHDP predictive NN 720... e , Φ d These are updated in the weight update module 960 using the regular training method. That is, based on the training and validation data D... tr (π i ), i = 1, ..., K, D val (π j ), j = 1, ..., P, based on the current Θ s Θ a Φ e , Φ d It can be any sample x∈D tr (π i Calculate the loss L(x, Θ) s Θ a , Φ e , Φ d , π i ), and for all samples x∈D val (π j Calculate the loss L(x, Θ) s Θ a , Φ e , Φd , π j The gradients of all these losses can be accumulated (e.g., added) to apply to Φ via regular backpropagation. e , Φ d Execution parameters update.

[0085] Please note that this disclosure does not impose any restrictions on the specific optimization algorithms or loss functions used to update these model parameters.

[0086] For the special case where the ASEP prediction module 512, AHEP prediction module 620, ASDP prediction module 820, and AHDP prediction module of the Ada-NIC model perform predictions only on a predefined set of training data distributions, the validation data distribution will be the same as the training data distribution. The same MAML training procedure can be used to train this simplified Ada-NIC model.

[0087] Figure 10 This is a flowchart of a method for automatically adaptive neural image compression using flexible super-prior parameters through meta-learning, according to an embodiment.

[0088] In some implementations... Figure 10 One or more process frames can be executed by platform 120. In some implementations, Figure 10 One or more process frames may be executed by another device or a group of devices that are separate from or include the platform 120, such as user equipment 110.

[0089] like Figure 10 As shown, in operation 1000, the method includes generating statistical features based on the input image and hyperparameters.

[0090] In operation 1001, Figure 10 The method includes generating first shared features and estimating adaptive coding parameters.

[0091] In operation 1002, Figure 10 The method involves encoding the input image based on the generated first shared features and the generated estimated adaptive coding parameters to obtain a signal-coded image.

[0092] In operation 1003 Figure 10 The method includes generating a second shared feature and estimating adaptive hypercoding parameters.

[0093] In operation 1004, Figure 10 The method involves generating super-features based on the signal-coded image, the generated second shared features, and the generated estimated adaptive supercoding parameters.

[0094] In operation 1005, Figure 10The method includes compressing the obtained signal-encoded image, generating statistical features, and generating super-features.

[0095] In operation 1006, Figure 10 The method includes decoding the compressed signal-coded image to obtain the recovered image, decoding the compressed statistical features to obtain the recovered statistical features, and decoding the compressed super features to obtain the recovered super features.

[0096] In operation 1007 Figure 10 The method includes generating third shared features and estimating adaptive super-decoding parameters.

[0097] In operation 1008, Figure 10 The method involves generating super-prior features based on the recovered statistical features, the generated third shared features, and the estimated adaptive super-decoding parameters.

[0098] In operation 1009, Figure 10 The method involves generating a reconstructed image based on the generated prior features and the obtained restored image.

[0099] Although Figure 10 An example box of the method is shown, but in some implementations, the method may include more than Figure 10 The described number of boxes, fewer boxes, different boxes, or boxes arranged differently. Alternatively, two or more boxes in this method can be executed in parallel.

[0100] According to an embodiment, Figure 10 The flowchart method shown can be implemented in each layer of the NN in the signal encoder and decoder modules. Additionally, when parameter partitioning exists within the NN layers of the encoder and decoder modules (as shown in Figure 4(b)), Figure 10 The method can also be implemented at each layer.

[0101] Figure 11 This is a block diagram of an apparatus for automatically adaptive neural image compression using flexible prior parameters through meta-learning, according to an embodiment.

[0102] like Figure 11 As shown, the device includes statistical feature generation code 1100, first shared feature generation code 1101, adaptive encoding code 1102, encoding code 1103, second shared feature generation code 1104, adaptive super encoding code 1105, super feature generation code 1106, compression code 1107, decoding code 1108, third shared feature generation code 1109, adaptive super decoding code 1110, super prior feature generation code 1111, and reconstruction code 1112.

[0103] Statistical feature generation code 1100 is configured to enable at least one processor to generate statistical features based on the input image and hyperparameters.

[0104] The first shared feature generation code 1101 is configured to cause at least one processor to generate the first shared feature.

[0105] Adaptive coding code 1102 is configured to cause at least one processor to generate estimated adaptive coding parameters.

[0106] Encoding code 1103 is configured to enable at least one processor to encode the input image based on the generated first shared features and the generated estimated adaptive encoding parameters to obtain a signal-encoded image.

[0107] The second shared feature generation code 1104 is configured to cause at least one processor to generate a second shared feature.

[0108] Adaptive supercoding code 1105 is configured to cause at least one processor to generate estimated adaptive supercoding parameters.

[0109] The super-feature generation code 1106 is configured to enable at least one processor to generate super-features based on the obtained signal-coded image, the generated second shared features, and the generated estimated adaptive super-coding parameters.

[0110] Compression code 1107 is configured to enable at least one processor to compress the obtained signal-encoded image, generated statistical features, and super-features.

[0111] Decoding code 1108 is configured to enable at least one processor to decode the compressed signal-coded image to obtain the recovered image, decode the compressed statistical features to obtain the recovered statistical features, and decode the compressed super-features to obtain the recovered super-features.

[0112] The third shared feature generation code 1109 is configured to cause at least one processor to generate a third shared feature.

[0113] The adaptive super-decoding code 1110 is configured to cause at least one processor to generate estimated adaptive super-decoding parameters.

[0114] The super-prior feature generation code 1111 is configured to enable at least one processor to generate super-prior features based on the recovered statistical features, the generated third shared features, and the generated estimated adaptive super-decoding parameters.

[0115] Reconstruction code 1112 is configured to enable at least one processor to generate a reconstructed image based on the prior features and the recovered image.

[0116] Although Figure 11An example block of the device is shown, but in some embodiments, the device may include more than Figure 11 The described number of boxes, fewer boxes, different boxes, or different arrangements of boxes. Alternatively, two or more boxes of the device can be combined.

[0117] The embodiments describe an automatically adaptive NIC based on the features of the input image. Its advantages include flexible model parameter prediction and pre-trained model selection supporting arbitrarily smooth data distributions. Furthermore, the embodiments include a flexible and general framework adaptable to various underlying NIC models, architectures, and meta-learning methods.

[0118] The proposed methods can be used individually or in any order. Furthermore, each of the methods (or embodiments), encoders, and decoders can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-volatile computer-readable medium.

[0119] The foregoing disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit the embodiments to the precise forms disclosed. Modifications and variations can be made based on the foregoing disclosure, or modifications and variations can be derived from practice of the embodiments.

[0120] As used herein, the term “component” is intended to be interpreted broadly as hardware, firmware, or a combination of hardware and software.

[0121] It is evident that the systems and / or methods described herein can be implemented in various forms, including hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, this document describes the operation and behavior of the systems and / or methods without referring to any specific software code—it should be understood that software and hardware can be designed to implement the system and / or method based on the description herein.

[0122] Even if specific combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible embodiments. In fact, many of these features can be combined in ways not specifically recited in the claims and / or disclosed in the specification. While each dependent claim listed below may directly depend on only one claim, the disclosure of possible embodiments includes combinations of each dependent claim in the claim set with each of the other claims.

[0123] Unless explicitly stated otherwise, elements, actions, or instructions used herein should not be construed as critical or necessary. Furthermore, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” Additionally, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, combinations of related and unrelated items, etc.) and may be used interchangeably with “one or more.” The term “one” or similar language is used where only one item is referred to. Furthermore, as used herein, the terms “has,” “have,” “having,” etc., are intended to be open-ended terms. Additionally, the phrase “based on” is intended to mean “at least partially based on,” unless explicitly stated otherwise.

Claims

1. A method for adaptive neural image compression using a super-prior model through meta-learning, characterized in that, The method includes: Statistical features are generated based on the input image and hyperparameters; Generate the first shared features and the estimated adaptive coding parameters; Based on the generated first shared features and the generated estimated adaptive coding parameters, the input image is encoded to obtain a signal-coded image; Generate a second shared feature and estimated adaptive hypercoding parameters; Based on the signal-coded image, the generated second shared feature, and the generated estimated adaptive supercoding parameters, a super feature is generated; The obtained signal-encoded image, the generated statistical features, and the generated super-features are compressed; The compressed signal-encoded image is decoded to obtain the recovered image; the compressed statistical features are decoded to obtain the recovered statistical features; and the compressed super features are decoded to obtain the recovered super features. Generate the third shared feature and the estimated adaptive super-decoding parameters; Based on the recovered statistical features, the generated third shared features, and the generated estimated adaptive super-decoding parameters, super-prior features are generated; and Based on the generated prior features and the obtained recovered image, a reconstructed image is generated.

2. The method according to claim 1, characterized in that, The generation of the first shared feature and the estimated adaptive coding parameters include: Based on the generated statistical features and shared signal encoding parameters, the first shared feature of one layer of a plurality of layers in a first neural network is generated; and Convolution is performed based on the generated first shared features, static features, and adaptive signal coding parameters to compute the estimated adaptive coding parameters.

3. The method according to claim 2, characterized in that, Further includes: The estimated adaptive coding parameters are updated in multiple layers of the first neural network.

4. The method according to claim 2, characterized in that, Further includes: Based on the generated first shared features and the estimated adaptive coding parameters, the coding output of the layers in the plurality of layers of the first neural network is calculated. The encoded output of the last layer in the plurality of layers of the first neural network is the signal encoded image.

5. The method according to claim 1, characterized in that, The generation of the second shared feature and the estimated adaptive hypercoding parameters include: Based on the generated statistical features, the obtained signal-encoded image, and the supercoding parameters, the second shared feature of one layer of a plurality of layers in a second neural network is generated; and Convolution is performed based on the generated second shared features, static features, and adaptive signal coding parameters to compute the estimated adaptive supercoding parameters.

6. The method according to claim 5, characterized in that, Further includes: The estimated adaptive supercoding parameters are updated in multiple layers of the second neural network.

7. The method according to claim 5, characterized in that, Further includes: Based on the generated second shared features and the estimated adaptive supercoding parameters, the superoutput of the layers in the second neural network is generated. The superoutput of the last layer in the second neural network is the superfeature.

8. The method according to claim 1, characterized in that, The generation of the third shared feature and the estimated adaptive super-decoding parameters include: Based on the compressed super-features and shared super-decoding parameters, the third shared feature of one layer in a plurality of layers of the third neural network is calculated; and Convolution is performed based on the generated third shared feature, the recovered statistical feature, and the adaptive super-decoding parameters to compute the estimated adaptive super-decoding parameters.

9. The method according to claim 8, characterized in that, Further includes: The estimated adaptive super-decoding parameters are updated in multiple layers of the third neural network.

10. The method according to claim 8, characterized in that, Further includes: Based on the generated third shared features and the estimated adaptive super-decoding parameters, the super-prior output of the layers in the third neural network is calculated. The super-prior output of the last layer in the third neural network is the super-prior feature.

11. The method according to any one of claims 1-10, characterized in that, Further includes: Based on the aforementioned prior features, the recovered signal-encoded image, and the shared signal decoding parameters, a fourth shared feature of one layer in a plurality of layers of the fourth neural network is generated; Convolution is performed based on the generated fourth shared feature, the recovered statistical features, and the adaptive super-decoding parameters to compute the estimated adaptive signal decoding parameters; as well as Generate the decoding output of the layer in the plurality of layers of the fourth neural network, the estimated adaptive signal decoding parameters, wherein the decoding output of the last layer in the plurality of layers of the fourth neural network is the reconstructed image.

12. An apparatus for adaptive neural image compression using a super-prior model through meta-learning, characterized in that, The device includes: At least one memory is configured to store program code; and At least one processor is configured to read the program code and operate according to the instructions of the program code to perform the method according to any one of claims 1-11.

13. An apparatus for adaptive neural image compression using a super-prior model through meta-learning, characterized in that, The device includes: The statistical feature generation module is configured to generate statistical features based on the input image and hyperparameters; The first shared feature generation module is configured to generate the first shared feature; The adaptive coding module is configured to generate estimated adaptive coding parameters; An encoding module is configured to encode the input image based on the generated first shared features and the generated estimated adaptive encoding parameters to obtain a signal-encoded image; The second shared feature generation module is configured to generate a second shared feature. The adaptive supercoding module is configured to generate estimated adaptive supercoding parameters; The super-feature generation module is configured to generate super-features based on the signal-encoded image, the generated second shared features, and the generated estimated adaptive super-coding parameters; The compression module is configured to compress the obtained signal-encoded image, the generated statistical features, and the generated super-features; The decoding module is configured to decode the compressed signal-encoded image to obtain the recovered image, decode the compressed statistical features to obtain the recovered statistical features, and decode the compressed super-features to obtain the recovered super-features. The third shared feature generation module is configured to generate third shared features; An adaptive super-decoding module is configured to generate estimated adaptive super-decoding parameters; The super-prior feature generation module is configured to generate super-prior features based on the recovered statistical features, the generated third shared features, and the generated estimated adaptive super-decoding parameters; and The reconstruction module is configured to generate a reconstructed image based on the generated prior features and the obtained recovered image.

14. The apparatus according to claim 13, characterized in that, The first shared feature generation module is configured to generate the first shared feature of one layer of a plurality of layers of a first neural network based on the generated statistical features and shared signal encoding parameters. as well as The adaptive coding module is configured to perform convolution based on the generated first shared features, static features, and adaptive signal coding parameters to compute the estimated adaptive coding parameters; The estimated adaptive coding parameters are updated in multiple layers of the first neural network. The encoding module is configured to calculate the encoded output of one of the multiple layers of the first neural network based on the generated first shared features and the estimated adaptive encoding parameters. The encoded output of the last layer in the plurality of layers of the first neural network is the signal encoded image.

15. The apparatus according to claim 13, characterized in that, The second shared feature generation module is configured to generate the second shared feature of one layer of a plurality of layers in a second neural network based on the generated statistical features, the obtained signal-encoded image, and the supercoding parameters; and The adaptive supercoding module is configured to perform convolution based on the generated second shared features, static features, and adaptive signal coding parameters to compute the estimated adaptive supercoding parameters; The estimated adaptive supercoding parameters are updated in multiple layers of the second neural network. The super-feature generation module is configured to generate the super-output of one of the multiple layers of the second neural network based on the generated second shared features and the estimated adaptive supercoding parameters. The superoutput of the last layer in the second neural network is the superfeature.

16. The apparatus according to claim 13, characterized in that, The third shared feature generation module is configured to calculate the third shared feature of one layer of a plurality of layers of a third neural network based on the compressed super-features and shared super-decoding parameters; and The adaptive super-decoding module is configured to perform convolution based on the generated third shared features, the recovered statistical features, and the adaptive super-decoding parameters to calculate the estimated adaptive super-decoding parameters; The estimated adaptive super-decoding parameters are updated in multiple layers of the third neural network. The super-prior feature generation module is configured to calculate the super-prior output of one of the multiple layers of the third neural network based on the generated third shared features and the estimated adaptive super-decoding parameters. The super-prior output of the last layer in the third neural network is the super-prior feature.

17. A non-volatile computer-readable medium for storing instructions that, when executed by at least one processor for adaptive neural image compression using a super-prior model via meta-learning, cause the at least one processor to perform the method of any one of claims 1-11.

Images