Method and system for image data processing

Abstract

Methods and systems for compressing image data based on object size modification information. According to some embodiments, an exemplary encoding method may include: detecting a first object from an input image; determining a first size modification parameter associated with the first object; and compressing the input image based on the first size modification parameter.

Description

Technical Field

[0001] The present disclosure relates generally to image data processing, and more particularly to methods and systems for compressing image data based on object size adaptation information. Background Art

[0002] Limited communication bandwidth poses a challenge to transmitting images or videos (hereinafter collectively referred to as "image data") with desired quality. Although various encoding techniques have been developed to compress image data before transmitting it, the increasing demand for large amounts of image data has put increasing pressure on image / video systems, especially mobile devices that often need to work in networks with limited bandwidth. Summary of the invention

[0003] Embodiments of the present disclosure relate to methods for compressing image data based on object size modification information. In some embodiments, a computer-implemented encoding method is provided. The method may include: detecting a first object from an input image; determining a first size modification parameter associated with the first object; and compressing the input image based on the first size modification parameter.

[0004] In some embodiments, a device for encoding image data is provided. The device includes a memory storing instructions, and one or more processors. The one or more processors are configured to execute the instructions to cause the device to perform operations, the operations including: detecting a first object from an input image; determining a first size modification parameter associated with the first object; and compressing the input image based on the first size modification parameter.

[0005] In some embodiments, a computer-implemented decoding method is provided. The method may include: receiving one or more bitstreams, the one or more bitstreams including image data representing a first object of a first size; decoding the one or more bitstreams; determining a first size modification parameter associated with the first object; and generating a reconstructed image based on the first size modification parameter, the reconstructed image including the first object of a second size different from the first size.

[0006] In some embodiments, a device for decoding image data is provided. The device includes a memory storing instructions, and one or more processors. The one or more processors are configured to execute the instructions to cause the device to perform the following operations, the operations including: receiving one or more bitstreams, the one or more bitstreams including image data representing a first object of a first size; decoding the one or more bitstreams; determining a first size modification parameter associated with the first object; and generating a reconstructed image based on the first size modification parameter, the reconstructed image including the first object of a second size different from the first size.

[0007] Aspects of the disclosed embodiments may include a non-transitory tangible computer-readable medium storing software instructions that, when executed by one or more processors, are configured to and capable of performing and executing one or more of methods, operations, etc. consistent with the disclosed embodiments. In addition, aspects of the disclosed embodiments may be performed by one or more processors configured as one or more special-purpose processors based on software instructions that are programmed using logic and instructions. The software instructions, when executed, perform one or more operations consistent with the disclosed embodiments.

[0008] Additional purposes and advantages of the disclosed embodiments will be explained in part in the following description, and in part will become apparent from the description, or can be learned through the practice of the embodiments. The purposes and advantages of the disclosed embodiments can be realized and obtained by the elements and combinations set forth in the claims.

[0009] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a block diagram illustrating an exemplary system for encoding and decoding image data consistent with embodiments of the present disclosure.

[0011] Figure 2 is a block diagram illustrating an exemplary encoding process consistent with an embodiment of the present disclosure.

[0012] Figure 3 is a block diagram illustrating an exemplary decoding process consistent with an embodiment of the present disclosure.

[0013] Figure 4 is a block diagram of an exemplary apparatus for encoding or decoding image data consistent with embodiments of the present disclosure.

[0014] Figure 5 is a block diagram illustrating an exemplary process for controlling encoding of image data using object size modification information consistent with an embodiment of the present disclosure.

[0015] Figure 6 is a flow chart of an exemplary method for encoding image data consistent with embodiments of the present disclosure.

[0016] Figure 7 is a flow chart of an exemplary process for compressing image data consistent with an embodiment of the present disclosure.

[0017] Figure 8 It is consistent with the embodiment of the present disclosure. Figure 6 A block diagram of an exemplary implementation of the method in FIG.

[0018] Fig. 9 is a schematic diagram illustrating an atlas image including a collection of rescaled object images consistent with an embodiment of the present disclosure.

[0019] Fig.10 is a flow chart of an exemplary method for decoding image data consistent with embodiments of the present disclosure. DETAILED DESCRIPTION

[0020] Reference will be made in detail to the exemplary embodiments, examples of which are shown in the accompanying drawings. The following description refers to the accompanying drawings, in which the same reference numerals in different drawings represent the same or similar elements unless otherwise indicated. The implementations set forth in the following description of the exemplary embodiments do not represent all implementations consistent with the present invention. Instead, these implementations are merely examples of devices and methods consistent with various aspects related to the present invention as described in the appended claims.

[0021] The present disclosure relates to an image data compression method for removing data redundancy. In an exemplary embodiment, the redundancy may be related to the size of an object shown in a source image or video. On the one hand, storing or transmitting image data representing a large-sized object requires a large number of bits. On the other hand, such a large object size may not be necessary for performing certain machine vision tasks (e.g., object recognition and tracking). In order to exploit the redundancy, the disclosed compression method reduces (i.e., shrinks) the size of an object represented by the image data to an extent that the performance (e.g., accuracy) of the machine vision task is not degraded or is slightly degraded.

[0022] Figure 1 is a block diagram illustrating a system 100 for encoding and decoding image data according to some disclosed embodiments. Image data may include an image (also referred to as a "picture" or "frame"), multiple images, or a video. An image is a static picture. Multiple images may be spatially or temporally related or unrelated. A video includes a set of images arranged in a time sequence.

[0023] like Figure 1 As shown, system 100 includes a source device 120 that provides encoded image data to be decoded by a target device 140. Consistent with the disclosed embodiments, each of source device 120 and target device 140 may include any of a wide range of devices, including desktop computers, notebook (e.g., laptop) computers, servers, tablet computers, set-top boxes, mobile phones, vehicles, cameras, image sensors, robots, televisions, cameras, wearable devices (e.g., smart watches or wearable cameras), display devices, digital media players, video game consoles, video streaming devices, etc. Source device 120 and target device 140 may be equipped for wireless or wired communication.

[0024] refer to Figure 1 , the source device 120 may include an image / video source 122, an image / video encoder 124, and an output interface 126. The target device 140 may include an input interface 142, an image / video decoder 144, and one or more image / video applications 146. The image / video source 122 of the source device 120 may include an image / video capture device, such as a camera, an image / video archive containing previously captured videos, or an image / video feed interface for receiving image / video data from a content provider. As a further alternative, the image / video source 122 may generate computer graphics-based data as a source image / video, or a combination of real-time images / videos, archived images / videos, and computer-generated images / videos. The captured image data, pre-captured image data, or computer-generated image data may be encoded by the image / video encoder 124. The encoded image data may then be output to the communication medium 160 by the output interface 126. The image / video encoder 124 encodes the input image data and outputs the encoded bit stream 162 via the output interface 126. The encoded bitstream 162 is transmitted over the communication medium 160 and received by the input interface 142. The image / video decoder 144 then decodes the encoded bitstream 162 to generate decoded data that can be utilized by the image / video application 146.

[0025] The image / video encoder 124 and the image / video decoder 144 may each be implemented as any of a variety of suitable encoder or decoder circuits, such as one or more microprocessors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), discrete logic, software, hardware, firmware, or any combination thereof. When encoding or decoding is partially implemented in software, the image / video encoder 124 or the image / video decoder 144 may store instructions for the software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform techniques consistent with the present disclosure. Each of the image / video encoder 124 or the image / video decoder 144 may be included in one or more encoders or decoders, any of which may be integrated as part of a combined encoder / decoder (CODEC) in a corresponding device.

[0026] The image / video encoder 124 and the image / video decoder 144 may operate according to any video coding standard, such as Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), Versatile Video Coding (VVC), AOMedia Video 1 (AV1), Joint Photographic Experts Group (JPEG), Moving Picture Experts Group (MPEG), etc. Alternatively, the image / video encoder 124 and the image / video decoder 144 may be custom devices that do not conform to existing standards. Although in Figure 1 Not shown, in some embodiments, the image / video encoder 124 and the image / video decoder 144 may each be integrated with an audio encoder and decoder, and may include appropriate MUX-DEMUX units or other hardware and software to handle the encoding of both audio and video in a common data stream or in separate data streams.

[0027] The output interface 126 may include any type of medium or device capable of transmitting the encoded bitstream 162 from the source device 120 to the target device 140. For example, the output interface 126 may include a transmitter or transceiver configured to transmit the encoded bitstream 162 directly from the source device 120 to the target device 140 in real time. The encoded bitstream 162 may be modulated according to a communication standard (e.g., a wireless communication protocol) and sent to the target device 140.

[0028] The communication medium 160 may include a transient medium, such as a wireless broadcast or a wired network transmission. For example, the communication medium 160 may include a radio frequency (RF) spectrum or one or more physical transmission lines (e.g., cables). The communication medium 160 may form a portion of a packet-based network (e.g., a local area network, a wide area network, or a global network such as the Internet). In some embodiments, the communication medium 160 may include a router, a switch, a base station, or any other equipment that can be used to facilitate communication from the source device 120 to the target device 140. For example, a network server (not shown) may receive a coded bit stream 162 from the source device 120, and provide the coded bit stream 162 to the target device 140, for example, via a network transmission.

[0029] Communication medium 160 may also be in the form of a storage medium (e.g., a non-transitory storage medium) such as a hard disk, a flash drive, a compact disk, a digital video disk, a Blu-ray disk, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded image data. In some embodiments, a computing device at a media production facility (e.g., a disc stamping facility) may receive the encoded image data from source device 120 and produce a disc containing the encoded video data.

[0030] Input interface 142 may include any type of medium or device capable of receiving information from communication medium 160. The received information includes encoded bitstream 162. For example, input interface 142 may include a receiver or transceiver configured to receive encoded bitstream 162 in real-time.

[0031] The image / video application 146 includes various hardware and / or software for utilizing the decoded image data generated by the image / video decoder 144. For example, the image / video application 146 may include a display device that displays the decoded image data to a user, and may include any of a variety of display devices, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device. As another example, the image / video application 146 may include one or more processors configured to use the decoded image data to perform various machine vision applications, such as object recognition and tracking, facial recognition, image matching, image / video search, augmented reality, robotic vision and navigation, autonomous driving, three-dimensional structure construction, stereo correspondence, motion tracking, etc.

[0032] As described above, the image / video encoder 124 and the image / video decoder 144 may or may not operate in accordance with a video coding standard. Figure 2 and Figure 3 An exemplary encoding process that may be performed by the image / video encoder 124 and an exemplary decoding process that may be performed by the image / video decoder 144 are described.

[0033] Figure 2 2 is a block diagram illustrating an exemplary encoding process 200 according to some embodiments of the present disclosure. The encoding process 200 may be performed by an encoder, such as the image / video encoder 124 ( Figure 1 ), execute. Figure 2The encoding process 200 includes a picture partitioning stage 210, an inter-frame prediction stage 220, an intra-frame prediction stage 225, a transform stage 230, a quantization stage 235, a rearrangement stage 260, an entropy encoding stage 265, an inverse quantization stage 240, an inverse transform stage 245, a filtering stage 250 and a buffering stage 255.

[0034] The picture partitioning stage 210 partitions the input picture 205 into at least one processing unit. A processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture partitioning stage 210 partitions the picture into a combination of a plurality of coding units, prediction units, and transform units. The picture may be encoded by selecting a combination of coding units, prediction units, and transform units based on a predetermined criterion (e.g., a cost function).

[0035] For example, a picture can be divided into multiple coding units. In order to divide the coding units in the picture, a recursive tree structure, such as a quadtree structure, can be used. Specifically, using the recursive tree structure, the picture serves the root node and is recursively divided into smaller areas, i.e., child nodes. Coding units that are no longer divided according to predetermined restrictions become leaf nodes. For example, when it is assumed that only square division is possible for the coding unit, the coding unit is divided into up to four different coding units.

[0036] It can be determined whether to use inter-frame prediction or perform intra-frame prediction for the prediction unit, and specific information (e.g., intra-frame prediction mode, motion vector, reference picture, etc.) can be determined according to the prediction method. The processing unit used to perform the prediction may be different from the processing unit used to determine the prediction method and specific content. For example, the prediction method and prediction mode can be determined in the prediction unit, and the prediction can be performed in the transform unit. The residual coefficient (residual block) between the reconstructed prediction block and the original block can be input into the transform stage 230. In addition, the prediction mode information, motion vector information, etc. used for prediction can be encoded by the entropy encoding stage 265 together with the residual coefficient and transmitted to the decoder. In some embodiments, the original block can be encoded and transmitted to the decoder without generating the prediction block through the inter-frame prediction stage 220 and the intra-frame prediction stage 225.

[0037] The inter-frame prediction stage 220 predicts a prediction unit based on information in at least one picture before or after the current picture. In some embodiments, the inter-frame prediction stage 220 predicts a prediction unit based on information about a partial area that has been encoded in the current picture. The inter-frame prediction stage 220 also includes a reference picture interpolation stage, a motion prediction stage, and a motion compensation stage (not shown).

[0038] The reference picture interpolation stage receives reference picture information from the buffer stage 255 and generates pixel information of an integer number of pixels or less from the reference picture. In the case of luma pixels, an 8-tap interpolation filter based on DCT with variable filter coefficients is used to generate pixel information of an integer number of pixels or less in units of 1 / 4 pixels. In the case of color difference signals, a 4-tap interpolation filter based on DCT with variable filter coefficients is used to generate pixel information of an integer number of pixels or less in units of 1 / 8 pixels.

[0039] The motion prediction stage performs motion prediction based on the reference picture interpolated by the reference picture interpolation stage. Various methods such as the full search-based block matching algorithm (FBMA), the three-step search (TSS) and the new three-step search (NTS) algorithm can be used as methods for calculating motion vectors. The motion vector may have a motion vector value based on 1 / 2 or 1 / 4 pixels of the interpolated pixel. The motion prediction stage predicts the current prediction unit based on the motion prediction mode. Various methods such as skip mode, merge mode, advanced motion vector prediction (AMVP) mode, intra-block copy mode, etc. can be used as motion prediction modes.

[0040] The intra prediction stage 225 generates a prediction unit based on the information of the reference pixel in the neighborhood of the current block, which is the pixel information in the current picture. When the block in the neighborhood of the current prediction unit is a block on which inter-frame prediction has been performed, and thus the reference pixel is a pixel on which inter-frame prediction has been performed, the reference pixel included in the block on which inter-frame prediction has been performed can be used to replace the reference pixel information of the block in the neighborhood on which intra-frame prediction has been performed. That is, when the reference pixel is unavailable, at least one reference pixel among the available reference pixels can be used to replace the unavailable reference pixel information.

[0041] In intra prediction, the prediction mode may have an angular prediction mode using reference pixel information according to a prediction direction and a non-angular prediction mode that does not use direction information when performing prediction. The mode for predicting luminance information may be different from the mode for predicting chrominance information, and chrominance information may be predicted using intra prediction mode information for predicting luminance information or predicted luminance signal information.

[0042] If the size of the prediction unit is the same as the size of the transform unit when performing intra prediction, intra prediction can be performed on the prediction unit based on the left pixel, the upper left pixel, and the top pixel of the prediction unit. If the size of the prediction unit is different from the size of the transform unit when performing intra prediction, intra prediction can be performed using reference pixels based on the transform unit.

[0043] The intra prediction method generates a prediction block after applying an adaptive intra smoothing (AIS) filter to reference pixels according to a prediction mode. The type of the AIS filter applied to the reference pixels may vary. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted based on the intra prediction mode of the prediction unit present in the neighborhood of the current prediction unit. When the prediction mode of the current prediction unit is predicted using the mode information predicted from the neighboring prediction unit, the following method may be performed. If the intra prediction mode of the current prediction unit is the same as the intra prediction mode of the prediction unit in the neighborhood, predetermined flag information may be used to transmit information indicating that the prediction mode of the current prediction unit is the same as the prediction mode of the prediction unit in the neighborhood. If the prediction modes of the current prediction unit and the prediction unit in the neighborhood are different from each other, the prediction mode information of the current block may be encoded by performing entropy coding.

[0044] In addition, a residual block is generated, which is the difference between the prediction unit and the original block. The generated residual block is input to the transform stage 230.

[0045] The transform stage 230 transforms the residual block using a transform method such as a discrete cosine transform (DCT) or a discrete sine transform (DST). The DCT transform core includes at least one of DCT2 and DCT8, and the DST transform core includes DST7. Whether to apply DCT or DST to transform the residual block can be determined based on the intra-frame prediction mode information of the prediction unit used to generate the residual block. The transform of the residual block can be skipped. A flag indicating whether to skip the transform of the residual block can be encoded. For residual blocks having a size less than or equal to a threshold, a luminance component, or a chrominance component in a 4:4:4 format, transform skipping can be allowed.

[0046] The quantization stage 235 quantizes the values ​​transformed into the frequency domain by the transform stage 230. The quantization coefficients may vary depending on the importance or block of the video. The values ​​calculated by the quantization stage 235 are provided to the inverse quantization stage 240 and the rearrangement stage 260.

[0047] The rearrangement stage 260 rearranges the values ​​of the quantized residual coefficients. The rearrangement stage 260 changes the coefficients of the two-dimensional block shape into a one-dimensional vector shape by a coefficient scanning method. For example, the rearrangement stage 260 can use a zigzag scanning method to scan the DC coefficients until the high-frequency domain coefficients, and change the coefficients into a one-dimensional vector shape. Depending on the size of the transform unit and the intra-frame prediction mode, the zigzag scan can be replaced by a vertical scan that scans the coefficients of the two-dimensional block shape in the column direction and a horizontal scan that scans the coefficients of the two-dimensional block shape in the row direction. In some embodiments, depending on the size of the transform unit and the intra-frame prediction mode, the scanning method to be used can be determined among zigzag scanning, vertical scanning, and horizontal scanning.

[0048] The entropy coding stage 265 performs entropy coding based on the values ​​determined by the rearrangement stage 260. The entropy coding may use various coding methods, such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), Context-Adaptive Binary Arithmetic Coding (CABAC), etc.

[0049] The entropy coding stage 265 encodes various information such as residual coefficient information and block type information, prediction mode information, partition unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information of the coding unit input from the rearrangement stage 260, the inter-frame prediction stage 220, and the intra-frame prediction stage 225. The entropy coding stage 265 can also entropy encode the coefficient values ​​of the coding unit input from the rearrangement stage 260. The output of the entropy coding stage 265 forms a coded bit stream 270, which can be sent to a decoding device.

[0050] The inverse quantization stage 240 and the inverse transform stage 245 inverse quantize the values ​​quantized by the quantization stage 235 and inverse transform the values ​​transformed by the transform stage 230. The residual coefficients generated by the inverse quantization stage 240 and the inverse transform stage 245 may be combined with a prediction unit predicted by motion estimation, motion compensation, and / or intra prediction to generate a reconstructed block.

[0051] The filtering stage 250 includes at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0052] The deblocking filter removes block distortion caused by the boundaries between blocks in the reconstructed picture. In order to determine whether to perform deblocking, information of pixels in multiple columns or rows included in the block can be used. When the deblocking filter is applied to the block, a strong filter or a weak filter can be applied according to the required deblocking filter strength. In addition, when vertical filtering and horizontal filtering are performed when applying the deblocking filter, the horizontal filtering and the vertical filtering can be processed in parallel.

[0053] The offset correction unit corrects the offset relative to the original video in units of pixels for the video on which deblocking has been performed. The offset correction for a specific picture can be performed by dividing the pixels included in the video into a certain number of regions, determining the region to perform the offset, and applying the offset to the region. Alternatively, the offset correction can be performed by applying the offset in consideration of edge information of each pixel.

[0054] Adaptive loop filtering is performed based on a value obtained by comparing the reconstructed and filtered video with the original video. After the pixels included in the video are divided into predetermined groups, a filter to be applied to the corresponding group is determined, and filtering can be performed differently for each group. Luma signaling can be sent for each coding unit, which is information related to whether ALF is applied, and the shape and filter coefficient of the ALF filter to be applied vary according to each block. In some implementations, the same type (fixed type) of ALF filtering is applied regardless of the characteristics of the block to be applied.

[0055] The buffer stage 255 temporarily stores the reconstructed block or the reconstructed picture calculated by the filtering stage 250, and provides the reconstructed block or the reconstructed picture to the inter prediction stage 220 when inter prediction is performed.

[0056] Figure 3 300 is a block diagram illustrating an exemplary decoding process 300 according to some embodiments of the present disclosure. The decoding process 300 may be performed by a decoder, such as the image / video decoder 144 ( Figure 1 ). refer to Figure 3 The decoding process 300 includes an entropy decoding stage 310, a rearrangement stage 315, an inverse quantization stage 320, an inverse transformation stage 325, an inter-frame prediction stage 330, an intra-frame prediction stage 335, a filtering stage 340 and a buffering stage 345.

[0057] Specifically, the input bit stream 302 is processed by an encoding process, such as the encoding process 200 ( Figure 2 ), is decoded in the reverse process. Specifically, Figure 3As shown, the entropy decoding stage 310 can perform entropy decoding in a process opposite to the process performed in the entropy encoding stage. For example, the entropy decoding stage 310 can use various methods corresponding to the methods performed in the entropy encoding stage, such as Exponential Golomb, context adaptive variable length coding (CAVLC), and context adaptive binary arithmetic coding (CABAC). The entropy decoding stage 310 decodes information related to intra-frame prediction and inter-frame prediction.

[0058] The rearrangement stage 315 performs rearrangement on the output of the entropy decoding stage 310 based on the rearrangement method used in the corresponding encoding process. The coefficients represented in the one-dimensional vector shape are reconstructed and rearranged into coefficients in the two-dimensional block shape. The rearrangement stage 315 receives information about the coefficient scanning performed in the corresponding encoding process and performs reconstruction by a method of reverse scanning based on the scanning order performed in the corresponding encoding process.

[0059] The inverse quantization stage 320 performs inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the rearranged block.

[0060] The inverse transform stage 325 performs inverse DCT or inverse DST. The DCT transform core may include at least one of DCT2 and DCT8, and the DST transform core may include DST7. Alternatively, when the transform is skipped in the corresponding encoding process, the inverse transform stage 325 may be skipped. The inverse transform may be performed based on the transmission unit determined in the corresponding encoding process. The inverse transform stage 325 may selectively perform a transform technique (e.g., DCT or DST) according to multiple pieces of information such as a prediction method, a size of a current block, a prediction direction, etc.

[0061] The inter prediction stage 330 and / or the intra prediction stage 335 generate a prediction block based on information related to the generation of the prediction block (provided by the entropy decoding stage 310) and information of previously decoded blocks or pictures (provided by the buffer stage 345). The information related to the generation of the prediction block may include intra prediction mode, motion vector, reference picture, etc.

[0062] The prediction type selection process may be used to determine whether to use one or both of the inter prediction stage 330 and the intra prediction stage 335. The prediction type selection process receives various information, such as prediction unit information input from the entropy decoding stage 310, prediction mode information of the intra prediction method, motion prediction related information of the inter prediction method, etc. The prediction type selection process identifies a prediction unit from the current coding unit and determines whether the prediction unit performs inter prediction or intra prediction. The inter prediction stage 330 uses information required for inter prediction of the current prediction unit to perform inter prediction on the current prediction unit based on information included in at least one picture before or after the current picture including the current prediction unit. In some embodiments, the inter prediction stage 330 may perform inter prediction based on information about a previously reconstructed partial area in the current picture including the current prediction unit. In order to perform inter prediction, it is determined based on the coding unit whether the motion prediction method of the prediction unit included in the corresponding coding unit is a skip mode, a merge mode, a motion vector prediction mode (AMVP mode), or an intra-block copy mode.

[0063] The intra prediction stage 335 generates a prediction block based on information about pixels in the current picture. When the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on the intra prediction mode information of the prediction unit. The intra prediction stage 335 may include adaptive intra smoothing (AIS) filtering, a reference pixel interpolation stage, and DC filtering. AIS filtering is to perform filtering on the reference pixels of the current block, and whether to apply filtering and the stage of applying filtering can be determined according to the prediction mode of the current prediction unit. The prediction mode of the prediction unit used in the corresponding encoding process and the AIS filtering information can be used to perform AIS filtering on the reference pixels of the current block. When the prediction mode of the current block is a mode in which AIS filtering is not performed, AIS filtering may not be applied.

[0064] When the prediction mode of the prediction unit is intra prediction based on pixel values ​​obtained by interpolating reference pixels, the reference pixel interpolation stage generates reference pixels of pixel units having integer values ​​or less by interpolating reference pixels. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating reference pixels, the reference pixels are not interpolated. When the prediction mode of the current block is the DC mode, the DC filter generates a prediction block by filtering.

[0065] The reconstructed block or picture is provided to the filtering stage 340. The filtering stage 340 may include a deblocking filter, an offset correction unit, and an ALF.

[0066] Information about whether to apply a deblocking filter to a corresponding block or picture and information about whether a strong filter or a weak filter is applied when applying a deblocking filter may be provided by a corresponding encoding process. The deblocking filter at the filtering stage 340 may be provided with information related to the deblocking filter used in the corresponding encoding process.

[0067] The offset correction unit of the filtering stage 340 may perform offset correction on the reconstructed video based on offset correction type and offset value information that may be provided by the encoder.

[0068] Based on the information on whether to apply ALF and the information on the ALF coefficients provided by the encoder, the ALF of the filtering stage 340 is applied to the coding unit. The ALF information may be provided in a specific parameter set.

[0069] The buffer stage 345 stores the reconstructed picture or block as a reference picture or reference block and outputs it to downstream applications, such as Figure 1 Image / video applications in 146.

[0070] The above encoding process 200 ( Figure 2 ) and decoding process 300( Figure 3 ) are provided as examples to illustrate possible encoding and decoding processes consistent with the present disclosure, but are not meant to be limiting of these processes. For example, some embodiments use wavelet transforms to encode and decode image data rather than directly encoding pixels in blocks.

[0071] Figure 4 4 is a block diagram of an exemplary apparatus 400 for encoding or decoding image data according to some embodiments of the present disclosure. Figure 4As shown, the device 400 may include a processor 402. When the processor 402 executes the instructions described herein, the device 400 may become a dedicated machine for processing image data. The processor 402 may be any type of circuit capable of manipulating or processing information. For example, the processor 402 may include any number of central processing units (or "CPUs"), graphics processing units (or "GPUs"), neural processing units ("NPUs"), microcontroller units ("MCUs"), optical processors, programmable logic controllers, microcontrollers, microprocessors, digital signal processors, intellectual property (Intellectual Property, IP) cores, programmable logic arrays (PLA), programmable array logic (PAL), generic array logic (GAL), complex programmable logic devices (CPLD), field programmable gate arrays (FPGA), system on chip (SoC), application-specific integrated circuits (ASIC), etc. Any combination. In some embodiments, processor 402 may also be a group of processors grouped into a single logical component. Figure 4 As shown, processor 402 may include a plurality of processors, including processor 402a, processor 402b, ..., and processor 402n.

[0072] The apparatus 400 may also include a memory 404 configured to store data (eg, instruction sets, computer codes, intermediate data, etc.). Figure 4As shown, the stored data may include program instructions (e.g., program instructions for implementing processing of image data) and image data to be processed. The processor 402 may access the program instructions and data for processing (e.g., via the bus 410), and execute the program instructions to perform operations or manipulations on the data. The memory 404 may include a high-speed random access memory device or a non-volatile memory device. In some embodiments, the memory 404 may include any number of random access memories (Random-Access Memory, RAM), read-only memories (Read Only Memory, ROM), optical disks, magnetic disks, hard disk drives, solid-state drives, flash drives, secure digital (Security Digital, SD) cards, memory sticks, compact flash (Compact Flash, CF) cards, etc. Any combination. The memory 404 may also be a group of memories ( Figure 4 not shown).

[0073] The bus 410 may be a communication device for transmitting data between components inside the apparatus 400 , such as an internal bus (eg, a CPU memory bus), an external bus (eg, a universal serial bus port, a peripheral component interconnect high-speed port), and the like.

[0074] For ease of explanation and without ambiguity, in the present disclosure, the processor 402 and other data processing circuits (e.g., analog / digital converters, dedicated circuits, digital signal processors, interface circuits, Figure 4 The data processing circuits (not shown) are collectively referred to as “data processing circuits”. The data processing circuits may be implemented entirely as hardware, or as a combination of software, hardware, or firmware. In addition, the data processing circuits may be a single stand-alone module, or may be fully or partially combined into any other component of the device 400.

[0075] The device 400 may also include a network interface 406 to provide wired or wireless communication with a network (e.g., the Internet, an intranet, a local area network, a mobile communication network, etc.). In some embodiments, the network interface 406 may include any number of network interface controllers (NICs), radio frequency (RF) modules, transponders, transceivers, modems, routers, gateways, wired network adaptors, wireless network adaptors, Bluetooth adaptors, infrared adaptors, near-field communication (NFC) adaptors, cellular network chips, etc.

[0076] In some embodiments, the apparatus 400 may optionally further include a peripheral interface 408 to provide a connection to one or more peripheral devices. Figure 4 As shown, peripheral devices may include, but are not limited to, a cursor control device (e.g., a mouse, touchpad, or touch screen), a keyboard, a display (e.g., a cathode ray tube display, a liquid crystal display, or a light emitting diode display), an image / video input device (e.g., a camera or input interface coupled to an image / video archive), and the like.

[0077] Consistent with the disclosed embodiments, the encoding and decoding processes may be optimized to further remove redundancy in the image data. For example, redundancy may exist downstream of the decoder (e.g., image / video application 146 ( Figure 1 )) of the machine vision task. This redundancy can be skipped from encoding and transmission without degrading the performance of the machine vision task. This is useful, for example, in low bitrate situations where the amount of image data that can be transmitted from the encoder to the decoder per time unit is limited.

[0078] In some embodiments, the redundancy is related to the size of the input image (including the size of the object shown in the input image). As used herein, "object" refers to the foreground depicted in the image, also referred to as "object representation" or "object image" to distinguish it from physical objects existing in the real world. In addition, "object size" as used herein refers to the image size (i.e., pixel size) of the foreground as shown in the image, rather than the actual size of the physical object. Consistent with the present disclosure, the performance of machine vision tasks (e.g., object recognition and tracking) can depend on the size and / or resolution (defined as the number of pixels per inch of image display) of the object representation analyzed by the machine vision task. For example, a convolutional neural network (CNN) can be trained to detect a specific type of object (e.g., face) of a given size (e.g., 64×128 pixels) from an image. Therefore, such a trained CNN is able to produce the highest accuracy when detecting objects of the same or similar size. Larger object sizes do not necessarily improve the accuracy of object detection, but may instead result in reduced accuracy. Therefore, the extra bits required to represent object sizes larger than 64×128 pixels are not necessary for machine vision tasks and can be removed from the encoding.

[0079] In order to exploit the above-mentioned redundancy associated with the size of the object, the object representation may be downsized before encoding it in order to reduce the number of bits that need to be encoded and transmitted. Figure 5 5 is a block diagram illustrating a process 500 for controlling encoding of image data using object size modification information according to some embodiments consistent with the present disclosure. The process 500 may be performed by an encoder (image / video encoder 124 ( Figure 1 ))Execute. Figure 5 As shown, the process 500 begins with the encoder receiving input image data 510 including one or more image frames. Before encoding, the encoder performs analysis 520 on the input image data 510 to generate control information 530 for the encoding process 540. The analysis 520 may include, for example, detecting objects from the input frames, and determining the optimal size of the object relative to a given machine vision task (not shown) performed downstream at a corresponding decoder (not shown). The machine vision task may analyze image data associated with the object and perform certain operations associated with the object, such as detection, tracking, shaping, etc. The optimal size of the object is the size that allows the machine vision task to analyze and operate the object with maximum accuracy. Based on the results of the analysis 520, the encoder generates control information 530 to adjust the size of the object for the encoding process 540. Specifically, the size of the object may be suitable for reducing the number of bits used to represent the object to a degree that the performance of the machine vision task is not degraded. For example, when the object has an initial size that is larger than the determined optimal size, the control information 530 may instruct the encoding process 540 to downsample (i.e., reduce) the pixel information associated with the object. As another example, when the object has an initial size that is smaller than the determined optimal size, the control information 530 may instruct the encoding process 540 to encode the pixel information associated with the object at its original size. Alternatively, the control information 530 may instruct the encoding process 540 to upsample (i.e., enlarge) the pixel information associated with the object.

[0080] In the case of shrinking the object, the number of bits to be encoded by the encoding process 540 is reduced, and thus a higher degree of compression of the image data can be achieved. The encoding process 540 can be any suitable encoding process consistent with the present disclosure. For example, the encoding process 540 can be implemented as the encoding process 200 ( Figure 2 ) and outputs a bitstream 550 including encoded pixel information associated with the rescaled object.

[0081] Upsampling can be used in situations where the size of an object after image compression becomes too small and cannot be effectively recognized by downstream machine vision applications. To address this problem, the object can be enlarged before image compression.

[0082] Consistent with the disclosed embodiments, both downsampling and upsampling may be used to pre-process the same input image before compressing it. Specifically, some objects in the input image may be downsampled, and other objects in the input image may be upsampled. The downsampling and / or upsampling may be codec-agnostic. For example, the control of a standard codec like AVC, HEC, and VVC may not be aware of, for example, the logical content of objects and background, since no additional information needs to be transmitted to control the absolute value of the quantization step.

[0083] Next, exemplary encoding and decoding methods consistent with process 500 will be described in detail.

[0084] Figure 6 6 is a flow chart of an exemplary method 600 for encoding image data according to some embodiments consistent with the present disclosure. The method 600 may be used to compress image data based on object size modification information. For example, the method 600 may be performed by an image data encoder (e.g., Figure 1 The image / video encoder 124 in FIG. Figure 6 As shown, method 600 includes the following steps 602-606.

[0085] At step 602, the encoder detects one or more objects from an input image. For example, Figure 7 An input image 710 is exemplarily shown, and the input image shows two objects 712 and 714. The encoder may detect each of the objects 712 and 714 from the input image 710, and determine the rest of the input image 710 as a background 716.

[0086] In some embodiments, detecting one or more objects from an input image includes three stages: image segmentation (i.e., object separation or identification), feature extraction, and object classification. In the image segmentation stage, the encoder can execute an image segmentation algorithm to divide the input image into multiple fragments, such as multiple groups of pixels, each group of pixels representing a portion of the input image. The image segmentation algorithm can assign a label (i.e., a category) to each group of pixels so that pixels with the same label share certain common characteristics. The encoder can then group the pixels according to their labels and designate each group as a different object.

[0087] After the input image is segmented, that is, after at least one object is identified, the encoder can extract features from the input image, which are characteristics of the object. Features can be based on the size of the object or the shape of the object, such as the area of ​​the object, the length and width of the object, the distance around the perimeter of the object, the rectangularity of the object shape, the circularity of the object shape. The features associated with each object can be combined to form a feature vector.

[0088] In the classification stage, the encoder can classify each identified object based on the feature vector associated with the corresponding object. The feature value can be regarded as the coordinates of a point in an N-dimensional space (one feature value means a one-dimensional space, two features mean a two-dimensional space, and so on), and classification is the processing of the subspace of the feature space to which the feature vector belongs. Since each subspace corresponds to a different object, the classification completes the task of object recognition. For example, the encoder can classify the identified object into one of a plurality of predetermined categories, such as a person, a face, an animal, a car, or a plant. In some embodiments, the encoder can perform classification based on a model. The model can be trained using a pre-labeled training data set associated with a known object. In some embodiments, the encoder can perform classification using machine learning techniques (e.g., convolutional neural networks (CNNs), deep neural networks (DNNs), etc.).

[0089] Still reference Figure 6 , at step 604, the encoder determines a size modification parameter for each of the one or more objects. Specifically, the object size may affect the accuracy of the machine vision task. For example, when a neural network is trained to recognize a face of size 224×224 pixels, performing reasoning on an object of the training size (i.e., 224×224 pixels) generally results in the highest accuracy. Therefore, each object category has a corresponding target object size that can maximize the accuracy of the machine vision task when analyzing objects belonging to a particular category. For example, the following Tables 1 to 3 show possible bitrate gains achieved by downscaling various object categories. The bitrate gain in Table 1 is measured by the detection quality with respect to the score parameters from the neural network for the machine vision task. The bitrate gain in Table 2 is measured by the detection quality with respect to the best obtained Intersection over Union (IoU). The bitrate gain in Table 3 is measured by the detection quality of the matching classification with respect to the threshold of IoU equal to 0.5. In Tables 1 to 3, the values ​​ranging from 0 to 1 indicate the ratio of the downsized object size to its original size. As shown in Tables 1 to 3, each object category has a specific size that can maximize the bit rate gain.

[0090] Table 1: Relationship between detection quality and object size with respect to scoring parameters from neural networks

[0091]

[0092]

[0093] Table 2: Detection quality and object size with respect to the best achieved IoU.

[0094]

[0095]

[0096] Table 3: Relationship between detection quality and object size for matching classification with IoU threshold equal to 0.5

[0097]

[0098]

[0099] Based on the relationship between the object category and its corresponding target object size, the encoder can determine the target object size for each of the one or more objects detected from the input image. The encoder can further determine a size modification parameter for each detected object to adjust its size to the corresponding target size. For example, if the initial size of a first object in the input image is larger than the target size of the first object, the size modification parameter associated with the first object can specify a ratio for reducing the first object to the target size of the first object. If the initial size of a second object in the input image is smaller than the target size of the second object, the size modification parameter associated with the second object can specify a ratio for enlarging the second object to the target size of the second object. If the initial size of a third object in the input image is the same as the target size of the third object, the size modification parameter associated with the third object can specify a 1:1 ratio for keeping the third object at the initial size of the third object.

[0100] In step 606, the encoder compresses the input image based on the determined size modification parameters. Specifically, before encoding, the encoder can modify the image by rescaling each of the one or more detected objects based on the corresponding size modification parameters of the one or more detected objects. In some embodiments, the encoder can perform rescaling by resampling the pixel information associated with each of the one or more detected objects. In particular, in the case where the target size of the object is less than the initial size of the object, the encoder can downsample the pixel information associated with the object. For example, the size modification parameter associated with the object can be a signal extraction parameter that specifies a pattern or frequency (e.g., one pixel is removed for every ten pixels) of pixels associated with the object. The modified input image with the downsampled object is encoded in a bitstream output by the encoder. In some embodiments, the size modification parameter associated with the object can be included in the encoded bitstream so that a decoder receiving the bitstream can directly use the size modification parameter to scale the object back to the original size of the object.

[0101] Figure 7An example of compressing an input image based on the determined size modification parameters is shown. As described above, the encoder detects object 712, object 714, and background 716 from input image 710. Object 712 and object 714 may have the same or different size modification parameters. The encoder may reduce object 712 to object 722 according to the size modification parameters associated with object 712, and reduce object 714 to object 724 according to the size modification parameters associated with object 714. In addition, the encoder may reduce background 716 to background 726, so that object 722, object 724, and background 726 together form image 720 scaled down from input image 710. Image 720 is then encoded by the encoder to generate an encoded bitstream.

[0102] Figure 8 6 is a block diagram illustrating an exemplary implementation of method 600 consistent with an embodiment of the present disclosure. Figure 8 As shown, input image data 802 includes one or more input images. For each input image, the encoder performs object detection 810 on the input image to detect one or more initial object images 811a, 811b, ..., 811n. Then, the encoder determines size modification information 812 associated with each initial object image 811. The size modification information 812 may include size modification parameters determined based on the object category associated with the corresponding initial object image 811. Based on the size modification information 812, the encoder modifies the input image by performing size modification 813 (e.g., reduction) on the corresponding initial object image 811 to generate a corresponding rescaled object image 814, and encodes the rescaled object image 814 (step 815) to generate an object bitstream 816. In some embodiments, multiple rescaled object images 814a-814n derived from the same input image can be separately encoded (815a-815n) to generate multiple bitstreams 816a-816n respectively. In some embodiments, rather than encoding multiple rescaled object images 814a-814n separately, the modified input image may be encoded to generate a single bitstream ( Figure 8 not shown).

[0103] Still reference Figure 8, the encoder also performs background extraction 820 on each input image to extract an initial background image 821. The initial background image 821 can be extracted by subtracting the initial object images 810a, 810b, ..., 810n from the input image. The encoder can determine size modification information 822 associated with the initial background image 821. The size modification information 822 can be determined based on the size modification information 812 associated with the initial object image 811. For example, the size modification information 822 may include a size modification parameter that is set equal to a size modification parameter associated with one of the initial object images 811a, 811b, ..., 811n detected from the same input image. The encoder uses the size modification information 822 to perform a size modification 823 (e.g., downscaling) on ​​the initial background image 821 to generate a rescaled background 824, and then encodes the rescaled background 824 (step 825) to generate a background bitstream 826. As Figure 8 As shown, the rescaled background image 824 may be encoded in parallel with the encoding 815 of the rescaled object image 814 to generate a separate bitstream. Alternatively, the rescaled background image 824 and one or more rescaled object images 814 may be encoded to generate a single bitstream ( Figure 8 not shown).

[0104] In some embodiments, the resizing information 812 , 822 may be encoded into the object bitstream 816 and the background bitstream 826 , such that the resizing information 812 , 822 may be signaled to a decoder and used by the decoder to reconstruct the original object image 811 and the original background 821 .

[0105] In some embodiments, the encoder may encode the rescaled object image 814 and the rescaled background image 824 in a single atlas image. Fig. 9 An atlas image 900 is exemplarily shown, which includes a set of rescaled object images 901, 904, ..., 912 and a rescaled background image 920. Consistent with the disclosed embodiments, the rescaled object images and rescaled background images combined in the atlas do not necessarily come from the same input image, but may come from multiple different input images.

[0106] On the decoder side, the bitstream output from the encoder can be decoded and used to restore objects and background in their original sizes. Fig.10 1 is a flow chart of an exemplary method 1000 for decoding a bitstream according to some embodiments consistent with the present disclosure. For example, the method 1000 may be performed by an image data decoder (e.g., Figure 1 The image / video decoder 144 in FIG. Fig.10As shown, method 1000 includes the following steps 1002-1008.

[0107] At step 1002, a decoder receives one or more bitstreams including image data representing one or more rescaled objects and a rescaled background.

[0108] At step 1004, the decoder decodes one or more bitstreams. For example, the decoder may use Figure 3 The entropy decoding 310 shown in FIG. 1 is used to decode one or more bitstreams.

[0109] Still reference Fig.10 In step 1006, the decoder determines the size modification information associated with one or more rescaled objects and the rescaled background, respectively. In some embodiments, the size modification information associated with each object or background may include a size modification parameter that specifies the rescaling ratio. In some embodiments, the size modification information is signaled in one or more bitstreams. Therefore, the decoder can extract the size modification information from the decoded bitstream. In some embodiments, the size modification information is not signaled in one or more bitstreams. The decoder can determine the object class associated with each rescaled object, and determine the corresponding size modification information based on the object class associated with the rescaled object and the target size associated with the object class. The target size can be obtained from a lookup table that specifies the relationship between the object class and the object size. The relationship can be signaled in one or more bitstreams, or the relationship can be predefined and stored in the decoder.

[0110] In step 1008, the decoder generates a reconstructed image including one or more objects and a background at their respective original sizes based on the size modification information. Specifically, the decoder restores the rescaled objects and the rescaled background to their original sizes by resampling (e.g., upsampling) the pixel information associated with the one or more objects and the background based on the size modification information. For example, if the size modification information indicates that the rescaled objects and the rescaled background are reduced from their original sizes, the decoder enlarges the rescaled objects and the rescaled background to their original sizes. In some embodiments, the size modification information associated with the rescaled object or the rescaled background may include a signal extraction parameter for generating the rescaled object or the rescaled background, and the decoder may interpolate or fill the pixel information associated with the rescaled object or the rescaled background based on the signal extraction parameter. For example, if the signal extraction parameter indicates that the rescaled object or the rescaled background is generated by removing one pixel for every ten pixels, the decoder may perform an interpolation or filling process to add new pixels to the object or background.

[0111] In some embodiments, the decoder may combine resampled pixel information associated with one or more objects and the background to generate a composite image. For example, the decoder may generate a composite image by pasting the restored object on the restored background. When two restored objects overlap each other, the decoder may overlap one restored object on top of the other restored object, or add the pixel values ​​of each pixel in the overlapping area.

[0112] In some embodiments, the decoder may omit steps 1006 and 1008. For example, when the downstream application is to perform a machine vision task, the decoder may not need to rescale the object and background back to their original size. The decoder may use the scaled object and the scaled background to reconstruct the image, and the size modification information may be omitted when the encoder encodes and transmits to the decoder. As described above, the encoder may shrink the object and background to improve the performance of the machine vision task, in which case the decoder may not need to rescale the object and background to their original size for the machine vision task.

[0113] The image data compression techniques described in this disclosure can be used in low bit rate situations. By performing a resizing of the input image, the overall bit rate used to transmit the image data can be reduced.

[0114] In some embodiments, a non-transitory computer-readable storage medium including instructions is also provided, and the instructions can be executed by a device for performing the above method (e.g., the disclosed encoder and decoder). Common forms of non-transitory media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, tapes or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with hole patterns, RAM, PROM and EPROM, FLASH-EPROM or any other flash memory, NVRAM, cache, registers, any other memory chip or cassette, and networked versions of the foregoing. The device may include one or more processors (CPUs), input / output interfaces, network interfaces, and / or memories.

[0115] It should be noted that relational terms such as "first" and "second" herein are used only to distinguish an entity or operation from another entity or operation, and do not require or imply any actual relationship or order between these entities or operations. In addition, the words "comprising", "having", "including", and "including" and other similar forms are intended to be equivalent in meaning and open-ended, in that one or more items following any of these words is not intended to be an exhaustive list of such one or more items, or to be limited to the one or more items listed.

[0116] As used herein, unless otherwise specifically stated, the term "or" encompasses all possible combinations unless not feasible. For example, if it is stated that a database may include A or B, then unless otherwise specifically stated or not feasible, the database may include A, or include B, or include A and B. As a second example, if it is stated that a database may include A, B, or C, then unless otherwise specifically stated or not feasible, the database may include A, or include B, or include C, or include A and B, or include A and C, or include B and C, or include A, B, and C.

[0117] It should be understood that the present invention is not limited to the exact construction described above and shown in the drawings and that various modifications and changes may be made without departing from the scope of the present invention, which should be limited only by the appended claims.

Claims

1. A computer-implemented encoding method comprising: detecting a first object from an input image; determining a first size modification parameter associated with the first object; as well as The input image is compressed based on the first resizing parameter.

2. The method according to claim 1, wherein: Detecting the first object from the input image comprises determining an object class associated with the first object; and Determining the first size modification parameter includes determining the first size modification parameter based on: the object class associated with the first object, and The target size associated with the object category.

3. The method according to claim 1 or 2, further comprising: A bitstream is output, the bitstream including information associated with the first size modification parameter.

4. The method according to any one of claims 1 to 3, wherein: Compressing the input image based on the first size modification parameter comprises: modifying the input image by resampling pixel information associated with the first object based on the first resizing parameter; and Compress the modified input image.

5. The method according to claim 4, wherein: The first resizing parameter includes a signal decimation parameter, and resampling the pixel information associated with the first object includes: The pixel information associated with the first object is extracted based on the signal extraction parameters.

6. The method according to any one of claims 1 to 5, further comprising: Extracting the background of the input image; as well as Prior to compressing the input image, pixel information associated with the background is resampled.

7. The method according to claim 6, further comprising: outputting a first bitstream, the first bitstream comprising compressed pixel information associated with the first object, and A second bitstream is output, the second bitstream including compressed pixel information associated with the background.

8. The method according to any one of claims 1 to 7, further comprising: detecting a second object from the input image; determining a second size modification parameter associated with the second object; as well as Prior to compressing the input image, pixel information associated with the second object is resampled based on the second resizing parameter.

9. The method according to claim 8, further comprising: outputting a first bitstream, the first bitstream comprising compressed pixel information associated with the first object, and A second bitstream is output, the second bitstream including compressed pixel information associated with the second object.

10. The method according to any one of claims 1 to 9, wherein: Compressing the input image comprises: detecting one or more objects from one or more input images; extracting a background of at least one of the one or more input images; determining size modification parameters respectively associated with the one or more objects; resampling pixel information associated with the one or more objects and the background based on the resizing parameters; and An encoded image representing the resampled one or more objects and the resampled background is generated.

11. A device comprising: a memory for storing instructions; as well as One or more processors are configured to execute the instructions to enable the device to perform the following operations, the operations comprising: detecting a first object from an input image; determining a first size modification parameter associated with the first object; and The input image is compressed based on the first resizing parameter.

12. The device according to claim 11, wherein Detecting the first object from the input image comprises determining an object class associated with the first object; and Determining the first size modification parameter includes determining the first size modification parameter based on: the object class associated with the first object, and The target size associated with the object category.

13. The device according to claim 11 or 12, wherein: The operations also include: A bitstream is output, the bitstream including information associated with the first size modification parameter.

14. The device according to any one of claims 11 to 13, wherein: Compressing the input image based on the first size modification parameter comprises: modifying the input image by resampling pixel information associated with the first object based on the first resizing parameter; and Compress the modified input image.

15. The device according to claim 14, wherein: The first resizing parameter includes a signal decimation parameter, and resampling the pixel information associated with the first object includes: The pixel information associated with the first object is extracted based on the signal extraction parameters.

16. The device according to any one of claims 11 to 15, wherein: The operations also include: extracting the background of the input image; and Prior to compressing the input image, pixel information associated with the background is resampled.

17. The apparatus according to claim 16, wherein: The operations also include: outputting a first bitstream, the first bitstream comprising compressed pixel information associated with the first object, and A second bitstream is output, the second bitstream including compressed pixel information associated with the background.

18. The apparatus according to any one of claims 11 to 17, wherein: The operations also include: detecting a second object from the input image; determining a second size modification parameter associated with the second object; and Prior to compressing the input image, pixel information associated with the second object is resampled based on the second resizing parameter.

19. The apparatus according to claim 18, wherein: The operations also include: outputting a first bitstream, the first bitstream comprising compressed pixel information associated with the first object, and A second bitstream is output, the second bitstream including compressed pixel information associated with the second object.

20. The apparatus according to any one of claims 11 to 19, wherein: Compressing the input image comprises: detecting one or more objects from one or more input images; extracting a background of at least one of the one or more input images; determining size modification parameters respectively associated with the one or more objects; resampling pixel information associated with the one or more objects and the background based on the resizing parameters; and An encoded image representing the resampled one or more objects and the resampled background is generated.

21. A non-transitory computer-readable medium storing a set of instructions, the set of instructions being executable by one or more processors of a device to cause the device to perform the following method, the method comprising: detecting a first object from an input image; determining a first size modification parameter associated with the first object; as well as The input image is compressed based on the first resizing parameter.

22. The non-transitory computer readable medium of claim 21, wherein: Detecting the first object from the input image comprises determining an object class associated with the first object; and Determining the first size modification parameter includes determining the first size modification parameter based on: the object class associated with the first object, and The target size associated with the object category.

23. The non-transitory computer readable medium of claim 21 or 22, wherein: The method further comprises: A bitstream is output, the bitstream including information associated with the first size modification parameter.

24. The non-transitory computer readable medium of any one of claims 21-23, wherein: Compressing the input image based on the first size modification parameter comprises: modifying the input image by resampling pixel information associated with the first object based on the first resizing parameter; and Compress the modified input image.

25. The non-transitory computer readable medium of claim 24, wherein: The first resizing parameter includes a signal decimation parameter, and resampling the pixel information associated with the first object includes: The pixel information associated with the first object is extracted based on the signal extraction parameters.

26. The non-transitory computer readable medium of any one of claims 21-25, wherein: The method further comprises: extracting the background of the input image; and Prior to compressing the input image, pixel information associated with the background is resampled.

27. The non-transitory computer readable medium of claim 26, wherein: The method further comprises: outputting a first bitstream, the first bitstream comprising compressed pixel information associated with the first object, and A second bitstream is output, the second bitstream including compressed pixel information associated with the background.

28. The non-transitory computer readable medium of any one of claims 21-27, wherein: The method further comprises: detecting a second object from the input image; determining a second size modification parameter associated with the second object; and Prior to compressing the input image, pixel information associated with the second object is resampled based on the second resizing parameter.

29. The non-transitory computer readable medium of claim 28, wherein: The method further comprises: outputting a first bitstream, the first bitstream comprising compressed pixel information associated with the first object, and A second bitstream is output, the second bitstream including compressed pixel information associated with the second object.

30. The non-transitory computer readable medium of any one of claims 21-29, wherein: Compressing the input image comprises: detecting one or more objects from one or more input images; extracting a background of at least one of the one or more input images; determining size modification parameters respectively associated with the one or more objects; resampling pixel information associated with the one or more objects and the background based on the resizing parameters; and An encoded image representing the resampled one or more objects and the resampled background is generated.

31. A computer-implemented decoding method comprising: receiving one or more bitstreams, the one or more bitstreams comprising image data representing a first object of a first size; decoding the one or more bitstreams; determining a first size modification parameter associated with the first object; and A reconstructed image is generated based on the first size modification parameter, the reconstructed image including the first object at a second size different from the first size.

32. The method according to claim 31, wherein: Determining the first size modification parameter associated with the first object includes: determining an object class associated with the first object; and The first size modification parameter is determined based on: the object class associated with the first object, and The target size associated with the object category.

33. The method of claim 32, wherein: The one or more bitstreams include information signaling a relationship between an object category and an object size, the method further comprising: The first size modification parameter is determined based on the signaled relationship between the object class and the object size.

34. The method of claim 32, further comprising: The first size modification parameter is determined based on a predetermined relationship between the object category and the object size.

35. The method of claim 31, wherein: The one or more bitstreams include information signaling the first size modification parameter.

36. The method according to any one of claims 31 to 35, wherein: Generating the reconstructed image including the first object of the second size comprises: Pixel information associated with the first object is resampled based on the first resizing parameter.

37. The method of claim 36, wherein: The first size modification parameters include signal decimation parameters, and generating the reconstructed image of the first object including the second size includes: The pixel information associated with the first object is interpolated based on the signal extraction parameters.

38. The method according to any one of claims 31-37, further comprising: extracting pixel information associated with the background from the decoded one or more bitstreams; resampling the pixel information associated with the background based on the first resizing parameter; as well as The reconstructed image is generated based on the resampled pixel information associated with the background.

39. The method according to any one of claims 31-38, further comprising: extracting pixel information associated with a plurality of objects from the decoded one or more bitstreams; determining a plurality of size modification parameters respectively associated with the plurality of objects; resampling the pixel information associated with the plurality of objects based on the plurality of resizing parameters; as well as The resampled pixel information is combined to generate a composite image.

40. An apparatus comprising: a memory for storing instructions; as well as One or more processors are configured to execute the instructions to enable the device to perform the following operations, the operations comprising: receiving one or more bitstreams, the one or more bitstreams comprising image data representing a first object of a first size; decoding the one or more bitstreams; determining a first size modification parameter associated with the first object; and A reconstructed image is generated, the reconstructed image including the first object at a second size different from the first size, the reconstruction being based on a modification parameter of the first size.

41. The apparatus of claim 40, wherein: Determining the first size modification parameter associated with the first object includes: determining an object class associated with the first object; and The first size modification parameter is determined based on: the object class associated with the first object, and The target size associated with the object category.

42. The apparatus of claim 41, wherein: The one or more bitstreams include information signaling a relationship between an object category and an object size, the operations further comprising: The first size modification parameter is determined based on the signaled relationship between the object class and the object size.

43. The apparatus of claim 41, wherein: The operations also include: The first size modification parameter is determined based on a predetermined relationship between the object category and the object size.

44. The apparatus of claim 40, wherein: The one or more bitstreams include information signaling the first size modification parameter.

45. The apparatus according to any one of claims 40 to 44, wherein: Generating the reconstructed image including the first object of the second size comprises: Pixel information associated with the first object is resampled based on the first resizing parameter.

46. ​​The apparatus of claim 45, wherein: The first size modification parameters include signal decimation parameters, and generating the reconstructed image of the first object including the second size includes: The pixel information associated with the first object is interpolated based on the signal extraction parameters.

47. The apparatus according to any one of claims 40 to 46, wherein: The operations also include: extracting pixel information associated with the background from the decoded one or more bitstreams; resampling the pixel information associated with the background based on the first resizing parameter; and The reconstructed image is generated based on the resampled pixel information associated with the background.

48. The apparatus according to any one of claims 40 to 47, wherein: The operations also include: extracting pixel information associated with a plurality of objects from the decoded one or more bitstreams; determining a plurality of size modification parameters respectively associated with the plurality of objects; resampling the pixel information associated with the plurality of objects based on the plurality of resizing parameters; and The resampled pixel information is combined to generate a composite image.

49. A non-transitory computer readable medium storing a set of instructions, the set of instructions being executable by one or more processors of a device to cause the device to perform the following method, the method comprising: receiving one or more bitstreams, the one or more bitstreams comprising image data representing a first object of a first size; decoding the one or more bitstreams; determining a first size modification parameter associated with the first object; and A reconstructed image is generated, the reconstructed image including the first object at a second size different from the first size, the reconstruction being based on the first size modification parameter.

50. The non-transitory computer readable medium of claim 49, wherein: Determining the first size modification parameter associated with the first object includes: determining an object class associated with the first object; and The first size modification parameter is determined based on: the object class associated with the first object, and The target size associated with the object category.

51. The non-transitory computer readable medium of claim 50, wherein: The one or more bitstreams include information signaling a relationship between an object category and an object size, the method further comprising: The first size modification parameter is determined based on the signaled relationship between the object class and the object size.

52. The non-transitory computer readable medium of claim 50, wherein: The method further comprises: The first size modification parameter is determined based on a predetermined relationship between the object category and the object size.

53. The non-transitory computer readable medium of claim 49, wherein: The one or more bitstreams include information signaling the first size modification parameter.

54. The non-transitory computer readable medium of any one of claims 49-53, wherein: Generating the reconstructed image including the first object of the second size comprises: Pixel information associated with the first object is resampled based on the first resizing parameter.

55. The non-transitory computer readable medium of claim 54, wherein: The first size modification parameters include signal decimation parameters, and generating the reconstructed image of the first object including the second size includes: The pixel information associated with the first object is interpolated based on the signal extraction parameters.

56. The non-transitory computer readable medium of any one of claims 49-55, wherein: The method further comprises: extracting pixel information associated with the background from the decoded one or more bitstreams; resampling the pixel information associated with the background based on the first resizing parameter; and The reconstructed image is generated based on the resampled pixel information associated with the background.

57. The non-transitory computer readable medium of any one of claims 49-56, wherein: The method further comprises: extracting pixel information associated with a plurality of objects from the decoded one or more bitstreams; determining a plurality of size modification parameters respectively associated with the plurality of objects; resampling the pixel information associated with the plurality of objects based on the plurality of resizing parameters; and The resampled pixel information is combined to generate a composite image.

Images