Systems and methods for region based package compression

Abstract

Systems and methods for video encoding and decoding using region encapsulation are provided. At an encoder, a region detection module receives a video frame for encoding, identifies regions of interest in the video frame, and generates a bounding box for each region of interest. A region extractor module obtains pixels within a bounding box from a video frame. A region encapsulation module receives the identified region of interest and arranges the bounding box within an encapsulation frame that substantially reduces data to be encoded outside the identified region of interest. A video encoder receives an encapsulated frame and generates an encoded bitstream according to the encapsulated frame. At a decoder, the encoded bitstream is decoded and parameters sufficient to place the region within the reconstructed frame are extracted. A reconstructed frame is generated that substantially maintains the spatial relationship and size of the region of interest in the original video frame.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to U.S. Provisional Application Serial No. 63 / 326,313, filed on April 1, 2022, entitled “Systems and Methods for Region Packing Based Compression,” the entire contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present application relates generally to video encoding and decoding, and more particularly to video encoding and decoding using object and / or region detection and packing at an encoder and region depacking at a decoder. Background Art

[0004] A video codec may include an electronic circuit or software that compresses or decompresses a digital video. It can convert an uncompressed video into a compressed format, and vice versa. In the case of video compression, a device that compresses a video (and / or performs some of its functions) may generally be referred to as an encoder, and a device that decompresses a video (and / or performs some of its functions) may be referred to as a decoder. The format of the compressed data may preferably conform to a standard video compression specification, such as HEVC, AV1, VVC, etc.

[0005] Although video content is generally considered for human use, there is a growing demand for videos in industrial and other environments where the content is evaluated by machines rather than humans. Recent trends in robotics, surveillance, monitoring, the Internet of Things, etc. have introduced use cases where a large portion of all images and videos recorded in the field are used only by machines and do not reach human eyes. These machines process images and videos with the goal of completing tasks such as object detection, object tracking, segmentation, event detection, etc. Recognizing that this trend is widespread and will only accelerate in the future, international standardization bodies have worked to standardize image and video encoding optimized primarily for machine use. For example, in addition to established standards such as compact descriptors for visual search and compact descriptors for video analysis, standards such as JPEG AI and video coding for machines have been initiated. Therefore, further improving the encoding and decoding of videos used for machine use and in hybrid systems where the video is used by both human viewers and machines is increasingly important in the field. As used herein, the term VCM refers broadly to video encoding and decoding for machine use, and although the disclosed systems and methods may be standard-compliant, the present disclosure is not limited to a particular proposed protocol or standard.

[0006] In many applications, such as surveillance systems with multiple cameras, smart transportation, smart city applications, and / or smart industrial applications, traditional video encoding may need to compress a large amount of video from the camera and transmit it over the network for machine use and human use. Subsequently, at the machine site, algorithms for feature extraction can be applied, typically using convolutional neural networks or deep learning techniques (including object detection, event action recognition, pose estimation, etc.).

[0007] Video and image analysis methods and applications typically attempt to detect and track specific classes of objects and regions of interest. In certain applications of machine use, tasks may depend only on specific objects or regions. The object classes and regions of interest in a video may depend on the tasks that the analysis engine or machine task system is expected to perform. In such cases, the video content can be compressed by identifying objects of interest in video frames and only transmitting information related to such objects and omitting other objects or regions of no interest. Further compression efficiencies can be achieved by encapsulating objects of interest identified in frames into continuous regions before video compression. Summary of the invention

[0008] Currently disclosed methods for compressing video and image data focus on compression that preserves objects in each frame. A general system using this approach detects one or more regions of interest or objects of interest in a video frame, packs the regions tightly into a frame, and discards regions of no interest. As used herein, the term region can refer to an area in an image that has common characteristics (e.g., color, texture, water, grass, sky, etc.) or includes a specific object of interest (e.g., cat, dog, person, car, etc.).

[0009] The compressed bitstream output by the encoder may include the region locations as well as the parameters needed at the receiver to place the regions into the correct locations in the decoded frames.

[0010] In one embodiment, a video encoder for compression using region packing according to the present disclosure may include a region detection module that receives a video frame for encoding, identifies a region of interest in the video frame based on a target task parameter, and generates a bounding box for the region of interest. A region extractor module may be coupled to the region detection module, and for each region of interest identified, the region extractor may obtain pixels within the bounding box from the video frame. The region packing module receives the identified region of interest and arranges the bounding box into a packed frame while substantially omitting data outside the identified region of interest in the frame. The video encoder receives the packed frame and generates an encoded bitstream based on the packed frame.

[0011] In some implementations, the bounding box is rectangular and the region detector module generates parameters representing the size and position of the bounding box, including coordinates of corners of the bounding box in the frame, a width parameter, and a height parameter.

[0012] A region detector may include one or more object detectors. A region detector may also detect a region including a region of color, texture, or other region characteristics or features.

[0013] A video decoder for decoding a video bitstream encoded using region packing is also provided. It includes a video decoder module that receives an encoded bitstream including at least one encoded region. A region depacketization module is coupled to the video decoder module and identifies parameters of a bounding box of the encoded region. A frame reconstruction module is provided that uses the parameters to locate and size the bounding box within a reconstructed frame and fill the bounding box with decoded pixels corresponding to the region.

[0014] These and other aspects and features of non-limiting embodiments of the present invention will become apparent to those skilled in the art from the following description of specific non-limiting embodiments of the present invention read in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a simplified block diagram showing components of a region encapsulation based video compression system.

[0016] Figure 2 is a simplified diagram showing an exemplary video frame having multiple objects therein.

[0017] Figure 3 is a simplified diagram. Figure 3 The "car" object and the "cat" object have been identified.

[0018] FIG. 4A to FIG. 4D are images showing an object of interest and various representations of the object of interest with different treatments of background pixels.

[0019] Figure 5A and Figure 5B Two examples of region encapsulation in which the objects in FIG. 4 are encapsulated are shown.

[0020] Fig. 6A and Figure 6B shows the decapsulated frame output from the decoder and used to reconstruct the decapsulated frame of FIG. Figure 5A An exemplary encapsulated frame (including an object of interest) of FIG.

[0021] Figure 7 is a simplified flow chart illustrating the process of depacketizing and reconstructing an image frame based on decoded packed frame data.

[0022] Figure 8 is a simplified block diagram further detailing an embodiment of a decoder according to the present disclosure.

[0023] The drawings are not necessarily drawn to scale and may be illustrated by phantom lines, diagrams, and partial views. In certain instances, details that are not necessary for understanding the embodiments or that render other details difficult to perceive may have been omitted. DETAILED DESCRIPTION

[0024] Figure 1 is a simplified block diagram showing components of a region encapsulation based video compression system including an encoder 100 , a transmission channel 105 for compressed video, and a receiver / decoder 110 .

[0025] In the encoder 100, the region detection module 115 takes at least one picture / frame as input and detects the region of interest in the picture. These regions can be different objects in the frame or parts of the picture with similar textures. In some embodiments, the region detector 115 can use two or more frames as input to identify regions with similar motion in the frame. The detection area can be a rectangle or any arbitrary shape. However, it should be understood that for efficient compression and packaging, the region can preferably be limited to a rectangular shape. In some embodiments, each region detected can correspond to an object, and in such a case, an object detector can be used to perform the function of the region detector 115.

[0026] The receiver system 110 may send target task parameters 120 to the region detector 115 to change the behavior of the region detection module 115. The target task parameters 120 may indicate the type of region that the region detection module 115 should identify and detect. The target task parameters 120 may also identify other region parameters, such as whether rectangular or arbitrarily shaped regions should be detected. Preferably, the receiver system 110 may dynamically request different types of regions or objects to be detected.

[0027] The region detection module 115 may be comprised of a plurality of detection systems that may be selected based on the target task parameters 120. For example, the region detection module 115 may select specialized detectors optimized for specific categories of objects, such as a first detector for human objects and a different detector for car objects. The region detection module 115 may be configured to detect specific color regions such as red regions or specific regions such as water or sky. In another example, the region detector may be configured to detect specific objects, such as backpacks. It should be understood that some region detection systems may be capable of detecting multiple types of objects.

[0028] The region detection module 115 can use the previously configured target task parameters 120 without the need for additional information from the receiver system. When the region is rectangular, the region detection module 115 generates a bounding box of the region of interest. The bounding box definition specifies the position, size, and shape of the bounding box. For example, the bounding box can be defined by the coordinates of the upper left corner of the box, the box width, and the box height. Any other protocol that allows the specification of position, size, and shape can also be used. For example, the coordinates of two diagonally opposite corners can define a rectangular bounding box. The bounding boxes of more than one region can overlap. In some cases, the entire range of the frame can be included in the detected region. In some cases, only a small portion of the input frame can be included in the detected region.

[0029] When outputting a region of arbitrary shape, a binary mask can be used to identify the region. For example, for each pixel of the image, a binary mask can be represented by 1 and 0, where a value of 1 indicates that the pixel belongs to the region of interest, and a value of 0 indicates that the pixel is not in the region of interest.

[0030] Figure 2 is an example of a sample frame with multiple objects in it. Region detection module 115 can be configured to identify all objects of interest or only a subset of objects of interest. In this case, there are five objects: white car 205, black car 210, black cat 215, white car 220, white car 225, and tree 230. Figure 3 As shown, a sample frame with multiple objects 305 (O2), 310 (O1), 315 (O3), 320 (O4), and 325 (O5) is detected by the detector using a car detector and a cat detector. Each object is defined by a bounding box with the (x, y) coordinates of the upper left corner, the width of the bounding box, and the height of the bounding box. For example, for object 310O1, (O1x, O1y) is the (x, y) coordinate of the upper left corner, and O1W is the width of the box and O1H is the height of the box. In this example, the tree object 330 is not detected and is not processed as a detected area.

[0031] The region detector in this example can be configured with a target task parameter set to detect at least cats and cars. It should be understood that these objects are merely exemplary and a variety of desired objects can be detected.

[0032] After being processed by the region detection module 115, the detected regions and / or objects may be applied to the region extraction module 125. Region extraction may be a separate functional element or may be combined with the region detection module 115 or the region encapsulation module 130. The region extraction module 125 uses the input image and bounding box as input data and extracts a sub-image corresponding to the detected region. When the region corresponds to one or more specific object categories, the extracted sub-image may have pixels in the bounding box that are not part of the detected object or region of interest. Such pixels are referred to as background pixels. Background pixels may be processed in three different ways: 1) replaced by black or another solid color pixel, 2) replaced by the average pixel value of all background pixels, and 3) left unchanged.

[0033] In some systems, background pixel information can help detect objects of interest at the receiver side and improve machine task performance at the receiver. FIG. 4A to FIG. 4D , where the penguin is the object of interest. Figure 4A An original image is shown, which includes many penguin objects. Figure 4B In , the areas outside the object are replaced with black pixels. Figure 4C In , the area outside the object is replaced with pixels having the average value of the background pixels in the object bounding box, and in Figure 4D In the , the area outside the object in the object bounding box remains unchanged.

[0034] Still refer to Figure 1 , the region packing module 130 extracts sub-images corresponding to each region and packs them into compact regions for compression. The detected regions are extracted and packed into compact regions and compressed using efficient video compression. Video compression can generally be performed using conventional compression methods, such as those adopted in known video codec standards such as VVC, AV1, HEVC, etc.

[0035] Area can be Figure 5A and Figure 5B The multiple arrangements shown in the package are Figure 5A and Figure 5B Two examples of region packing arrangements according to the present disclosure are shown. The arrangement of objects of interest 505, 510, 515, 520, and 525 may be selected to maximize the compression performance of the video encoder used. The region packing arrangement may be changed as part of the encoding process. Figure 5A In the example shown, placing the black cat 515a (object O3) above the black car 510a (object O1) produces the best compression. In each case, the tree object ( Figure 3 , 330) is not among the objects of interest and is not detected or included in the encapsulated frame.

[0036] Object parameters such as bounding boxes and object positions are required at the decoder to restore the position of the object in the reconstructed frame. The object list, bounding boxes, and object placement in the encapsulated frame are preferably included in the video bitstream header. An exemplary syntax of the frame region information header is shown in the table below.

[0037] The frame region information may be included in a header of the frame, such as a picture or slice header.

[0038]

[0039] Frame region information semantics

[0040] fri_num_regions_minus1 – the number of object regions in the packed frame

[0041] fri_object_x_pos – x coordinate of the object in the encapsulated frame

[0042] fri_object_y_pos – the y coordinate of the object in the encapsulated frame

[0043] fri_object_bbox_x_pos – x coordinate of the object in the reconstructed frame

[0044] fri_object_bbox_y_pos – y coordinate of the object in the reconstructed frame

[0045] fri_object_bbox_width – width of the object

[0046] fri_object_bbox_height – height of the object

[0047] The frame region information semantics can be extended to support more than 2 dimensions. For example, to support 3D video, the semantics will be extended with three additional parameters:

[0048] fri_object_z_pos – z coordinate of the object in the encapsulated frame

[0049] fri_object_bbox_z_pos – z coordinate of the object in the reconstructed frame

[0050] fri_object_bbox_depth – depth of the object

[0051] The video encoder 135 is suitable for encoding individual frames, or sequences of frames. An image encoder may also be used. Frames with packed regions are encoded with a compression efficiency suitable for the intended use at the receiver / decoder 140. The frame packing arrangement is typically determined as part of the encoding step. The encoder 135 receives the original frame and the region bounding box as input, and as part of the encoding process, the encoder 135 determines the region packing arrangement that maximizes the compression performance. The encoder 135 includes frame region information in the compressed video bitstream. The original video width and height are also encoded in the compressed video bitstream. In the case of three-dimensional video, a point cloud compression (PCC) encoder may be used instead of or in conjunction with the video encoder.

[0052] The corresponding video decoder 140 uses the compressed video bitstream as input and outputs the decoded region packing frame and frame region information. The original video width and height are also decoded from the video bitstream. The video decoder 140 can take the form of a known video decoder that conforms to the coding scheme used by the encoder 135, such as an encoder that conforms to the VVC, HEVC, AV1 standards, etc.

[0053] like Fig. 6A and Figure 6B As generally shown in FIG. 1 , the region decapsulation stage 145 receives from the video decoder 140 a packet object ( Fig. 6A ), frame region information, and a decoded frame of the original frame size as input, and reconstructs the frame so that objects / regions 605, 610, 615, 620, 625 are placed in their correct positions according to the original frame ( Figure 6B ). In this case, the reconstruction process copies the pixels in the bounding box of a given object to the corresponding location of the object in the original frame. Figure 6B The reconstructed frame in is used as input to the machine task system 150 which performs the desired operation.

[0054] Use the bounding box information of each encapsulated region to extract the Fig. 6A ) extracts regions and places them into the reconstructed frame ( Figure 6B ) in the corresponding position in the reconstructed frame. Although scaling of the reconstructed frame is possible, the reconstructed frame preferably has the same size as the input frame. The reconstructed frame generally does not have areas that are not detected and packed at the encoder system 100. In this example, Figure 3 The tree region object 330 shown is not detected and is not encapsulated in the bitstream and will not be present in the reconstructed frame. Similarly, background information around the object in the original frame may not be present in the encapsulated bitstream, further reducing the data to be encoded and decoded.

[0055] The machine task system 150 uses the reconstructed frame ( Figure 6B) as input to perform the intended task. The machine task system 150 can dynamically send the target task parameters to the encoding system 100. In some embodiments, in response to the updated target task parameters, the encoding system 100 can preferably update the type and number of region / object detectors selected for encoding video frames.

[0056] exist Figure 7 A simplified example of region depackaging for a single region / object in a decoded frame is presented in FIG. The figure further illustrates a process for depackaging an object (e.g., object "O4"). As noted in conjunction with the object packing process, each detected object is packed with information sufficient to identify the location and size of the object / region in the original frame. In one example, this can take the form of coordinates of one corner (e.g., the upper left corner) of a rectangular bounding box and the width and height of the object. Figure 7 , the video decoder 140 will output the packed frame 705. In the region unpacking stage 145, the information about each object is used to locate the object in the reconstructed frame. In the region unpacking 705 of object O4, the coordinates O4x and O4y locate the upper left corner of the rectangular bounding box of the object in the reconstructed frame, O4W specifies the width of the bounding box of O4, and O4H specifies the height of the bounding box of O4. Simultaneously or subsequently, using substantially the same process, the remaining objects are extracted and placed into the reconstructed frame 715.

[0057] Figure 8 8 is a simplified block diagram further illustrating an example of a decoder according to the present invention. Encoded video is received at an entropy decoding module 805. In the entropy decoding module 805, semantic and video payload information are decoded from a binary representation and passed to an inverse quantization (for video payload) module 810 and an in-loop filter 825 (for video information), and to a frame decapsulation component 845 (for encapsulation semantics). The inverse quantization module 810 applies an operation that inverts the quantization employed during encoding and produces residual frequency coefficients. An inverse transform processor 815 is coupled to the inverse quantization module 810 and applies a complementary operation that inverts the forward transform employed during encoding and produces residual pixel values. These values ​​are added to the previously decoded frame in a summation stage 820 to reconstruct the current frame. The in-loop filter 825 applies processing at the boundaries of the prediction blocks to eliminate abrupt changes between blocks.

[0058] The decoded picture buffer 830 stores decoded video frames used to predict other frames in the independent picture group. The size of the buffer is usually controlled by decoder parameters.

[0059] The decoder comprises an intra prediction processing block 835 in which pixel value prediction is performed based on information contained in the current frame. All previously decoded blocks of a frame can be used to predict the next block in the frame. The decoder also comprises a motion compensated prediction module 840 in which blocks in the current frame are predicted from collocated or displaced matching blocks in adjacent frames using motion vectors describing the displacement.

[0060] The frame decapsulation module 845 is coupled to the decoded picture buffer and entropy decoder 805. For example Figure 7 As shown, the frame decapsulation module 845 obtains a fully decoded video frame and uses the packing semantic information received from the entropy decoder 805 to decapsulate the regions and place them into designated locations in the reconstructed frame.

[0061] The reconstructed frame processor 850 provides the final output of the decoder, which is typically the size of the input frame at the encoder side and contains all regions of interest at positions as in the input frame. However, it should be understood that in some applications, the encoder / decoder may decide to encode the positions and scales of regions that do not match the input positions and scales.

[0062] Preliminary experimental results

[0063]

[0064] The preliminary experimental results are shown in the table above. Figure 1 An implementation of the region encapsulation based video system is shown to process a sample dataset consisting of 100 images.

[0065] Using the object detector from the Detectron2 library (Girshick et al., 2018, Detectron, obtained from https: / / github.com / facebookresearch / detectron), the inference for each frame is used to blank all pixels outside the object boundary. The region coordinates output by the model are then used to perform packaging so that all regions are arranged into an optimal bin size. Each packaged frame is used as the input of the video encoder.

[0066] At the decoder side, the compressed frames are decapsulated using the region and position parameters included in the bitstream, and the reconstructed images are then processed using the object segmentation model implemented by Detectron2.

[0067] This table describes the results of using the VVC reference encoder (Bros et al., Overview of the Versatile Video Coding (VVC) Standard and its Applications. IEEE Transactions on Circuits and Systems for Video Technology 31, 10 (October 2021), 3736–3764. DOI: https: / / doi.org / 10.1109 / TCSVT.2021.3101953) VTM in intra-coding mode. The columns represent the average bits per pixel (BPP) and mean average precision (mAP) for quantization parameters 22, 27, 32, 37, 42, and 47 for the above 100 images. "Blk Packing" corresponds to a packed frame in which black is used for any pixel outside the region box. The "Original" column shows the results of the same 100 images without region packing processing.

[0068] Overall, in this example, region packing significantly reduces BPP while maintaining high mAP. In most cases, the encoded frames processed with region packing have comparable accuracy to the original untransformed video frames. Overall, the BD-rate numbers show that for the same accuracy, this packing system can produce output with lower BPP. Similarly, the BD-mAP results show that for equivalent BPP, there is some potential for improving mAP.

[0069] It should be noted that any one or more aspects and embodiments described herein may be conveniently implemented using digital electronic circuits, integrated circuits, specially designed application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), computer hardware, firmware, software, and / or combinations thereof as implemented and / or implemented in one or more machines programmed according to the teachings of this specification (e.g., one or more computing devices used as user computing devices for electronic documents, one or more server devices, such as document servers, etc.), which will be apparent to those of ordinary skill in the computer arts. These various aspects or features may include implementation in one or more computer programs and / or software executable and / or interpretable on a programmable system, the programmable system comprising at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from and send data and instructions to a storage system, at least one input device, and at least one output device.

[0070] Skilled programmers can easily prepare appropriate software coding based on the teachings of this disclosure, which is obvious to those of ordinary skill in the software field. The various aspects and implementations of using software and / or software modules discussed above may also include appropriate hardware for assisting in implementing the machine executable instructions of the software and / or software modules.

[0071] Such software can be a computer program product using a machine-readable storage medium. A machine-readable storage medium can be any medium capable of storing and / or encoding a sequence of instructions executed by a machine (e.g., a computing device) that enables the machine to execute any one of the methods and / or embodiments described herein. Examples of machine-readable storage media include, but are not limited to, disks, optical disks (e.g., CDs, CD-Rs, DVDs, DVD-Rs, etc.), magneto-optical disks, read-only memory "ROM" devices, random access memory "RAM" devices, magnetic cards, optical cards, solid-state memory devices, EPROMs, EEPROMs, programmable logic devices (PLDs), and / or any combination thereof. As used herein, machine-readable media is intended to include a single medium and a collection of physically separated media, such as a collection of optical disks or one or more hard disk drives combined with a computer memory. As used herein, a machine-readable storage medium does not include signal transmission in transient form.

[0072] Such software may also include information (e.g., data) carried as a data signal on a data carrier (such as a carrier wave). For example, machine executable information may be included as a data-bearing signal embodied in a data carrier (where the signal encodes a sequence of instructions or a portion thereof for execution by a machine (e.g., a computing device)) and any related information (e.g., data structures and data) that enables the machine to perform any of the methods and / or embodiments described herein.

[0073] Examples of computing devices include, but are not limited to, electronic book reading devices, computer workstations, terminal computers, server computers, handheld devices (e.g., tablet computers, smart phones, etc.), network devices, network routers, network switches, bridges, any machine capable of executing a sequence of instructions specifying actions to be taken by the machine, and any combination thereof. In one example, the computing device may include and / or be included in a kiosk.

[0074] It should be noted that any one or more aspects and embodiments described herein can be conveniently implemented using one or more machines programmed according to the teachings of this specification (e.g., one or more decoders and / or encoders used as user decoders and / or encoders for electronic documents, one or more server devices, such as document servers, etc.), which will be apparent to those of ordinary skill in the computer arts. A skilled programmer can easily prepare appropriate software coding based on the teachings of this disclosure, which will be apparent to those of ordinary skill in the software arts. The various aspects and implementations using software and / or software modules discussed above may also include appropriate hardware for assisting in implementing the machine executable instructions of the software and / or software modules.

[0075] The above is a detailed description of an illustrative embodiment of the present invention. Various modifications and additions may be made without departing from the spirit and scope of the present invention. The features of each of the various embodiments described above may be appropriately combined with the features of other embodiments described so as to provide a variety of feature combinations in associated new embodiments. In addition, although a plurality of separate embodiments have been described above, what has been described herein is merely an illustration of the application of the principles of the present invention. In addition, although the specific methods herein may be illustrated and / or described as being performed in a particular order, this order is highly variable in a person of ordinary skill in the art to implement the embodiments disclosed herein.

[0076] Accordingly, this description is intended to be merely illustrative, and not otherwise limiting the scope of the present invention.

[0077] In the above description, phrases such as "at least one of ... " or "one or more of ... " may be followed by a list of combinations of elements or features. The term "and / or" may also appear in a list of two or more elements or features. Unless otherwise implicitly or explicitly contradictory to the context in which the phrase is used, such a phrase is intended to represent any element or feature in the listed elements or features individually or any element or feature in the listed elements or features and any element or feature combination in the listed other elements or features. For example, the phrases "at least one of A and B", "one or more of A and B" and "A and / or B" are intended to represent "single A, single B, or A and B together". Similar explanations are also intended to be used for lists including three or more items. For example, the phrases "at least one of A, B and C", "one or more of A, B and C" and "A, B and / or C" are intended to represent "single A, single B, single C, A and B together, A and C together, B and C together, or A and B and C together". Additionally, use of the term “based on” above and in the claims is intended to mean “based, at least in part, on” such that non-recited features or elements are also allowable.

[0078] Depending on the desired configuration, the subject matter described herein may be embodied in systems, devices, methods and / or articles. The embodiments set forth in the foregoing description do not represent all embodiments consistent with the subject matter described herein. On the contrary, they are merely some examples consistent with aspects related to the described subject matter. Although some variations have been described in detail above, other modifications or additions are also possible. In particular, in addition to those set forth herein, additional features and / or variations may also be provided. For example, the embodiments described above may be directed to various combinations and sub-combinations of the disclosed features and / or combinations and sub-combinations of several other features disclosed above. In addition, the logical flows depicted in the drawings and / or described herein do not necessarily require the specific order shown or the sequential order to achieve the desired results. Other embodiments may be within the scope of the present disclosure.

Claims

1. A video encoder for compression using region packing, comprising: A region detection module, the region detection module receiving a video frame for encoding, identifying at least one region of interest in the video frame based on a target task parameter, and generating a bounding box for each identified region of interest; a region extractor module coupled to the region detection module and for each identified region of interest, the region extractor obtains pixels within the bounding box from the video frame; a region packing module that receives the identified region of interest and arranges the bounding box into a packing frame that substantially omits data outside the identified region of interest in the frame; as well as A video encoder receives the encapsulated frame and generates a coded bit stream according to the encapsulated frame.

2. The encoder of claim 1, wherein the bounding box is rectangular, and wherein the region detector module generates parameters comprising coordinates of corners of the bounding box in the frame, a width parameter, and a height parameter.

3. The encoder of claim 1, wherein the parameters of the bounding box are encoded in a header of the encoded bitstream.

4. The encoder of claim 1, wherein the region detector comprises at least one object detector.

5. The encoder of claim 1, wherein the region can include a color region.

6. A video decoder for a video bitstream encoded using region packing: a video decoder receiving an encoded bitstream including at least one encoded region; a region decapsulation module that identifies parameters of a bounding box of the at least one encoded region; as well as A frame reconstruction module is configured to locate and size the bounding box within a reconstructed frame using the parameters and to fill the bounding box with decoded pixels corresponding to the region.

7. The decoder of claim 6, wherein the bounding box is a rectangle and the parameters include coordinates of corners of the bounding box, a width parameter, and a height parameter.

8. The decoder of claim 6, wherein the reconstructed frame has substantially the same size as a frame of the video before encoding, and the reconstruction module places each encoded region at a position in the reconstructed frame substantially the same as the position of the encoded region in the video before encoding.

9. The decoder of claim 6, wherein the parameters are extracted from header information in the encoded bitstream.

Images