Method and apparatus for identifying video decoding errors - Patents.com

Abstract

A method, apparatus, system, and article of manufacture for identifying video decoding errors are disclosed. An example apparatus includes an atlas generator that generates atlas data for one or more atlases generated from input views of a video, a hash generator that performs a hash operation on the atlas data to generate a hash value and includes the hash value in a message, and a multiplexer that combines the one or more atlases, coding atlas data corresponding to the atlas data, and the message to generate a video bitstream.

Description

[Technical field]

[0001] [Related Applications] This patent claims the benefit of U.S. Provisional Application No. 63 / 004,741, filed April 3, 2020. U.S. Provisional Application No. 63 / 004,741 is incorporated herein by reference in its entirety. Priority to U.S. Provisional Application No. 63 / 004,741 is hereby claimed.

[0002] [Technical field] FIELD OF THE DISCLOSURE This disclosure relates generally to video processing, and more particularly, to methods and apparatus for identifying video decoding errors. [Background technology]

[0003] In video compression / decompression (codec) systems, compression efficiency and video quality are important performance criteria. For example, visual quality is an important aspect of the user experience in many video applications. Compression efficiency impacts the amount of memory required to store a video file and / or the amount of bandwidth required to transmit and / or stream the video content. Video encoders typically compress video information so that more information can be transmitted in a given bandwidth or stored in a given memory space, etc. The compressed signal or data is then decoded by a decoder, which decodes or decompresses the signal or data for display to a user. In many instances, the higher the visual quality, the more compression is desired.

[0004] Currently, standards are being developed for immersive video coding and point cloud coding, including Video-based Point Cloud Compression (V-PCC) and MPEG Immersive Video Coding (MIV). Such standards seek to establish and improve compression efficiency and reconstruction quality in the context of immersive video and point cloud coding. [Brief description of the drawings]

[0005] [Figure 1] 1 is an exemplary environment for encoding and / or decoding video in accordance with examples disclosed herein.

[0006] [Diagram 2] 2 is a flowchart representing machine-readable instructions that may be executed to implement the example encoding system of FIG. 1.

[0007] [Diagram 3] 2 is a flowchart representing machine-readable instructions that may be executed to implement the example decoding system of FIG. 1.

[0008] [Figure 4] 4 is a block diagram of an exemplary processing platform configured to execute the instructions of FIG. 3 to implement the encoding system of FIG. 1.

[0009] [Diagram 5] 4 is a block diagram of an exemplary processing platform configured to execute the instructions of FIG. 3 to implement the decoding system of FIG. 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The figures are not to scale. Alternatively, thicknesses of layers or regions may be exaggerated in the figures. Generally, the same reference numbers are used throughout the figures and the accompanying description that refer to the figures or portions thereof. As used in this patent, the statement that any part (e.g., layer, film, area, region, or plane) is on (e.g., located, positioned, disposed, formed, etc.) another part in any manner indicates that the referenced part is in contact with the other part or that the referenced part is above the other part with one or more intermediate parts disposed therebetween. Connection references (e.g., attached, coupled, connected, joined) should be construed broadly and may include intermediate members between a set of elements and relative movement between the elements, unless otherwise specified. As such, connection references do not necessarily infer that two elements are directly connected and in a fixed relationship to each other. A statement that any part is "in contact" with another part means that there are no intermediate parts between the two parts. Although the figures show layers and regions with clear lines and boundaries, some or all of these lines and / or boundaries may be idealized. In reality, the lines and / or boundaries may be non-observable, blended, and / or irregular.

[0011] The descriptors "first," "second," "third," and the like are used herein to distinguish between multiple elements or components that may be referenced separately. Unless otherwise specified and understood based on the context of their use, such descriptors do not imply any priority, physical order or placement within a list, or chronological order, but are used merely as labels to refer to multiple elements or components separately to facilitate understanding of the disclosed examples. In some instances, the descriptor "first" may be used to represent an element in the detailed description, but the same element may be represented in the claims by a different descriptor, such as "second" or "third." In such instances, it should be understood that such descriptors are used merely to facilitate reference to multiple elements or components.

[0012] In the context of immersive video coding and point cloud coding, video standards such as Visual Volumetric Video-based Coding (V3C) and Video-based Point Cloud Compression (V-PCC) and MPEG Immersive Video Coding (MIV) may be utilized. For such standards, there may be requirements to ensure that the decoder complies with such standards. For example, there may be one or more requirements that the decoder obtains an unaltered and / or uncorrupted bitstream and / or that the decoder correctly decodes the obtained bitstream.

[0013] In V3C / V-PCC, dynamic point clouds are compressed using multiple projections of the point cloud onto a plane of texture, such as color, and geometry, such as depth. After the dynamic point cloud is compressed, it is segmented to extract rectangular regions, called patches, of the same depth from the projection plane. The patches are packed into an atlas or atlas tile along with an occupancy map. The occupancy map indicates the part of the atlas that should be used (e.g., the occupancy area within the patch packed into the atlas). In addition, patch information metadata is used to indicate how the patch is mapped between the projection plan and the canvas (e.g., the atlas or atlas tile). Two-dimensional (2-D) video codecs, such as high efficiency video coding (HEVC), are used to exploit spatial and temporal redundancies in the geometry and texture components of the canvas. Information about the size and location of the patches in the original projection and the atlas or atlas tile are signaled as coding atlas tile data. Occupancy information is also signaled. The atlas may be divided into a grid of non-overlapping data blocks, and the block size may be included in the bitstream. The atlas data includes a patch value for each block indication (e.g., patch identification and / or index of a block of data). A block may correspond to a patch or may be a subset of a patch. In some cases, patches overlap other patches. The order in which patch data is signaled in the bitstream (e.g., which corresponds to a patch identifier) ​​may be used to determine the priority of patches at a particular block location, e.g., patches with higher patch identifier values ​​may be prioritized.

[0014] In coding according to the V3C / V-PCC and MIV standards, rectangular patches are formed from the projection views and placed in an atlas. Patches may overlap in the atlas, and the last signaled patch takes priority to resolve the overlap. The standards define a signaling mechanism for transmitting coding atlas data for each patch in the atlas, including information including the size and position of the patch in the corresponding projection view and / or a patch identifier. The atlas data is a mapping of data blocks to patches in each atlas. In some examples, such information is included in a coding atlas style data syntax structure. The coding atlas data is signaled on a per-patch basis. The coding atlas data can be decoded by a decoder to obtain the atlas data. In some examples, the coding atlas style data syntax structure is signaled at each frame time or instance. In some examples, the coding atlas style data syntax structure is maintained for multiple consecutive frame times or instances. Multiple coding atlas style data syntax structures corresponding to different atlases or atlas styles may be signaled within the same access unit (part of the components of the bitstream (e.g., atlas data, texture video, geometry video, occupancy video, etc.) that have the same decoding order count (e.g., data of a V-PCC / MIV bitstream corresponding to a particular time instance), all of which correspond to the same frame time, instance, or PictureOrderCount (POC) value. For example, a standard may define an operation to compute the atlas data (e.g., a variable 2-dimensional array BlockToPatchMap[y][x] (also called DecoderToPatchMap[y][x]), which represents a mapping of blocks and corresponding patches in the atlas space, where x and y are the coordinates of the block at block size granularity). The coding atlas data is included in the bitstream from the encoder to the decoder.

[0015] The MIV standard uses a similar approach to code the multiple image views that V3C / V-PCC uses for its multiple projections. For example, multiple atlases can be used in MIV. In MIV, the placement of a patch on an atlas may be retained across multiple frame times. Consistency checks are available for multiple atlases and for atlas persistence. Persistence corresponds to how long the data corresponding to an atlas can be retained before the data is replaced by additional data.

[0016] In the examples disclosed herein, a supplemental enhancement information (SEI) message is provided that includes an atlas data hash. Such a message may be implemented in the MIV standard, the V3C / V-PCC standard, and / or other standards that generate a message that is included in the bitstream. The atlas data hash (also called hashed atlas data) is the result of a hash operation performed on the atlas data. The examples disclosed herein include the atlas data hash in an SEI message, and include the SEI message in the bitstream along with the coding atlas data. A decoder uses the hash atlas value in the SEI message to help the decoder identify transmission and / or decoding errors, as described below.

[0017] As mentioned above, examples disclosed herein include hashed atlas data (e.g., hash values ​​for each atlas per block indication of the patch preferred for that block, or per atlas of the view or projection direction that the patch represents) in a supplemental enhancement information (SEI) message in the bitstream along with the corresponding coding atlas data. The hashed atlas data may include a per-atlas hash of a two-dimensional array variable BlockToPatchMap[y][x]. For example, a hash function may be applied to the scanned (e.g., raster-scanned) atlas data such that the atlas data is a two-dimensional data structure of numbers indicating which patch identifier (e.g., a number) or view identifier a particular sample corresponds to. In some examples, the encoder determines a value for BlockToPatchMap[y][x] for each atlas at each access unit for which patch data is signaled in the coding atlas tile data. The hash function may be implemented as one or more hash operations. In some examples, the encoder generates a hash value of the entire two-dimensional array and signals the hash value of the atlas data in the SEI message.

[0018] In the example disclosed herein, when the decoder receives the coding bitstream, the decoder separates the SEI message from the bitstream. The decoder determines an atlas hash value from the SEI message. Furthermore, the decoder decodes the coding atlas data (e.g., the value of the BlockToPatchMap[y][x] two-dimensional array) for each coding atlas in each access unit to generate decoded atlas data. After the decoder decodes the coding atlas data, the decoder generates a hash value (e.g., the BlockToPatchMap[y][x] two-dimensional array) for the decoded atlas data using the same hashing technique implemented by the encoder to generate the decoder-side hash value. In this manner, the decoder compares the decoder-side hash value with the hash value signaled in the SEI message. If the values ​​match, the decoder verifies with a high degree of confidence that the bitstream is not corrupted and verifies that the decoder's operation of decoding the coding atlas data complies with the specification (e.g., verifies that the decoder is decoding correctly). If the values ​​do not match, the decoder can determine that a decoding error occurred (e.g., the bitstream was corrupted and / or the decoder operation did not comply with the specification for decoding the coding atlas data).

[0019] As provided herein, decoded atlas data hashes and atlas data hashes are generated for use in video coding or for use in standardized codecs such as the V3C / V-PCC standard and / or the MIV standard. The SEI message may signal a hash value for each atlas data instance (e.g., for each unique 2-dimensional atlas array).

[0020] 1 is an example environment 100 including an example block diagram of an example encoding system 101 and an example decoding system 110. The example encoding system 101 includes an example atlas generator 102, an example volumetric video encoder 104, an example hash generator 106, and an example multiplexer 108. The example decoding system 110 includes an example demultiplexer 112, an example volumetric video decoder 114, an example data reconstructor 116, an example hash generator 118, and an example comparator 120.

[0021] 1 (also referred to as an encoder, encoding device, etc.) and the exemplary decoding system 110 may be implemented in any suitable form factor or one or more of such devices, including a server computer, a cloud computing environment, a personal computer, a laptop computer, a tablet, a phablet, a smartphone, a gaming console, a wearable device, a display device, an all-in-one device, a two-in-one device, etc. Although the exemplary encoding system 101 and the exemplary decoding system 110 of FIG. 1 are implemented in different computing devices, the exemplary encoding system 101 and the exemplary decoding system 110 may be implemented in the same device.

[0022] The atlas generator 102 of the encoding system 101 of FIG. 1 receives multiple views of an input video to be encoded. There may be any number of views of a scene, and each view may include a corresponding texture picture or frame (or texture video) and a corresponding geometry picture or frame (or geometry video). For each time instance, video data of one or more views of the scene may be provided, resulting in multiple texture frames and multiple depth or geometry frames. For example, each texture frame may include per-pixel color information, such as brightness / color (YUV) data, red-green-blue (RGB) data, or similar data, and each geometry or depth frame may include per-pixel depth data or similar data. The texture and geometry frames may have the same or different resolutions. For example, the texture frames may be video graphics array (VGA), high definition (HD), Full-HD (e.g., 1080p), 4K resolution video, 8K resolution video, etc. Examples disclosed herein are described with respect to frames and pictures, which are used synonymously.

[0023] The example atlas generator 102 of FIG. 1 generates a texture atlas and a geometry or depth atlas for a particular frame index or instance (e.g., converting from an input view). For example, the atlas generator 102 decomposes each of the multiple views into patches (e.g., rectangular regions) and arranges them to form an atlas. The atlas generator 102 can format the atlas to reduce redundancy between the multiple views. The example atlas generator 102 forms a texture atlas (including texture information) and a geometry or depth atlas (including geometry or depth information) and provides the data to a volumetric video encoder 104 of the encoding system 101.

[0024] The example atlas generator 102 of FIG. 1 determines atlas data including metadata indicating the source of each patch in the texture (e.g., the corresponding view associated with that patch) and the depth atlas for a particular frame index or instance. In some examples, the atlas generator 102 generates a single instance of atlas data for the texture and depth atlas such that patches in the depth atlas follow or match patches in the texture atlas, and vice versa. The atlas data provides metadata for reconstructing multiple views (or portions thereof) using the texture and depth patch data in the atlas. As used herein, the term atlas data refers to any data structure, such as a two-dimensional array of numbers indicating the source of a patch of texture and the depth atlas, indicating the source of such a patch. In some examples, if a pixel does not correspond to a patch, the atlas data includes a two-dimensional array of numbers per pixel, each representing a patch identifier (or a default value (e.g., as the maximum value available in the data structure)). In some examples, the atlas generator 102 generates the atlas data to include a two-dimensional array of numeric values ​​per pixel, each of which indicates a source view for the pixel (or a default value if the pixel does not correspond to a source view). In some examples, the atlas generator 102 generates the atlas data to include a two-dimensional array of numeric values ​​per pixel, each of which indicates a source view and patch number for the pixel (or a default value if the pixel does not correspond to a source view). In some examples, the two-dimensional array of data can be represented in a one-dimensional form, such as that created by a raster scan operation (e.g., by listing all data in the two-dimensional array in a one-dimensional vector). In some examples, the atlas generator 102 generates the atlas data using a DecoderToPatchMap[y][x] syntax element, as described further below.

[0025] In some examples, the atlas generator 102 generates an atlas data instance, as illustrated with respect to the pseudocode shown in Table 1 below. In the pseudocode in Table 1, the atlas data is DecoderToPatchMap[y][x] (also called BlockToPatchMap[y][x]). [Table 1]

[0026] In the above pseudocode (1) in Table 1, the first section of the code defines the width and height of the BlockToPatchMap2D array based on the coding atlas tile size and block size. The second section of pseudocode (1) sets an initial value for the block (e.g., -1) indicating that the block is unoccupied. The third section of pseudocode (1) sets the patch index values ​​of the blocks in the BlockToPatchMap2D array that are covered by the patch, using the x and y positions of the block, and the patch width and patch height, for a patch in the coding atlas data.

[0027] In some examples, the atlas generator 102 generates atlas data instances (e.g., patch indices for each block) as shown with respect to the pseudocode shown in Table 2. In the pseudocode of Table 2, the atlas data is again DecoderToPatchMap[y][x]. In some examples, the format of BlockToPatchMap is assigned as 16-bit unsigned, and the default value of each sample of BlockToPatchMap[y][x] is maintained in the absence of a patch that overwrites the value and is changed from -1 to 0xFFFF (or 65535) relative to the pseudocode of Table 1. [Table 2]

[0028] In the above pseudocode (2) in Table 2, the first section of the code defines the width and height of the BlockToPatchMap2D array based on the coded atlas tile size and block size. The second section of pseudocode (2) sets an initial value for the block (e.g., 0xFFFF) indicating that the block is unoccupied. The third section of pseudocode (2) sets the patch index values ​​of the blocks in the BlockToPatchMap2D array that are covered by the patch, using the x and y positions of the block, and the patch width and patch height, for a patch in the coded atlas data.

[0029] The exemplary atlas generator 102 outputs the atlas data to the exemplary volumetric video encoder 104 and the exemplary hash generator 106 of the encoding system 101 .

[0030] The volumetric video encoder 104 of the encoding system 101 of FIG. 1 encodes the texture and depth atlas into a coding texture and depth bitstream representing the atlas and outputs the encoded data to an exemplary multiplexer 108 of the encoding system 101. In some examples, the volumetric video encoder 104 encodes the texture and depth atlas based on a standard-compliant coding, such as HEVC, to generate a standard-compliant bitstream. In the illustrated example, the volumetric video encoder 104 encodes the atlas data to generate encoded atlas data, includes the encoded atlas data in a coding atlas data bitstream representing the atlas data, and outputs the coding atlas data bitstream to the exemplary multiplexer 108. In some examples, the volumetric video encoder 104 encodes the atlas data based on a standard-compliant coding, such as HEVC, to generate a standard-compliant bitstream.

[0031] The hash generator 106 of the encoding system 101 of FIG. 1 uses the atlas data to generate a hash value using any one or more suitable hashing operations, such as a 16-byte Message Digest Algorithm 5 (MD5) hash, a Cyclic Redundancy Check (CRC) hash, a checksum hash, etc. After the hash generator 106 hashes the atlas data, the hash generator 106 signals (e.g., includes or embeds) the hashed atlas data within one or more SEI messages. The hash generator 106 may generate an SEI message to include the hashed atlas data, or may receive an SEI message generated from another component and include the hashed atlas data in an already generated SEI message. The exemplary hash generator 106 outputs the SEI message including the hashed atlas data to a multiplexer 108 of the encoding system 101. In some examples, the hash generator 106 includes a single hash value for a single atlas data instance in the atlas data hash SEI message. In some examples, the hash generator 106 includes multiple hash values ​​for multiple atlas data instances in the atlas data hash SEI message (e.g., each hash value is indexed to a particular atlas data instance). In some examples, the hash generator 106 includes persistent data corresponding to the atlas data instances in the SEI message, as described below. In some examples, the hash generator 106 generates a hash value for each atlas data instance. In some examples, the hash generator 106 generates hash values ​​for several atlas data instances, such as a single instance in a group of pictures (GOP), such that hash value verification is provided intermittently for the transmitted bitstream.

[0032] The multiplexer 108 of the encoding system 101 of Figure 1 combines or multiplexes the coding texture and depth bitstream, the coding atlas data bitstream, and the hashed SEI message of the atlas data. The encoding system 101 transmits the resulting bitstream to the decoding system 110 or to a memory for eventual use by the decoding system 110. In some examples, the multiplexer 108 transmits the resulting bitstream to the decoding system 110 using an interface (e.g., the interface 420 of Figure 4). In such examples, the interface 420 can transmit the resulting bitstream to the decoding system 110 via a wired or wireless connection (e.g., via Bluetooth, via the Internet, via a local network, via Wi-Fi, etc.).

[0033] The decoding system 110 (also referred to as a decoder, a decoding device, etc.) of FIG. 1 obtains a bitstream via a demultiplexer 112 of the decoding system 110. In some examples, the demultiplexer 112 obtains the bitstream via an interface (e.g., interface 520 of FIG. 5). The decoding system 110 may be implemented via a client system, and the encoding system 101 may be implemented via a server or source system. The demultiplexer 112 of the decoding system 110 demultiplexes or parses the received bitstream to generate (e.g., extract) (a) a coding texture and depth bitstream that includes a coding representation of texture and depth video and is expected to match the coding texture and depth bitstream described for the encoding system 101, (b) a coding atlas data bitstream that is expected to match the atlas data generated by the encoding system 101, and (c) one or more atlas data hash SEI messages from the obtained bitstream data. The coding texture and depth bitstream, coding atlas data bitstream, and atlas hash SEI messages are characterized as decoder-side coding texture and depth bitstream, coding atlas data bitstream, and atlas data hash SEI messages. The coding texture and depth bitstream, coding atlas data bitstream, and atlas data hash SEI messages replicate their counterparts in the encoder system 101 in the absence of corresponding bitstream errors in the decoding and / or transmission of the bitstream. However, mismatches may occur due to bitstream corruption, transmission failures, and / or other reasons.

[0034] An example volumetric video decoder 114 of the decoding system 110 of FIG. 1 receives (e.g., obtains) and decodes the coding texture, depth bitstream, and coding atlas data bitstream (e.g., to generate a decoded texture atlas, a depth atlas, and / or corresponding atlas data instances). The volumetric video decoder 114 transmits the decoded data to a data reconstructor 116 of the decoding system 110. The volumetric video decoder 114 decodes the coding texture to generate decoded texture values, decodes the coding depth values ​​to generate decoded depth values, and decodes the coding atlas data bitstream to generate a decoded atlas data bitstream, a depth atlas, and / or corresponding atlas data instances based on the encoding technique used in the volumetric video encoder 104.

[0035] The data reconstructor 116 in Figure 1 uses the decoded texture atlas, the decoded depth atlas, and the decoded atlas data to reconstruct decoded versions of the input views provided to the encoding system 101. For example, the data reconstructor 116 in the decoding system 110 can perform the inverse operations of the atlas generator 102 in the encoding system 101 to generate reconstructed views that can be used to generate a user's viewport view.

[0036] The example hash generator 118 of FIG. 1 obtains the decoded atlas data from the volumetric video decoder 114. The decoding system 110 attempts to replicate the atlas data generated by the encoding system 101, which is decoded in the decoding system 110 so that the encoder-side hashed atlas data and the decoder-side hashed atlas data match (e.g., in the absence of decoding errors). However, as discussed above, due to bitstream corruption and / or transmission failures and / or failures of the volumetric video decoder 114 of the decoding system 110, the encoder-side atlas data and the decoder-side atlas data may not match. As discussed below, the comparator 120 may compare the encoder-side hash value and the decoder-side hash value to determine a mismatch.

[0037] 1 applies the same hash operation as the hash generator of the encoding system 101 to the same atlas data (e.g., matched in the absence of decoding / transmission errors, or not matched in the presence of decoding / transmission errors) to generate hash values ​​for instances of atlas data. The applicable hash operation may be a default hash operation corresponding to the coding standard, an applicable hash operation signaled in the atlas data hash SEI message, etc. For each hash value generated by the hash generator of the decoding system 110, a decoder-side hash value (e.g., the hash value generated at the decoder) is compared (by comparator 120) with an encoder-side hash value (e.g., the hash value received in the hash SEI message).

[0038] The example comparator 120 of FIG. 1 compares the decoder-side hash (e.g., from the hash generator 118) with the encoder-side hash (e.g., from the demultiplexer 112). If the comparator 120 determines that the two hash values ​​match, the decoding system 110 (e.g., a client system) determines with high certainty that the received bitstream was not corrupted and that the operation of the volumetric video decoder 114 on the atlas data (e.g., metadata) was compliant with the bitstream and / or an applicable video coding standard such as V3C / V-PCC or MIV. If the comparator 120 determines that any two hash values ​​do not match, the comparator 120 outputs an error flag indicating a corrupted bitstream or non-compliant decoder operation. In some examples, the comparator 120 outputs an error flag for each atlas data instance. The error flag for each atlas data instance is a value that identifies a match or mismatch of the hash values ​​(e.g., "1" for match, "0" for mismatch, or vice versa). In some examples, if a mismatch is detected, comparator 120 outputs an error flag for each fault in the atlas data instance (otherwise, a match is presumed).

[0039] Next, exemplary syntax and semantics are provided for implementing verification based on atlas data hash values. In some examples, a syntax element is included in the SEI message to signal the number of atlases (e.g., atlas data instances). In some examples, a hash value is signaled for each atlas. Exemplary hash generators 106, 118 each generate a hash value using a 16-byte MD5 sum, CRC, checksum, or another type of hash value. The determination of the hash value is described below, but in some examples, each of the two bytes of the 16-bit unsigned BlockToPatchMap[y][x] value is input in turn into the hash calculation. If the number of patches is 2, then the hash value is signaled. 16(For example, 16 bits or 2 bytes are sufficient to represent the patch identifier.) In such an example, a hash calculation can be performed on the bytes.

[0040] In some examples, MIV and VPCC bitstreams contain a VPCC unit. The VPCC unit contains a header and payload information. The header contains a vuh_atlas_id syntax element that indicates the atlas ID to which the payload applies. The MIV standard supports a "common atlas" that contains data that is applicable to all atlases. In some examples, if an SEI message is present in the "common atlas", it may contain hash data for multiple atlases. In some examples, a separate SEI message with the same vuh_atlas_id value as the associated atlas may be used for each atlas. If hash data is signaled for more than one atlas, the atlas_id value to which each hash applies is explicitly signaled.

[0041] In some examples, the atlas data is consistent across multiple access units, and the proposed SEI hash data may be retained until cancelled. Persistence may be signaled for the entire SEI message, which includes data for multiple atlases, or for each atlas individually. In some examples, if the persistence flag value is set to 1 (or some other value), the hash value is retained for future access units in the bitstream until the cancel flag value is set to 1 (or some other value) or until an instantaneous random access point (IRAP) atlas is present in the bitstream (e.g., because a random access decode is present in an IRAP atlas). As mentioned above, an access unit is part of a component of a bitstream (e.g., atlas data, texture video, geometry video, occupancy video, etc.) that has the same decoding order count (e.g., data of a V-PCC / MIV bitstream corresponding to a particular time instance). In some examples, an overall persistence flag and an overall cancel flag are included in the SEI message, which applies to all atlases. In some examples, if more than one atlas is indicated, individual persistence and cancel flags are provided for each atlas. Thus, if comparator 120 (or another device) generates a flag corresponding to a first access unit in a bitstream, the comparator may generate an additional flag for a second (e.g., subsequent) access unit in the bitstream when the message corresponding to the bitstream indicates persistence.

[0042] Table 3 below shows an exemplary syntax used by the exemplary hash generators 106, 118 to signal the atlas data hash value and associated data such as the atlas indicator and persistence flag. [Table 3]

[0043] The example syntax and semantics shown in Table 3 signals the atlas data hash value of the decoded atlas data block to one or more atlas patch maps.

[0044] The syntax element dadh_cancel_flag is set to 1 to indicate that the SEI message cancels the persistence of a previously decoded atlas data hash SEI message in coding order applied to the current layer. The syntax element dadh_cancel_flag is set to 0 to indicate that hash information of the decoded atlas data follows. The syntax element dadh_persistence_flag is set to 1 to indicate that the SEI message specifies persistence of the decoded atlas data hash SEI message. The syntax element dadh_persistence_flag is set to 0 to specify that the decoded atlas data hash SEI message applies only to the current decoded access unit.

[0045] In some examples, let auA be the current access unit, and dadh_persistence_flag is equal to 1, specify that the decoded atlas data hash SEI message is retained in output order until one of the following conditions is met: (1) a new coding video sequence (CVS) for the current layer starts, (2) the bitstream ends, or (3) if FrameOrderCnt(auB) is greater than FrameOrderCnt(auA) (e.g., to determine which access unit corresponds to a later video sequence because a higher count corresponds to a later time in the video sequence), then the current layer's access unit auB is output in the access unit that contains the decoded atlas data hash SEI message, where FrameOrderCnt(auB) and FrameOrderCnt(auA) are the FrameOrderCntVal values ​​of auB and auA, respectively, immediately after performing the decoding process of the picture order count of auB. FrameOrderCnt is the order in which the frames are intended to be coded and / or displayed. FrameOrderCnt is used because the order in which frames are coded and / or displayed may differ from the order in which they exist in the bitstream (e.g., because encoding system 101 may choose to code frames out of order). If atlases are not explicitly signaled for later frames, the hash value may persist for additional frames. The syntax element dadh_num_atlases_minus1 plus 1 specifies the number of atlases or atlas tiles for which the hash value exists.

[0046] The syntax element dadh_atlas_id[a] specifies the atlas ID of the i-th signaling hash data. In some examples, the value of dadh_atlas_id[a] is inferred to be equal to vuh_atlas_id if not present.

[0047] The syntax element dadh_atlas_cancel_flag[dadh_atlas_id[a]] equal to 1 indicates that the SEI message cancels the persistence of the previously decoded atlas data hash in the coding order applied to the dadh_atlas_id[a]th atlas. The syntax element dadh_atlas_cancel_flag[dadh_atlas_id[a]] equal to 0 indicates that decoded atlas data hash information follows for the dadh_atlas_id[a]th atlas. In some examples, the value of dadh_atlas_cancel_flag[dadh_atlas_id[a] is inferred to be equal to 0 if not present.

[0048] The syntax element dadh_atlas_persistence_flag[dadh_atlas_id[a]] specifies the persistence of the decoded atlas data hash SEI message for the dadh_atlas_id[a]-th atlas. The syntax element dadh_atlas_persistence_flag[dadh_atlas_id[a]] equal to 0 specifies that the decoded atlas data applies only to the [dadh_atlas_id[a]]-th atlas of the current access unit.

[0049] In some examples, for auA as the current picture unit, dadh_atlas_persistence_flag[dadh_atlas_id[a]] equal to 1 specifies that the decoded atlas data hash SEI message is retained for the [dadh_atlas_id[a]]th atlas in output order until any of the following conditions are true: (1) the start of a new CLVS, (2) the bitstream ends, or atlas data unit B in the access unit containing the decoded atlas data hash SEI message that applies to the dadh_atlasI-d[a]th atlas is output if FrameOrderCnt(auB) is greater than FrameOrderCnt(aauA), where FrameOrderCnt(auB) and FrameOrderCnt(auA) are the FrameOrderCntVal values ​​of auB and auA, respectively, immediately after performance of the atlas data decoding process with the atlas frame order count of auB.

[0050] In some examples, before computing the hash value, the hash generator 106, 118 arranges the atlas block patch map data into a byte array called btpmData of length dataLen, as follows in pseudocode (3): [Table 4]

[0051] The syntax element dadh_patch_map_md5[i] is the 16-byte MD5 hash of the BlockToPatchMap[][] variable of the atlas of the current access unit whose vuh_atlas_id value is equal to dadh_atlas_id. The value of dadh_patch_map_md5[i] shall be equal to the value of digestVal, obtained in pseudocode (4) using the MD5 function defined in IETF RFC1321 as follows: [Table 5]

[0052] Although illustrated with respect to a 16-byte MD5 hash, any suitable hash operation and value can be implemented (eg, a cyclic redundancy check, a checksum, etc.).

[0053] Operationally, in the encoding system 101, the atlas generator 102 can generate atlas data (e.g., atlas data instance, file, mapping) based on multiple input views, such that the atlas data indicates patch source data for a corresponding patch in the texture atlas and / or depth atlas. For example, the atlas generator 102 can generate atlas data including an indicator for each pixel in the texture atlas and / or depth atlas indicating the source patch, source view, and / or source location of the pixel, such that the source patch, source view, and source location are relative to the source view of the multiple views. The hash generator 106 can generate a hash value based on the atlas data using one or more corresponding hash operations. For example, the atlas data can be scanned from a two-dimensional array to a one-dimensional array, and the hash generator 106 can apply one or more hash operations to the scanned one-dimensional array. The hash generator 106 can code the hash value and, optionally, additional hash values ​​of other atlas data into a hash value message, such as an atlas data hash SEI message, for transmission to the decoding system 110. In some examples, the multiplexer 108 includes (e.g., combines) the atlas data hash SEI message into a bitstream that also includes bitstream portions corresponding to the coding texture atlas, the coding depth atlas, and the coding version of the atlas data. The coding texture atlas, the coding depth atlas, and the coding atlas data may be coded by the volumetric video encoder 104. The coding texture atlas, the coding depth atlas, and the coding atlas data may be coded by the volumetric video encoder 104 to provide a structure for reconstructing multiple input views. In some examples, the hash generator 106 includes in the atlas data hash SEI message a hash operation indicator that indicates one or more hash operations used to generate the hash value.In some examples, the hash generator includes an indicator to cancel persistence of the previous atlas data or to start persistence of the current atlas data in the atlas data hash SEI message. In some examples, the hash generator 106 includes an indicator of the number of atlases and hash values ​​provided and one or more indexes for indexing the hash values ​​and atlases in the atlas data hash SEI message.

[0054] In the decoding system 110, an interface of the decoding system 110 receives a bitstream including bitstream portions representing the coding texture atlas, the coding depth atlas, and the coding atlas data, and an atlas data hash value message, such as an atlas data hash SEI message, which may include any data corresponding to the encoder. The volumetric video decoder 114 decodes the bitstream portions to generate the decoded texture atlas, the decoded depth atlas, and the decoded atlas data (e.g., an instance, file, mapping of the atlas data). The volumetric video decoder 114 applies the same hash operation as the encoder (e.g., selected based on a hash operation indicator signaled in the atlas data hash SEI message) to the decoded atlas data to generate a decoder-side hash value. The comparator 120 compares the encoder-side hash value obtained in the atlas data hash SEI message with the decoder-side hash value. If the comparator 120 determines that the encoder-side hash value matches the decoder-side hash value (generated by the hash generator 118), the comparator 120 can output an indicator to the data reconstructor 116 to release the decoded atlas data for use in generating a decoded texture atlas, a decoded depth atlas, and a view (e.g., representing a viewport in a reconstructed multiview) for presentation to a user using the decoded atlas data. If the comparator 120 determines that the encoder-side hash value does not match the decoder-side hash value, the comparator 120 outputs an indicator (e.g., an error flag) indicating the mismatch, and the decoder can prevent presentation of the view, request a new copy of the associated bitstream, attempt a modified decoding of the received bitstream, discard the retrieved data, or send an instruction to the user.

[0055] Although an exemplary method for implementing the encoding system 101 and / or the decoding system 110 of Figure 1 is illustrated in Figure 1, one or more of the elements, processes and / or devices illustrated in Figure 1 may be combined, divided, rearranged, omitted, deleted and / or implemented in other ways. Furthermore, the exemplary atlas generator 102, the exemplary volumetric video encoder 104, the exemplary hash generator 106, the exemplary multiplexer 108, and / or more generally the exemplary encoding system 101 of Figure 1 and / or the exemplary demultiplexer 112 of Figure 1, the exemplary volumetric video decoder 114, the exemplary hash generator 118, the exemplary data reconstructor 116, the exemplary comparator 120, and / or more generally the exemplary decoding system 110 of Figure 1 may be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Thus, for example, the exemplary atlas generator 102, the exemplary volumetric video encoder 104, the exemplary hash generator 106, the exemplary multiplexer 108, and / or more generally the exemplary encoding system 101 of FIG. 1 and / or the exemplary demultiplexer 112, the exemplary volumetric video decoder 114, the exemplary hash generator 118, the exemplary data reconstructor 116, the exemplary comparator 120, and / or more generally the exemplary decoding system 110 of FIG. 1 may be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers, graphics processing units (GPUs), digital signal processors (DSPs), application specific integrated circuits (ASICs), programmable logic devices (PLDs), and / or field programmable logic devices (FPLDs).When reading any of the apparatus or system claims of this patent to cover pure software and / or firmware implementations, the example atlas generator 102, the example volumetric video encoder 104, the example hash generator 106, the example multiplexer 108, and / or more generally the example encoding system 101 of FIG. 1 and / or the example demultiplexer 112, the example volumetric video decoder 114, the example hash generator 118, the example data reconstructor 116, the example comparator 120, and / or more generally the example decoding system 110 of FIG. 1 are hereby expressly defined to include non-unitary computer-readable storage devices or storage disks, such as memories, digital versatile disks (DVDs), compact disks (CDs), Blu-ray disks, etc., that contain software and / or firmware. Still further, the example encoding system and / or the example decoding system 110 of FIG. 1 may include one or more elements, processes, and / or devices in addition to or instead of those shown in FIG. 1 and / or may include any or all of more than one of the illustrated elements, processes, and devices. As used herein, the phrase "in communication" and variations thereof encompasses direct communication and / or indirect communication through one or more intermediate components and does not require direct physical (e.g., wired) communication and / or constant communication, but rather further includes selective communication at periodic intervals, scheduled intervals, aperiodic intervals, and / or one-off.

[0056] Flowcharts depicting exemplary hardware logic, machine-readable instructions, hardware-implemented state machines, and / or any combination thereof for implementing the encoding system 101 and / or the decoding system 110 of FIG. 1 are shown in FIGS. 2 and 3. The machine-readable instructions may be one or more executable programs or parts of executable programs for execution by a computer processor, such as the processor 412, 512 shown in the exemplary processor platform 400, 500 described below in connection with FIGS. 4 and 5. 45. The programs may be embodied in software stored on a non-transitory computer-readable storage medium, such as a CD-ROM, a floppy disk, a hard drive, a DVD, a Blu-ray disk, or a memory associated with the processor 412, 512, although the entire program and / or parts thereof may alternatively be executed by devices other than the processor 412, 512 and / or embodied in firmware or dedicated hardware. Furthermore, although the exemplary programs are described with reference to the flowcharts shown in FIGS. 4 and 5, many other ways of implementing the exemplary encoding system 110 and / or the decoding system 110 may alternatively be used. For example, the order of execution of the blocks may be changed, some of the described blocks may be modified, eliminated, or combined. Additionally or alternatively, any or all of the blocks may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured to perform the corresponding operations without executing software or firmware.

[0057] The machine-readable instructions described herein may be stored in one or more of a compressed, encrypted, fragmented, compiled, executable, packaged, etc. format. The machine-readable instructions described herein may be stored as data (e.g., portions of instructions, code, representations of code, etc.) that can be utilized to generate, manufacture, and / or produce machine-executable instructions. For example, the machine-readable instructions may be fragmented and stored on one or more storage and / or computing devices (e.g., servers). The machine-readable instructions may require one or more of installing, modifying, adapting, updating, combining, augmenting, configuring, combining, decompressing, unpackaging, distributing, reassigning, compiling, etc., to make them directly readable, interpretable, and / or executable by computing devices and / or other machines. For example, the machine-readable instructions may be stored in multiple portions that are individually compressed, encrypted, and stored on separate computing devices, and the portions when decoded, decompressed, and combined form a set of executable instructions that implement a program as described herein.

[0058] In another example, the machine-readable instructions may be stored in a state that is readable by a computer, although this requires the addition of a library (e.g., a dynamic link library (DLL)), a software development kit (SDK), an application programming interface (API), etc., to execute the instructions on a particular computing device or other device. In another example, the machine-readable instructions may need to be configured (e.g., stored settings, entered data, recorded network addresses, etc.) before the machine-readable instructions and / or corresponding programs can be executed in whole or in part. Thus, the disclosed machine-readable instructions and / or corresponding programs may include such machine-readable instructions and / or programs regardless of the particular format or state of the machine-readable instructions when stored or at rest or in transport.

[0059] The machine-readable instructions described herein may be expressed in any past, present, or future command language, scripting language, programming language, etc. For example, the machine-readable instructions may be expressed using any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, HyperText Markup Language (HTML), and Structured Query Language (SQL).

[0060] 2 and 3 may be implemented using executable instructions (e.g., computer and / or machine readable instructions) stored on a non-transitory computer and / or machine readable medium, such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random access memory, and / or any other storage device or storage disk in which information is stored for any period of time (e.g., for an extended period of time, permanently, for a short moment, during temporary buffering, and / or during caching of information). As used herein, the term "non-transitory computer readable medium" is expressly defined to include any type of computer readable storage device and / or storage disk, to exclude propagating signals, and to exclude transmission media.

[0061] The terms "including" and "having" (and all forms and tenses thereof) are used herein as broad terms. Thus, whenever a claim utilizes any form of "including" or "having" (e.g., comprises, includes, comprising, including, having, etc.) as a preamble or in any type of claim recitation, it should be understood that additional elements, terms, etc. may be present without departing from the scope of the corresponding claim or recitation. As used herein, the term "at least," when used as a term of variation, for example, in a claim preamble, is as broad as the terms "having" and "including." The term "and / or," when used in a format such as A, B, and / or C, refers to any combination or subset of A, B, and C, for example, (1) A only, (2) B only, (3) C only, (4) A and B, (5) A and C, (6) B and C, (7) A, B, and C. When used herein in the context of describing a structure, component, item, object, and / or thing, the phrase "at least one of A and B" is intended to describe an implementation that includes either (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. Similarly, when used herein in the context of describing a structure, component, item, object, and / or thing, the phrase "at least one of A or B" is intended to describe an implementation that includes either (1) at least one A, (2) at least one B, or (3) at least one A and at least one B. When used herein in the context of describing the execution of a process, instruction, action, activity, and / or step, the phrase "at least one of A and B" is intended to describe an implementation that includes either (1) at least one A, (2) at least one B, or (3) at least one A and at least one B.Similarly, when used herein in the context of describing the performance of instructions, actions, activities, and / or steps, the phrase “at least one of A or B” is intended to describe an implementation that includes any of the following: (1) at least one A; (2) at least one B; or (3) at least one A and at least one B.

[0062] As used herein, singular terms (e.g., "a", "an", "first", "second") do not exclude a plurality. The term ("a" or "an") entity, as used herein, denotes one or more entities. The terms "a" (or "an"), "one or more", and "at least one" can be used interchangeably herein. Furthermore, although individually listed, the actions of a plurality of means, elements or methods may be performed by, for example, a single unit or processor. Furthermore, although individual features may be included in different examples or claims, these may be combinable and the inclusion in different examples or claims does not imply that a combination of features is not feasible and / or advantageous.

[0063] 2 is a flowchart representing example machine-readable instructions 200 that may be executed to implement an exemplary encoding system 101 to generate a bitstream having an SEI message that includes hashed atlas data that can be used by the encoding system 101 to ensure that the encoding system 101 is decoding properly. Although the instructions 200 are described in conjunction with the example encoding system 101 of FIG. 2, the instructions 200 may be described in conjunction with any type of encoding system.

[0064] In block 202, the atlas generator 102 determines whether input data has been received or acquired. The input data is image data corresponding to one or more views of an image to be displayed as part of a video. The input data can be acquired from a processor, a storage device, a sensor, a video generating device, etc. If the example atlas generator 102 determines that no input data has been received (block 202, NO), control returns to block 202 until input data is received (e.g., acquired). If the example atlas generator 102 determines that input data has been received (block 202, YES), the example atlas generator 102 generates atlas data, texture atlas data, depth atlas data, etc. (e.g., using a V3C or MIV encoder) based on the received input data (block 204). For example, the atlas generator 102 can generate atlas data using a two-dimensional array variable (e.g., BlockToPatchMap[y][x] defined above).

[0065] At block 206, the example hash generator 106 hashes the atlas data and includes the hashed atlas data in the SEI message. In some examples, the hash generator 106 also includes a hash protocol in the SEI message (e.g., so that the decoding system 110 can hash the bitstream using the same hash protocol). As mentioned above, the hash generator 106 can generate the SEI message and / or obtain the SEI message from another component and include (e.g., embed) the hashed atlas data in the SEI message. The hash generator 106 can generate a hash of the atlas data using a 16-byte MD5 hash, a cyclic redundancy check (CRC) hash, a checksum hash, and / or any other type of hash protocol. At block 208, the example volumetric video encoder 104 generates an encoded bitstream (e.g., encoded bitstream data) based on the atlas data, the texture atlas data, and / or the depth atlas data by encoding and / or compressing the atlas data, the texture atlas data, and / or the depth atlas data. The volumetric video encoder 104 may use any type of atlas encoding, video encoding technique, atlas compression, and / or video compression technique.

[0066] At block 210, the exemplary multiplexer 108 generates bitstream data by combining the encoded bitstream of the exemplary volumetric video encoder 104 and the SEI message including the hashed atlas data from the exemplary hash generator 106. The exemplary multiplexer 108 can combine the two signals using any type of multiplexing protocol. At block 212, the exemplary multiplexer 108 transmits the bitstream data to the decoding system 110. For example, the multiplexer 108 can output the bitstream data signal to an interface (e.g., an example of the interface 420 in FIG. 4) to the decoding system 110 via a wired or wireless connection.

[0067] 3 is a flowchart representing example machine-readable instructions 300 that may be executed to implement an example decoding system 110 to generate a bitstream that includes an SEI message that includes hashed atlas data that can be used by the decoding system 110 to ensure that the decoding system 110 is decoding correctly and / or that the bitstream does not contain errors. Although the instructions 300 will be described in conjunction with the example decoding system 110 of FIG. 1, the instructions 300 may be described in conjunction with any type of decoding system.

[0068] In block 302, the demultiplexer 112 determines whether bitstream data has been received or obtained from the encoding system 101. As described above, the bitstream data included the encoded video data and the corresponding SEI message. The multiplexer 112 can obtain the bitstream data from an interface (e.g., interface 520 of FIG. 5) that receives the bitstream via a wired or wireless connection. If the exemplary demultiplexer 112 determines that the bitstream data has not been received (block 302, NO), control returns to block 302 until the bitstream data is received (e.g., obtained). If the exemplary demultiplexer 112 determines that the bitstream data has been received (block 302, YES), the exemplary demultiplexer 112 demultiplexes (e.g., separates / parses) the bitstream data and separates the data bitstream and the SEI message from the bitstream data (block 304).

[0069] At block 306, the exemplary volumetric video decoder 114 decodes the bitstream data and uses the bitstream data to generate decoded atlas data, decoded texture atlas data, decoded depth atlas data, etc. (e.g., using a V3C or MIV decoder). The exemplary volumetric video decoder 114 decodes the data using a decoding technique that corresponds to the technique used to encode the data (e.g., the technique used by the volumetric video encoder 104). At block 308, the exemplary hash generator 118 hashes the decoded atlas data to generate a decoder-side atlas hash value. The exemplary hash generator 118 hashes the decoded atlas data using the same technique used to hash the atlas data in the encoding system 101. In this manner, if the decoder-side hash value matches the encoder-side hash value (e.g., from the obtained SEI message), the decoding system 110 determines that a decoding error has occurred. In some examples, the hash generator 118 extracts a hash protocol from the SEI message.

[0070] At block 310, the exemplary comparator 120 extracts or obtains an encoder-side atlas hash value from the SEI message. As previously described, the encoding system 101 performs one or more hash operations on the atlas data during the encoding process and transmits the encoder-side hashed atlas as part of the SEI message. At block 312, the exemplary comparator 120 compares the encoder-side atlas hash value (e.g., determined at the encoding system 101 and extracted from the SEI message) with the decoder-side atlas hash value (e.g., a hash value generated by the hash generator 118 at the decoding system 110). As described above, the exemplary comparator 120 compares the encoder-side atlas hash value with the decoder-side atlas value to determine whether correct transfer of the bitstream and / or decoding of the bitstream has occurred.

[0071] At block 314, the exemplary comparator 120 determines whether the encoder-side atlas hash value is different from the decoder-side atlas hash value. Since the encoder-side atlas hash value and the decoder-side atlas hash value are hashed using the same hash protocol and the atlas data are supposed to be the same, the encoder-side hash value and the decoder-side hash value will be the same unless there is a transmission or decoding error. Thus, if the exemplary comparator 120 determines that the encoder-side atlas hash value is different from the decoder-side atlas hash value (block 314, YES), the exemplary comparator 120 flags the mismatch (e.g., outputs a value indicative of the mismatch) and / or executes instructions based on the mismatch (block 316) because the decoding system 110 determines that there was a transmission and / or decoding error when the two hashed atlas values ​​are different. For example, the output of the comparator 120 can reflect the mismatch in a user interface to inform the user of the decoding error. The error flag may be sent to another processor or component to perform an action corresponding to the mismatch. In some examples, an error flag may be output to the data reconstructor 116 and / or another device to cause the corresponding data to be discarded. In some examples, the comparator 120, the data reconstructor 115, and / or another device may determine whether the obtained bitstream data and / or the SEI message includes a value indicating persistence. By way of example, if the comparator 120, the data reconstructor 115, and / or another device determines that the obtained bitstream data includes a value indicating persistence, the comparator 120 device flags the corresponding access unit corresponding to the bitstream as having an error.

[0072] If the comparator 120 determines that the encoder-side atlas hash value does not differ from the decoder-side atlas hash value (block 314, NO), control ends because the decoding system 110 determines that the bitstream was correctly decoded. In some examples, the output of the comparator 120 is sent to the data reconstructor 116. In this manner, the data reconstructor 116 can determine whether to discard the decoded atlas data and / or reconstruct the decoded atlas data into a reconstructed view based on the result of the comparator 120 (e.g., the onset of a match or mismatch).

[0073] Figure 4 is a block diagram of an exemplary processor platform 400 configured to execute the instructions of Figure 2 to implement the encoding system 101 of Figure 1. The processor platform 400 may be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., neural network), a mobile device (e.g., a mobile phone, a smart phone, a tablet such as an iPad), a personal digital assistant (PDA), an Internet appliance, a DVD player, a CD player, a digital video recorder, a Blu-ray player, a gaming console, a personal video recorder, a set-top box, a headset, or other wearable device, or any other type of computing device.

[0074] The processor platform 400 of the illustrated example includes a processor 412. The processor 412 of the illustrated example is hardware. For example, the processor 412 can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. A hardware processor can be a semiconductor-based (silicon-based) device. In this example, the processor implements the atlas generator 102, volumetric video encoder 104, hash generator 106, and multiplexer 108 of FIG. 1.

[0075] The processor 412 of the illustrated example includes a local memory 413 (e.g., a cache). The processor 412 of the illustrated example communicates with a main memory, including a volatile memory 414 and a non-volatile memory 416, via a bus 418. The volatile memory 414 may be implemented with Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS®, Dynamic Random Access Memory (RDRAM®), and / or any other type of random access memory device. The non-volatile memory 416 may be implemented with flash memory and / or any other desired type of memory device. Access to the main memory 414, 416 is controlled by a memory controller.

[0076] The processor platform 400 of the illustrated example also includes an interface circuit 420. The interface circuit 420 may be implemented with any type of interface standard, such as an Ethernet interface, a Universal Serial Bus (USB), a Bluetooth interface, a near field communication (NFC) interface, and / or a PCI Express interface.

[0077] In the depicted example, one or more input devices 422 are coupled to interface circuitry 420. The input devices 422 allow a user to input data and / or commands to processor 412. The input devices may be implemented, for example, by audio sensors, microphones, cameras (still or video), keyboards, buttons, mice, touch screens, track pads, track balls, isopoints, and / or voice recognition systems.

[0078] One or more output devices 424 are also connected to the interface circuitry 420 of the illustrated example. The output device(s) 424 may be implemented by, for example, a display device (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube display (CRT), an in-place switching (IPS) display, a touch screen, etc.), a tactile output device, a printer, and / or a speaker. The interface circuitry 420 of the illustrated example thus typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0079] The interface circuitry 420 of the illustrated example also includes communications devices such as transmitters, receivers, transceivers, modems, home gateways, wireless access points, and / or network interfaces to facilitate data exchange with external devices (e.g., any type of computing device) over a network 426. Communications can be via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a high- and low-line radio system, a cellular telephone system, etc.

[0080] The processor platform 400 of the depicted example also includes one or more mass storage devices 428 for storing software and / or data. Examples of such mass storage devices 428 include a floppy disk drive, a hard drive disk, a compact disk drive, a Blu-ray disk drive, a redundant array of independent disks (RAID) system, and a digital versatile disk (DVD) drive.

[0081] The machine-executable instructions 432 of FIG. 2 may be stored in the mass storage device 428, in the volatile memory 414, in the non-volatile memory 416, and / or in a removable non-transitory computer-readable storage medium such as a CD or DVD.

[0082] Figure 5 is a block diagram of an exemplary processor platform 500 configured to execute the instructions of Figure 3 to implement the decoding system 110 of Figure 1. The processor platform 500 may be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., neural network), a mobile device (e.g., a mobile phone, a smart phone, a tablet such as an iPad), a personal digital assistant (PDA), an Internet appliance, a DVD player, a CD player, a digital video recorder, a Blu-ray player, a gaming console, a personal video recorder, a set-top box, a headset, or other wearable device, or any other type of computing device.

[0083] The illustrated example processor platform 500 includes a processor 512. The illustrated example processor 512 is hardware. For example, the processor 512 may be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. A hardware processor may be a semiconductor-based (silicon-based) device. In this example, the processor implements the example demultiplexer 112, the example volumetric video decoder 114, the example data reconstructor 116, the example hash generator 118, and the example comparator 120 of FIG. 1.

[0084] The processor 512 of the illustrated example includes a local memory 513 (e.g., a cache). The processor 512 of the illustrated example communicates with a main memory, including a volatile memory 514 and a non-volatile memory 516, via a bus 518. The volatile memory 514 may be implemented with Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), RAMBUS®, Dynamic Random Access Memory (RDRAM®), and / or any other type of random access memory device. The non-volatile memory 516 may be implemented with flash memory and / or any other desired type of memory device. Access to the main memory 514, 516 is controlled by a memory controller.

[0085] The processor platform 500 of the illustrated example also includes an interface circuit 520. The interface circuit 520 may be implemented with any type of interface standard, such as an Ethernet interface, a Universal Serial Bus (USB), a Bluetooth® interface, a near field communication (NFC) interface, and / or a PCI Express interface.

[0086] In the depicted example, one or more input devices 522 are coupled to interface circuitry 520. The input devices 522 allow a user to input data and / or commands to processor 512. The input devices may be implemented, for example, by audio sensors, microphones, cameras (still or video), keyboards, buttons, mice, touch screens, track pads, track balls, isopoints, and / or voice recognition systems.

[0087] One or more output devices 524 are also connected to the interface circuitry 520 of the illustrated example. The output device(s) 524 may be implemented by, for example, a display device (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube display (CRT), an in-place switching (IPS) display, a touch screen, etc.), a tactile output device, a printer, and / or a speaker. The interface circuitry 520 of the illustrated example thus typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0088] The interface circuitry 520 of the depicted example also includes communications devices, such as transmitters, receivers, transceivers, modems, home gateways, wireless access points, and / or network interfaces, to facilitate data exchange with external devices (e.g., any type of computing device) over the network 526. Communications can be via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a high- and low-line radio system, a cellular telephone system, etc.

[0089] The processor platform 500 of the depicted example also includes one or more mass storage devices 528 for storing software and / or data. Examples of such mass storage devices 528 include a floppy disk drive, a hard drive disk, a compact disk drive, a Blu-ray disk drive, a redundant array of independent disks (RAID) system, and a digital versatile disk (DVD) drive.

[0090] The machine-executable instructions 532 of FIG. 3 may be stored in the mass storage device 528, in the volatile memory 514, in the non-volatile memory 516, and / or in a removable non-transitory computer-readable storage medium such as a CD or DVD.

[0091] Exemplary methods, devices, systems, and articles of manufacture for identifying video decoding errors are disclosed herein. Further examples and combinations thereof include the following. Example 1 is a video encoding device, comprising: an atlas generator for generating atlas data for one or more atlases generated from an input view of the video; 1. A hash generator, comprising: performing a hash operation on the atlas data to generate a hash value; a hash generator for including said hash value in a message; a multiplexer for combining the one or more atlases, coding atlas data corresponding to the atlas data, and the message to generate a video bitstream; This includes equipment including:

[0092] Example 2 includes the apparatus of example 1, further including an interface for transmitting the bitstream data to a decoding device.

[0093] Example 3 includes the apparatus of example 1, wherein the atlas data includes a variable two-dimensional array corresponding to a block-to-patch map of the input views.

[0094] Example 4 includes the apparatus of example 1, wherein the atlas data corresponds to a mapping of blocks to corresponding patches.

[0095] Example 5 is a diagram illustrating a configuration of the atlas generator. Transforming the input view of the video into at least one of a texture atlas or a depth atlas; The apparatus of Example 1 further includes an apparatus for encoding at least one of the texture atlas, the depth atlas, or the atlas data to generate encoded bitstream data.

[0096] Example 6 includes the device of example 1, wherein the hash generator at least one of generates the message or obtains the message from another device.

[0097] Example 7 includes the device of example 1, wherein the hash generator hashes the atlas data using a 16-byte Message Digest Algorithm 5 (MD5) sum.

[0098] Example 8 is a non-transitory computer-readable storage medium including instructions that, when executed, cause one or more processors to perform at least: generating atlas data for one or more atlases generated from input views of the video; performing a hash operation on the atlas data to generate a hash value; Including said hash value in a message; combining the one or more atlases, coding atlas data corresponding to the atlas data, and the message to generate a video bitstream. Includes a computer readable storage medium.

[0099] Example 9 includes the computer-readable storage medium of example 8, wherein the instructions cause the one or more processors to transmit the bitstream data to a decoding device.

[0100] Example 10 includes the computer-readable storage medium of example 8, wherein the atlas data is a variable two-dimensional array corresponding to a block-to-patch map of the input views.

[0101] Example 11 includes the computer-readable storage medium of example 8, wherein the atlas data corresponds to a mapping of blocks in an atlas space to corresponding patches.

[0102] Example 12 is a block diagram of a method for executing a program that includes the steps of: Transforming the input view of the video into at least one of a texture atlas or a depth atlas; 9. The computer-readable storage medium of example 8, further comprising: encoding at least one of the texture atlas, the depth atlas, or the atlas data to generate encoded bitstream data.

[0103] Example 13 is a block diagram of a method for executing a program on the one or more processors, the method comprising: Example 9 includes a computer-readable storage medium for causing a computer to perform at least one of: generating the message; or obtaining the message from another device.

[0104] Example 14 is a block diagram of a method for executing a program that includes the steps of: Example 9. The computer-readable storage medium of Example 8, further comprising hashing the atlas data using a 16-byte Message Digest Algorithm 5 (MD5) sum.

[0105] Example 15 is a video encoding method, comprising: generating atlas data for one or more atlases generated from input views of the video by executing instructions by a processor; executing instructions by the processor to perform a hashing operation on the atlas data to generate a hash value; including the hash value in a message by executing instructions with the processor; combining, by executing instructions by the processor, the one or more atlases, coding atlas data corresponding to the atlas data, and the message to generate a video bitstream; The method includes the steps of:

[0106] Example 16 includes the method of example 15, further including transmitting the bitstream data to a decoding device.

[0107] Example 17 includes the method of example 15, wherein the atlas data is a variable two-dimensional array corresponding to a block-to-patch map of the input views.

[0108] Example 18 includes the method of example 15, wherein the atlas data corresponds to a mapping of blocks to corresponding patches.

[0109] Example 19 includes converting the input view of the video into at least one of a texture atlas or a depth atlas; encoding at least one of the texture atlas, the depth atlas, or the atlas data to generate encoded bitstream data; The method of Example 15 further comprises:

[0110] Example 20 includes the method of example 15, further including at least one of generating the message or obtaining the message from another device.

[0111] Example 21 includes the apparatus of example 15, further including hashing the atlas data using a 16-byte Message Digest Algorithm 5 (MD5) sum.

[0112] Example 22 is a video decoding device, a hash generator that performs a hashing operation on atlas data to determine a first hash value of the atlas data, the atlas data being decoded from a volumetric video bitstream; a comparator for comparing the first hash value with a second hash value of the atlas data, the second hash value being included in the video bitstream, and generating an error flag when the first hash value does not match the second hash value.

[0113] Example 23 includes the device of example 22, further including a demultiplexer that obtains the video bitstream and separates the atlas data and messages from the bitstream.

[0114] Example 24 includes the device of example 23, wherein the message includes the second hash value.

[0115] Example 25 includes the device of example 22, wherein the hash generator generates the first hash value using a same hash operation used to generate the second hash value.

[0116] Example 26 includes the device of example 22, wherein the demultiplexer obtains the bitstream from an encoder via a network interface.

[0117] Example 27 includes the device of example 22, wherein the error flag is for identifying that an error occurred in at least one of transmitting the video bitstream or decoding the bitstream.

[0118] Example 28 includes the apparatus of example 22, wherein the error flag is a first error flag corresponding to a first access unit, and the comparator generates a second error flag for a second access unit in the bitstream when the message indicates persistence.

[0119] Example 29 includes the apparatus of example 22, wherein the error flag causes at least one of the error to be displayed on a user interface, the video bitstream to be discarded, or a new copy of the video bitstream to be requested.

[0120] Example 30 includes the apparatus of example 22, wherein the atlas data includes patch indexes for blocks of data.

[0121] Example 31 is a non-transitory computer-readable storage medium including instructions that, when executed, cause one or more processors to: performing a hashing operation on the atlas data to determine a first hash value of the atlas data, the atlas data being decoded from the volumetric video bitstream; The present invention includes a computer-readable storage medium that can cause the first hash value to be compared with a second hash value of the atlas data, the second hash value being included in the video bitstream, and an error flag to be generated when the first hash value does not match the second hash value.

[0122] Example 32 is a block diagram of a method for executing a program that includes the steps of: Example 32. The computer-readable storage medium of Example 31, further comprising: a computer-readable storage medium for causing a computer to perform at least one of: obtaining the video bitstream; and separating the atlas data and messages from the bitstream.

[0123] Example 33 includes the computer-readable storage medium of example 32, wherein the message includes the second hash value.

[0124] Example 34 includes the computer-readable storage medium of example 31, wherein the instructions cause the one or more processors to generate the first hash value using at least the same hash operation used to generate the second hash value.

[0125] Example 35 includes the computer-readable storage medium of example 31, wherein the instructions cause the one or more processors to at least obtain the bitstream from an encoder via a network interface.

[0126] Example 36 includes the computer-readable storage medium of Example 31, wherein the error flag is for identifying that an error has occurred in at least one of transmitting the video bitstream or decoding the bitstream.

[0127] Example 37 includes the computer-readable storage medium of Example 31, wherein the error flag is a first error flag corresponding to a first access unit, and the instructions cause the one or more processors to generate a second error flag for a second access unit in the bitstream when the message indicates persistence.

[0128] Example 38 includes the computer-readable storage medium of Example 31, wherein the error flag causes at least one of an error to be displayed on a user interface, the video bitstream to be discarded, or a new copy of the video bitstream to be requested.

[0129] Example 39 includes the computer-readable storage medium of example 31, wherein the atlas data includes patch indexes for blocks of data.

[0130] Example 40 is a video decoding method, comprising: executing instructions by a processor to perform a hashing operation on atlas data to determine a first hash value of the atlas data, the atlas data being decoded from a video bitstream; comparing the first hash value with a second hash value of the atlas data by executing instructions by the processor, the second hash value being included in the video bitstream; and generating an error flag when the first hash value does not match the second hash value by executing instructions by the processor.

[0131] Example 41 includes the method of example 40, further including obtaining the video bitstream and separating the atlas data and messages from the bitstream.

[0132] Example 42 includes the method of example 41, wherein the message includes the second hash value.

[0133] Example 43 includes the method of example 40, further including generating the first hash value using a same hash operation used to generate the second hash value.

[0134] Example 44 includes the method of example 40, further including obtaining the bitstream from an encoder via a network interface.

[0135] Example 45 includes the method of example 40, wherein the error flag is for identifying that an error occurred in at least one of transmitting the video bitstream or decoding the bitstream.

[0136] Example 46 includes the method of example 40, further including the step of generating a second error flag for a second access unit in the bitstream when the error flag is a first error flag corresponding to a first access unit and the message indicates persistence.

[0137] Example 47 includes the method of example 40, wherein the error flag causes at least one of the error to be displayed on a user interface, the video bitstream to be discarded, or a new copy of the video bitstream to be requested.

[0138] Example 48 includes the method of example 40, wherein the atlas data includes patch indexes for blocks of data.

[0139] From the foregoing, it should be appreciated that exemplary methods, apparatus, and articles of manufacture are disclosed for identifying video decoding errors. The disclosed methods, apparatus, and articles of manufacture increase the efficiency of using a computing device by identifying transmission and / or decoding errors corresponding to the transmission and / or decoding of video bitstream data. In this manner, the errors may be handled based on user preferences. The disclosed methods, apparatus, and articles of manufacture are therefore contemplated for one or more improvements in the functionality of a computer.

[0140] Although certain exemplary methods, apparatus, and articles of manufacture have been disclosed herein, the coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus, and articles of manufacture fairly falling within the scope of the claims of this patent.

Claims

1. 1. A video encoding device, comprising: an atlas generator for generating atlas data for one or more atlases generated from an input view of the video; 1. A hash generator, comprising: performing a hash operation on the atlas data to generate a hash value; a hash generator for including said hash value in a message; a multiplexer for combining the one or more atlases, coding atlas data corresponding to the atlas data, and the message to generate a video bitstream; Including equipment.

2. The apparatus of claim 1 , further comprising an interface for transmitting the bitstream data to a decoding device.

3. The apparatus of claim 1 , wherein the atlas data comprises a variable two-dimensional array corresponding to a block-to-patch map of the input views.

4. An apparatus according to any one of claims 1 to 3, wherein the atlas data corresponds to a mapping of blocks to corresponding patches.

5. The atlas generator includes: Transforming the input view of the video into at least one of a texture atlas or a depth atlas; 5. An apparatus according to claim 1, further comprising: an at least one of said texture atlas, said depth atlas, or said atlas data encoding means for generating encoded bitstream data.

6. The apparatus of any one of claims 1 to 5, wherein the hash generator at least one of generates the message or obtains the message from another device.

7. The apparatus of any one of claims 1 to 6, wherein the hash generator hashes the atlas data using a 16-byte Message Digest Algorithm 5 (MD5) sum.

8. A non-transitory computer-readable storage medium containing instructions that, when executed, cause one or more processors to perform at least: generating atlas data for one or more atlases generated from input views of the video; performing a hash operation on the atlas data to generate a hash value; Including said hash value in a message; combining the one or more atlases, coding atlas data corresponding to the atlas data, and the message to generate a video bitstream. A computer-readable storage medium.

9. The computer-readable storage medium of claim 8 , wherein the instructions cause the one or more processors to transmit the bitstream data to a decoding device.

10. The computer-readable storage medium of any one of claims 8 to 9, wherein the atlas data is a variable two-dimensional array corresponding to a block-to-patch map of the input views.

11. The computer-readable storage medium of any one of claims 8 to 10, wherein the atlas data corresponds to a mapping of blocks in an atlas space to corresponding patches.

12. The instructions cause the one or more processors to: Transforming the input view of the video into at least one of a texture atlas or a depth atlas; 12. The computer-readable storage medium of claim 8, further comprising: encoding at least one of the texture atlas, the depth atlas, or the atlas data to generate encoded bitstream data.

13. The instructions cause the one or more processors to: The computer-readable storage medium according to any one of claims 8 to 12, further configured to perform at least one of: generating the message; or obtaining the message from another device.

14. The instructions cause the one or more processors to: The computer-readable storage medium of any one of claims 8 to 13, wherein the atlas data is hashed using a 16-byte Message Digest Algorithm 5 (MD5) sum.

15. 1. A video encoding method comprising the steps of: generating atlas data for one or more atlases generated from input views of the video by executing instructions by a processor; executing instructions by the processor to perform a hashing operation on the atlas data to generate a hash value; including the hash value in a message by executing instructions with the processor; combining, by executing instructions by the processor, the one or more atlases, coding atlas data corresponding to the atlas data, and the message to generate a video bitstream; The method includes:

16. 16. The method of claim 15, further comprising the step of transmitting the bitstream data to a decoding device.

17. The method of any one of claims 15 to 16, wherein the atlas data is a variable two-dimensional array corresponding to a block-to-patch map of the input views.

18. The method according to any one of claims 15 to 17, wherein the atlas data corresponds to a mapping of blocks to corresponding patches.

19. converting the input view of the video into at least one of a texture atlas or a depth atlas; encoding at least one of the texture atlas, the depth atlas, or the atlas data to generate encoded bitstream data; The method of any one of claims 15 to 18, further comprising:

20. The method according to any one of claims 15 to 19, further comprising at least one of the steps of generating the message or obtaining the message from another device.

21. The method of any one of claims 15 to 20, further comprising the step of hashing the atlas data using a 16-byte Message Digest Algorithm 5 (MD5) sum.

22. 1. A video encoding device, comprising: Memory, With orders, a processor circuit that executes the instructions, generating atlas data for one or more atlases generated from the input views of the video; performing a hash operation on the atlas data to generate a hash value; a processor circuit for including the hash value in a message; a multiplexer for combining the one or more atlases, coding atlas data corresponding to the atlas data, and the message to generate a video bitstream; Including equipment.

23. 23. The apparatus of claim 22, further comprising an interface for transmitting the bitstream data to a decoding device.

24. An apparatus according to any one of claims 22 to 23, wherein the atlas data comprises a variable two-dimensional array corresponding to a block-to-patch map of the input views.

25. An apparatus according to any one of claims 22 to 24, wherein the atlas data corresponds to a mapping of blocks to corresponding patches.

Images