Method and apparatus for identifying video decoding errors

Abstract

Example methods, apparatus, systems, and articles of manufacture to identify video decoding errors are disclosed. An example apparatus includes an atlas generator to generate atlas data for one or more atlas generated from an input view of a video; a hash generator for performing the following operations: performing a hash operation on the atlas data to generate a hash value; and including the hash value in the message; and a multiplexer to combine the one or more atlas, the encoded atlas data corresponding to the atlas data, and the message to generate a video bitstream.

Description

[0001] Related applications

[0002] This patent claims the benefit of U.S. Provisional Application No. 63 / 004,741, filed April 3, 2020. The entirety of U.S. Provisional Application No. 63 / 004,741 is incorporated herein by reference. Priority of U.S. Provisional Application No. 63 / 004,741 is hereby claimed. Technical Field

[0003] This disclosure relates generally to video processing, and more specifically to methods and apparatus for identifying video decoding errors. Background Technology

[0004] In video compression / decompression (codec) systems, compression efficiency and video quality are important performance metrics. For example, visual quality is a crucial aspect of user experience in many video applications. Compression efficiency affects the amount of memory required to store video files and / or the amount of bandwidth required to transmit and / or stream video content. Video encoders typically compress video information so that more information can be transmitted over a given bandwidth or stored in a given memory space, and so on. The compressed signal or data is then decoded by a decoder, which decodes or decompresses the signal or data to display it to the user. In most cases, higher visual quality with greater compression is desirable.

[0005] Currently, standards are being developed for immersive video coding and point cloud coding, including Video-based Point Cloud Compression (V-PCC) and MPEG Immersive Video Coding (MIV). These standards attempt to establish and improve compression efficiency and reconstruction quality in the context of immersive video and point cloud coding. Attached Figure Description

[0006] Figure 1 This is an example environment for video encoding and / or decoding, combined with the examples disclosed in this article.

[0007] Figure 2 It represents something that can be executed to achieve. Figure 1 A flowchart of machine-readable instructions for an example encoding system.

[0008] Figure 3 It represents something that can be executed to achieve. Figure 1 The flowchart shows an example of machine-readable instructions for a decoding system.

[0009] Figure 4 It is constructed to execute Figure 3 Instructions to achieve Figure 1 A block diagram of an example processing platform for an encoding system.

[0010] Figure 5 It is constructed to execute Figure 3 Instructions to achieve Figure 1 A block diagram of an example processing platform for a decoding system.

[0011] The accompanying drawings are not to scale. Instead, the thickness of layers or regions may be enlarged in the drawings. Generally, the same reference numerals will be used throughout the various drawings and accompanying written descriptions to refer to the same or similar components. As used in this patent, a statement that any component (e.g., layer, film, region, area, or plate) is on another component in any way (e.g., positioned thereon, located on thereon, arranged thereon, or formed thereon, etc.) indicates that the mentioned component is either in contact with the other component or that the mentioned component is above the other component with one or more intermediate components located therebetween. References to connections (e.g., attachment, coupling, joining, and joining) should be interpreted broadly and may include intermediate members between sets of elements and relative movement between elements, unless otherwise indicated. Therefore, a reference to a connection does not necessarily imply that two elements are directly connected and have a fixed relationship with each other. A statement that any component is “in contact” with another component means that there is no intermediate component between the two components. Although layers and regions with clearly defined lines and boundaries are shown in the drawings, some or all of these lines and / or boundaries may be idealized. In reality, boundaries and / or lines can be imperceptible, mixed, and / or irregular.

[0012] In this document, descriptive terms such as "first," "second," "third," etc., are used to identify multiple elements or components that can be mentioned separately. Unless otherwise specified or understood based on the context of their use, these descriptive terms are not intended to imply any priority, physical order, or arrangement in a list or chronological order, but are merely used as labels to separately refer to multiple elements or components for ease of understanding of the disclosed examples. In some examples, the descriptive term "first" may be used to refer to an element in a detailed description, while the same element may be referred to in the claims using different descriptive terms, such as "second" or "third." In such cases, it should be understood that these descriptive terms are used merely for ease of reference to multiple elements or components. Detailed Implementation

[0013] In the context of immersive video coding and point cloud coding, video standards such as Visual Volumetric Video-based Coding (V3C) and Video-based Point Cloud Compression (V-PCC), as well as MPEG Immersive Video Coding (MIV), can be utilized. For such standards, there may be requirements to verify that the decoder conforms to them. For example, there may be one or more requirements that the decoder obtains an unaltered and / or undamaged bitstream, and / or that the decoder correctly decodes the obtained bitstream.

[0014] In V3C / V-PCC, dynamic point clouds are compressed using multiple projections of the point cloud onto planes used for texture (e.g., color) and geometry (e.g., depth). After compression, the compressed dynamic point cloud is segmented to extract rectangular regions of similar depth from the projection planes; these are called patches. These patches are packaged into atlases or atlas tiles with an occupancy map. The occupancy map indicates the portion of the atlas to be used (e.g., the occupied area within a patch packaged in the atlas). Furthermore, patch information metadata is used to indicate how the patches are mapped between the projection planes and the canvas (e.g., the atlas or atlas tiles). Two-dimensional (2D) video codecs, such as High Efficiency Video Coding (HEVC), are used to leverage the spatial and temporal redundancy of the canvas's geometric and textural components. Information about the size and location of patches in the original projections and in the atlas or atlas tiles is indicated as encoded atlas tile data. Occupancy information is also indicated. The atlas can be subdivided into a grid of non-overlapping data blocks, and the block size can be included in the bitstream. The atlas data includes a patch value indicating each block (e.g., a patch identifier and / or index of the data block). This block can correspond to a patch, or it can be a subset of patches. In some examples, patches overlap with other patches. The order in which the patch data is identified in the bitstream (e.g., corresponding to patch identifiers) is used to determine patch priority at a particular block location; for example, patches with higher patch identifier values ​​may have priority.

[0015] When encoded via V3C / V-PCC and MIV standards, rectangular patches are formed by projected views and arranged into atlases. Patches may overlap within the atlas, and the last patch labeled has priority in resolving overlaps. These standards define a labeling mechanism to send encoded atlas data for each patch in the atlas, including information such as the patch's size and position in its corresponding projected view and / or patch identifier. Atlas data is a mapping between data blocks and patches in each atlas. In some examples, such information is included within the encoded atlas tile data syntax structure. Encoded atlas data is a per-patch label. Encoded atlas data can be decoded by a decoder to obtain atlas data. In some examples, the encoded atlas tile data syntax structure is labeled in each frame time or instance. In some examples, the encoded atlas tile data syntax structure persists across multiple consecutive frame times or instances. Multiple encoded atlas tile data syntax structures corresponding to different atlases or atlas tiles can be identified within the same access unit (a portion of a bitstream of components (e.g., atlas data, texture video, geometric video, occupancy video, etc.) with the same decoding order count (e.g., data of a V-PCC / MIV bitstream corresponding to a specific time instance)). All of these correspond to the same frame time, instance, or Picture Order Count (POC) value. For example, a standard could define an operation to compute atlas data (e.g., a variable 2D array, BlockToPatchMap[y][x] (also known as DecoderToPatchMap[y][x])) that represents a mapping from blocks in the atlas space to corresponding patches, where x and y are block coordinates at the block size granularity. Encoded atlas data is included in the bitstream from encoder to decoder.

[0016] The MIV standard uses a similar approach to V3C / V-PCC for encoding multiple image views, similar to how V3C / V-PCC uses its multiple projections. For example, multiple atlases can be used in MIV. In MIV, arranging patches into atlases can persist across multiple frame times. Conformity checks can be applied to multiple atlases, as well as the persistence of the atlases. Persistence corresponds to how long the data corresponding to the atlas can persist before being replaced by additional data.

[0017] The examples disclosed herein provide supplemental enhancement information (SEI) messages that include atlas data hashes. Such messages can be implemented in the MIV standard, the V3C / V-PCC standard, and / or any other standard that includes messages in the generated bitstream. Atlas data hashes (also referred to as hashed atlas data) are the result of a hash operation performed on atlas data. The examples disclosed herein include the atlas data hashes in the SEI message and include the SEI message along with the encoded atlas data in the bitstream. The decoder uses the hashed atlas values ​​of the SEI message to help the decoder identify transmission and / or decoding errors, as further described below.

[0018] As described above, the examples disclosed herein include hashed atlas data (e.g., hash values ​​for each atlas indicating which view or projection direction a patch represents) along with the corresponding encoded atlas data in a Supplemental Enhancement Information (SEI) message within the bitstream. The hashed atlas data may include a per-atlas hash for the 2D array variable BlockToPatchMap[y][x]. For example, a hash function may be applied to scanned (e.g., raster-scanned) atlas data such that the atlas data refers to a 2D data structure that indicates a value representing a patch identifier (e.g., a number) or view identifier corresponding to a particular sample. In some examples, the encoder determines the value of BlockToPatchMap[y][x] for each atlas at each access unit where patch data is identified in the encoded atlas tile data. The hash function may be implemented as one or more hash operations. In some examples, the encoder generates hash values ​​for the entire 2D array and identifies the atlas data hash values ​​in the SEI message.

[0019] In the example disclosed herein, when the decoder receives the encoded bitstream, it separates the SEI message from the bitstream. The decoder determines the atlas hash value from the SEI message. Furthermore, the decoder decodes the encoded atlas data (e.g., the values ​​of a 2D array of BlockToPatchMap[y][x]) for each encoded atlas in each access unit to generate decoded atlas data. After decoding the encoded atlas data, the decoder generates a hash value for the decoded atlas data (e.g., a 2D array of BlockToPatchMap[y][x]) using the same hashing technique implemented by the encoder, to generate a decoder-side hash value. In this way, the decoder compares the decoder-side hash value with the hash value indicated in the SEI message. If the values ​​match, the decoder verifies with high confidence that the bitstream has not been corrupted and that the decoder's decoding operation on the encoded atlas tile data conforms to specifications (e.g., verifying that the decoder is decoding correctly). If the values ​​do not match, the decoder can determine that a decoding error has occurred (e.g., the bitstream has been corrupted and / or the decoder operation does not conform to specifications in decoding the encoded atlas tile data).

[0020] As specified herein, decoded atlas data hashes and atlas data hashes are generated for use in video encoding and for standardized codecs, such as the V3C / V-PCC standard and / or the MIV standard. SEI messages can identify the hash value for each atlas data instance (e.g., for each unique 2D atlas array).

[0021] Figure 1 This is an example environment 100, which includes an example block diagram of an example encoding system 101 and an example decoding system 110. The example encoding system 101 includes an example atlas generator 102, an example volumetric video encoder 104, an example hash generator 106, and an example multiplexer 108. The example decoding system 110 includes an example demultiplexer 112, an example volumetric video decoder 114, an example data reconstructor 116, an example hash generator 118, and an example comparator 120.

[0022] Figure 1 The example encoding system 101 (also called an encoder, encoding device, etc.) and the example decoding system 110 can be implemented in any device with suitable form factor or one or more such devices, including server computers, cloud computing environments, personal computers, laptop computers, tablet devices, phablets, smartphones, game consoles, wearable devices, display devices, all-in-one devices, 2-in-1 devices, etc. Although Figure 1 The example encoding system 101 and the example decoding system 110 are implemented in different computing devices, but the example encoding system 101 and the decoding system 110 can be implemented in the same device.

[0023] Figure 1 The atlas generator 102 of the encoding system 101 receives multiple views of the input video for encoding. Any number of views of the scene can exist, and each of these views may include a corresponding texture image or frame (or texture video) and a corresponding geometry image or frame (or geometry video). For each time instance, video data of one or more views of the scene is provided so that multiple texture frames and multiple depth or geometry frames can be generated. For example, each texture frame may include color information per pixel, such as luminance / color (YUV) data, red-green-blue (RGB) data, or similar data, and each geometry or depth frame may include depth data per pixel or similar data. Texture and geometry frames may have the same or different resolutions. For example, a texture frame may be a video graphics array (VGA), high-definition (HD), full high-definition (e.g., 1080p), 4K resolution video, 8K resolution video, and so on. The examples disclosed herein are described with respect to frames and images (which are interchangeable).

[0024] Figure 1 Atlas generator 102, for a specific frame index or instance, generates (e.g., converted from an input view) texture atlases and geometry or depth atlases. For example, atlas generator 102 breaks down each of multiple views into patches (e.g., rectangular regions), which are arranged to form an atlas. Atlas generator 102 can format the atlas to reduce redundancy between multiple views. Example: Atlas generator 102 forms texture atlases (including texture information) and geometry or depth atlases (including geometry or depth information) and provides this data to volumetric video encoder 104 of encoding system 101.

[0025] Figure 1The example atlas generator 102 also determines atlas data for a specific frame index or instance, which includes metadata indicating the source of each patch in the texture and depth atlas (e.g., the corresponding view associated with that patch). In some examples, the atlas generator 102 generates a single instance of atlas data for the texture and depth atlases such that patches in the depth atlas follow or match patches in the texture atlas, and vice versa. The atlas data provides metadata for reconstructing multiple views (or portions thereof) using texture and depth patch data from the atlas. As used herein, the term atlas data refers to any data structure that indicates the source of patches in the texture and depth atlases, such as a 2D array of values ​​indicating the source of such patches. In some examples, the atlas data includes a 2D array of pixel-level values, each value representing a patch identifier (or a default value if a pixel does not correspond to a patch, e.g., as the highest available value in the data structure)). In some examples, atlas generator 102 generates atlas data as a 2D array comprising pixel-level values, each value indicating the source view of the pixel (or the default if the pixel does not correspond to a source view). In some examples, atlas generator 102 generates atlas data as a 2D array comprising pixel-level values, each value indicating the source view and patch number of the pixel (or the default if the pixel does not correspond to a source view). In some examples, the 2D data array can be represented in a 1D format (e.g., by listing all the data of the 2D array as a 1D vector), such as that created in a raster scan operation. In some examples, atlas generator 102 generates atlas data using the DecoderToPatchMap[y][x] syntax element, as further described below.

[0026] In some examples, atlas generator 102 generates atlas data instances, as shown in the pseudocode provided in Table 1 below. In the pseudocode of Table 1, the atlas data is DecoderToPatchMap[y][x] (also known as BlockToPatchMap[y][x]).

[0027] Table 1 - Pseudocode (1)

[0028]

[0029]

[0030] In the pseudocode (1) above in Table 1, the first part of the code defines the width and height of the BlockToPatchMap 2D array based on the size of the coded atlas tiles and the block size. The second part of pseudocode (1) sets an initial value (e.g., -1) for the block to indicate that the block is not occupied. The third part of pseudocode (1) sets the patch index value for the block covered by the patch in the BlockToPatchMap 2D array, using the x and y positions of the block as well as the patch width and patch height, for patches in the coded atlas data.

[0031] In some examples, atlas generator 102 generates atlas data instances (e.g., patch indices for each patch), as shown in the pseudocode provided in Table 2. In the pseudocode of Table 2, the atlas data is again DecoderToPatchMap[y][x]. In some examples, the format of BlockToPatchMap is assigned as 16-bit unsigned, and the default value for each sample of BlockToPatchMap[y][x]—which is maintained if no patch overrides the value—is changed from -1 to 0xFFFF (or 65535) relative to the pseudocode in Table 1.

[0032] Table 2 - Pseudocode (2)

[0033]

[0034]

[0035] In the pseudocode (2) above in Table 2, the first part of the code defines the width and height of the BlockToPatchMap 2D array based on the size of the coded atlas tiles and the block size. The second part of pseudocode (2) sets an initial value (e.g., 0xFFFF) for the block to indicate that the block is not occupied. The third part of pseudocode (2) sets the patch index value for the block covered by the patch in the BlockToPatchMap 2D array using the x and y positions of the block as well as the patch width and patch height.

[0036] Example atlas generator 102 outputs atlas data to encoding system 101. Example volumetric video encoder 104 and example hash generator 106 are also provided.

[0037] Figure 1The volumetric video encoder 104 of the encoding system 101 encodes texture and depth atlases into encoded texture and depth bitstreams representing the atlases, and outputs the encoded data to the example multiplexer 108 of the encoding system 101. In some examples, the volumetric video encoder 104 encodes the texture and depth atlases based on a standards-compliant encoding (e.g., HEVC) to generate a standards-compliant bitstream. In the illustrated example, the volumetric video encoder 104 encodes atlas data to generate encoded atlas data, includes the encoded atlas data in an encoded atlas data bitstream representing the atlas data, and outputs the encoded atlas data bitstream to the example multiplexer 108. In some examples, the volumetric video encoder 104 encodes the atlas data based on a standards-compliant encoding (e.g., HEVC) to generate a standards-compliant bitstream.

[0038] Figure 1 The hash generator 106 of the encoding system 101 uses one or more appropriate hash operations to generate hash values ​​from the atlas data, such as 16-byte message digest algorithm 5 (MD5) hashing, cyclic redundancy check (CRC) hashing, checksum hashing, etc. After hashing the atlas data, the hash generator 106 identifies (e.g., includes or embeds) the hashed atlas data into one or more SEI messages. The hash generator 106 can generate SEI messages to include the hashed atlas data, or it can receive generated SEI messages from another component and include the hashed atlas data in the already generated SEI messages. Example hash generator 106 outputs an SEI message including the hashed atlas data to the multiplexer 108 of the encoding system 101. In some examples, the hash generator 106 includes a single hash value of a single atlas data instance in the atlas data hashed SEI message. In some examples, hash generator 106 includes multiple hash values ​​for multiple atlas data instances in the atlas data hash SEI message (e.g., each hash value is indexed to a specific atlas data instance). In some examples, hash generator 106 includes persistent data corresponding to the atlas data instance in the SEI message, as further described below. In some examples, hash generator 106 generates a hash value for each atlas data instance. In some examples, hash generator 106 generates hash values ​​for several atlas data instances, such as a single instance within a group of pictures (GOP), thereby providing intermittent hash value verification for the transmitted bitstream.

[0039] Figure 1The multiplexer 108 of the encoding system 101 combines or obtains encoded texture and depth bitstreams, encoded atlas data bitstreams, and atlas data hash SEI messages. The encoding system 101 transmits the resulting bitstreams to the decoding system 110 or a memory for eventual use by the decoding system 110. In some examples, the multiplexer 108 uses an interface (e.g., ...) Figure 4 The interface 420 transmits the generated bitstream to the decoding system 110. In such an example, the interface 420 may transmit the generated bitstream to the decoding system 110 via a wired or wireless connection (e.g., via Bluetooth, via the Internet, via a local network, via Wi-Fi, etc.).

[0040] Figure 1 The decoding system 110 (also referred to as a decoder, decoding device, etc.) obtains the bitstream via the demultiplexer 112 of the decoding system 110. In some examples, the demultiplexer 112 is located via an interface (e.g., Figure 5 The decoding system 110 obtains the bitstream via interface 520. The decoding system 110 can be implemented via a client system, while the encoding system 101 can be implemented via a server or source system. The demultiplexer 112 of the decoding system 110 demultiplexes or parses the received bitstream to generate (e.g., extract) (a) an encoded texture and depth bitstream, including encoded representations of the texture and depth video, and intended to match the encoded texture and depth bitstream described for the encoding system 101; (b) an encoded atlas data bitstream, intended to match the atlas data generated by the encoding system 101; and (c) one or more atlas data hash SEI messages from the obtained bitstream data. The encoded texture and depth bitstream, the encoded atlas data bitstream, and the atlas data hash SEI messages can be characterized as decoder-side encoded texture and depth bitstream, encoded atlas data bitstream, and atlas data hash SEI messages. When there are no corresponding bitstream errors for decoding and / or transmission of the bitstream, the encoded texture and depth bitstream, the encoded atlas data bitstream, and the atlas data hash SEI message are copied to their counterparts at encoder system 101. However, mismatches may occur due to bitstream corruption, transmission failures, and / or other reasons.

[0041] Figure 1An example volumetric video decoder 114 of the decoding system 110 receives (e.g., acquires) and decodes encoded textures, depth bitstreams, and encoded atlas data bitstreams (e.g., generating decoded texture atlases, depth atlases, and / or corresponding atlas data instances). The volumetric video decoder 114 transmits the decoded data to the data reconstructor 116 of the decoding system 110. Based on the encoding techniques used by the volumetric video encoder 104, the volumetric video decoder 114 decodes the encoded textures to generate decoded texture values, decodes the encoded depth to generate decoded depth values, and decodes the encoded atlas data bitstream to generate decoded atlas data bitstreams, depth atlases, and / or corresponding atlas data instances.

[0042] Figure 1 The data reconstructor 116 uses the decoded texture atlas, decoded depth atlas, and decoded atlas data to reconstruct a decoded version of the input view provided to the encoding system 101. For example, the data reconstructor 116 of the decoding system 110 can perform the inverse operation of the atlas generator 102 of the encoding system 101 to generate a reconstructed view, which can be used to generate a viewport view for the user.

[0043] Figure 1 Example hash generator 118 obtains decoded atlas data from volumetric video decoder 114. Decoding system 110 attempts to replicate the atlas data generated at encoding system 101, which can be decoded at decoding system 110, so that encoder-side hash atlas data and decoder-side hash atlas data match (e.g., if there are no decoding errors). However, as described above, bitstream corruption and / or transmission failures, and / or failures of volumetric video decoder 114 of decoding system 110, can cause mismatches between encoder-side and decoder-side atlas data. As further described below, comparator 120 can compare encoder-side hash values ​​with decoder-side hash values ​​to determine any mismatches.

[0044] Figure 1 Hash generator 118 generates hash values ​​for atlas data instances by applying the same hash(s) as the hash generator of encoding system 101 to the same atlas data (e.g., matching when there are no decoding / transmission errors, or not matching when there are decoding / transmission errors). The applicable hash(s) can be one or more default hash(s) corresponding to the encoding standard, one or more applicable hash(s) indicated in the atlas data hash SEI message, etc. For each hash value generated by the hash generator of decoding system 110, a comparison is made (by comparator 120) between the decoder-side hash value (e.g., the hash value generated at the decoder) and the encoder-side hash value (e.g., the hash value received in the hash SEI message).

[0045] Figure 1 Example comparator 120 compares a decoder-side hash value (e.g., from hash generator 118) with an encoder-side hash value (e.g., from demultiplexer 112). If comparator 120 determines that the two hash values ​​match, the decoding system 110 (e.g., a client system) is highly certain that the received bitstream is not corrupted and that the volumetric video decoder 114's operation with respect to atlas data (e.g., metadata) conforms to the bitstream and / or applicable video coding standards, such as V3C / V-PCC or MIV. If comparator 120 determines that any two hash values ​​do not match, comparator 120 outputs an error flag indicating a corrupted bitstream or non-conforming decoder operation. In some examples, comparator 120 outputs an error flag for each atlas data instance. The error flag for each atlas data instance is a value that identifies a match or non-match between hash values ​​(e.g., "1" for a match, or "0" for a non-match, and vice versa). In some examples, when a mismatch is detected, comparator 120 outputs an error flag for each atlas data instance (otherwise it is presumed to be a match).

[0046] Below are sample syntax and semantics for implementing verification based on atlas data hash values. In some examples, the SEI message includes a syntax element indicating the number of atlases (e.g., atlas data instances). In some examples, for each atlas, a hash value is indicated. Example hash generators 106 and 118 each use a 16-byte MD5, CRC, checksum, or another type of hash value to generate hash values. Hash value determination is further described below, where, in some examples, each of the two bytes of the 16-bit unsigned BlockToPatchMap[y][x] value is sequentially input into the hash calculation. When the number of patches is limited to 2^16, these two bytes can be used to represent patch identifiers (e.g., 16 bits or 2 bytes are sufficient to represent patch identifiers). In such examples, the hash calculation can be performed on a byte-by-byte basis.

[0047] In some examples, the bitstreams of MIV and VPCC include VPCC units. A VPCC unit includes a header and payload information. The header includes a `vuh_atlas_id` syntax element, which indicates the atlas ID to which the payload applies. The MIV standard supports "universal atlases," which include data that may apply to all atlases. In some examples, if the SEI message exists in a "universal atlas," it may contain hash data for multiple atlases. In some examples, a separate SEI message can be used for each atlas, with its `vuh_atlas_id` value being the same as the associated atlas. When hash data is identified for more than one atlas, the `atlasid` value applicable to each hash is explicitly specified.

[0048] In some examples, the atlas data is consistent across multiple access units, and the proposed SEI hash data can persist until it is cancelled. Persistence can be indicated for the entire SEI message; this can include data for multiple atlases or be indicated separately for each atlas. In some examples, if the persistence flag value is set to 1 (or some other value), the hash value persists for future access units in the bitstream until the cancellation flag value is set to 1 (or some other value) or until an instantaneous random access point (IRAP) atlas exists in the bitstream (e.g., because random access decoding might occur at an IRAP atlas). As mentioned above, an access unit is part of a bitstream (e.g., atlas data, texture video, geometric video, occupancy video, etc.) that has the same decoding order count (e.g., data from a V-PCC / MIV bitstream corresponding to a specific time instance). In some examples, an overall persistence flag and an overall cancellation flag are included in the SEI message, which apply to all atlases. In some examples, if more than one set is indicated, a separate persistence and cancellation flag is provided for each set. Therefore, if comparator 120 (or another device) generates a flag corresponding to the first access unit in the bitstream, then when the message corresponding to the bitstream indicates persistence, the comparator can generate an additional flag for the second (e.g., subsequent) access unit in the bitstream.

[0049] Table 3 below illustrates example syntax for example hash generators 106 and 118 that can be used to identify atlas data hash values ​​and related data (such as atlas indicators and persistence flags).

[0050] Table 3 - Exemplary Grammar

[0051]

[0052] The example syntax and semantics provided in Table 3 indicate the atlas data hash values ​​for the block-to-patch map of one or more atlases.

[0053] The syntax element `dadh_cancel_flag` is set to 1 to indicate that this SEI message cancels the persistence of any previous decoded atlas data hash SEI messages applicable to the current layer's encoding order. The syntax element `dadh_cancel_flag` is set to 0 to indicate that decoded atlas data hash information follows. The syntax element `dadh_persistence_flag` is set to 1 to indicate that this SEI message specifies the persistence of decoded atlas data hash SEI messages. The syntax element `dadh_persistence_flag` is set to 0 to specify that decoded atlas data hash SEI messages apply only to the current decode access unit.

[0054] In some examples, let auA be the current access unit, and dadh_persistence_flag equal to 1 to specify that the decoded atlas data hash SEI message persists in the output order until any of the following conditions are true: (1) a new encoded video sequence (CVS) begins in the current layer, (2) the bitstream ends, or (3) access unit auB in the current layer containing the decoded atlas data hash SEI message is output, for which FrameOrderCnt(auB) is greater than FrameOrderCnt(auA) (e.g., to determine which access unit corresponds to a later video sequence, since a higher count corresponds to a later time in the video sequence), where FrameOrderCnt(auB) and FrameOrderCnt(auA) are the FrameOrderCntVal values ​​of auB and auA, respectively, immediately after the decoding process is called for the picture order count of auB. FrameOrderCnt is the order in which frames are encoded and / or intended to be displayed. FrameOrderCnt is used because the order in which frames are encoded and / or intended to be displayed may differ from the order in which they exist in the bitstream (e.g., because encoding system 101 may choose not to encode frames in order). Hash values ​​may persist for additional frames when the atlas is not explicitly marked for later frames. The syntax element dadh_num_atlases_minus1 plus 1 specifies the number of atlases or atlas tiles for which hash values ​​exist.

[0055] The syntax element dadh_atlas_id[a] specifies the atlas ID of the i-th identified hash data. In some examples, when it does not exist, the value of dadh_atlas_id[a] is inferred to be equal to vuh_atlas_id.

[0056] The syntax element `dadh_atlas_cancei_flag[dadh_atlas_id[a]]` equal to 1 indicates that the SEI message cancels the continuation of any previously decoded atlas data hashes in the encoding order applicable to the `dadh_atlas_id[a]` atlas. The syntax element `dadh_atlas_cancel_flag[dadh_atlas_id[a]]` equal to 0 indicates that the following is the decoded atlas data hash information for the `dadh_atlas_id[a]` atlas. In some examples, when it does not exist, the value of `dadh_atlas_cancel_flag[dadh_atlas_id[a]]` is inferred to be equal to 0.

[0057] The syntax element `dadh_atlas_persistence_flag[dadh_atlas_id[a]]` specifies the persistence of the decoded atlas data hash SEI message for the `dadh_atlas_id[a]` atlas. A value of 0 for `dadh_atlas_persistence_flag[dadh_atlas_id[a]]` indicates that the decoded atlas data is only applicable to the `[dadh_atlas_id[a]]` atlas of the current access unit.

[0058] In some examples, let auA be the current picture, and dadh_atlas_persistence_flag[dadh_atlas_id[a]] equal to 1, specifying that the decoded atlas data hash SEI message persists for the [dadh_atlas_id[a]] atlas in the encoding order until any of the following conditions are true: (1) a new CLVS begins, (2) the bit stream ends, or atlas data unit B in the access unit containing the decoded atlas data hash SEI message for the dadh_atlasI-d[a] atlas is output, for which FrameOrderCnt(auB) is greater than FrameOrderCnt(aauA), where FrameOrderCnt(auB) and FrameOrderCnt(auA) are the FrameOrderCntVal values ​​of auB and auA, respectively, immediately following the atlas data decoding process called for the atlas frame order count for auB.

[0059] In some examples, before calculating the hash value, hash generators 106 and 118 arrange the atlas patch mapping map data into a string of bytes called btpmData, which has a length of dataLen, as shown in the following pseudocode (3):

[0060] Table 4 Pseudocode (3)

[0061]

[0062]

[0063] The syntax element dadh_patch_map_md5[i] is a 16-byte MD5 hash of the BlockToPatchMap[][] variable of the atlas in the currently accessed unit, where the value of vuh_atlas_id is equal to dadh__atlas_id. The value of dadh_patch_map_md5[i] should be equal to the value of digestVal obtained as shown in pseudocode (4) below, using the MD5 function defined in IETF RFC1321.

[0064] Table 5 Pseudocode (4)

[0065] MD5Init(context) MD5Update(context, btpmData, dataLen) MD5Final(digestVal, context)

[0066] Although this specification is for a 16-byte MD5 hash, any appropriate hash operation (one or more) and value (e.g., cyclic redundancy check, and checksum) can be implemented.

[0067] In operation, at encoding system 101, atlas generator 102 can generate atlas data (e.g., atlas data instances, files, or mappings) based on multiple input views, such that the atlas data indicates patch source data for patches in the corresponding texture atlas and / or depth atlas. For example, atlas generator 102 can generate atlas data including an indicator for each pixel in the texture atlas and / or depth atlas, indicating the source patch, source view, and / or source location of that pixel, such that the source patch, source view, and source location are source views relative to multiple views. Hash generator 106 can generate hash values ​​based on the atlas data using one or more corresponding hash operations. For example, atlas data can be scanned from a 2D array to a 1D array, and hash generator 106 can apply one or more hash operations to the scanned 1D array. Hash generator 106 can encode the hash values, along with optional additional hash values ​​from other atlas data, into a hash value message, such as an atlas data hash SEI message, for transmission to decoding system 110. In some examples, multiplexer 108 includes (e.g., combines) atlas data hash SEI messages into a bitstream that also includes bitstream portions corresponding to encoded versions of the encoded texture atlas, encoded depth atlas, and atlas data. The encoded texture atlas, encoded depth atlas, and encoded atlas data can be encoded by volumetric video encoder 104. The encoded texture atlas, encoded depth atlas, and encoded atlas data can be encoded by volumetric video encoder 104 to provide a structure for reconstructing multiple input views. In some examples, hash generator 106 includes a hash operation indicator in the atlas data hash SEI message indicating one or more hash operations for generating hash values. In some examples, the hash generator includes an indicator to cancel the continuation of previous atlas data or start the continuation of current atlas data in the atlas data hash SEI message. In some examples, hash generator 106 includes an indicator of the number of atlases and hash values ​​provided, and one or more indexes to index the hash values ​​and atlases in the atlas data hash SEI message.

[0068] At decoding system 110, the interface of decoding system 110 obtains a bitstream comprising a bitstream portion representing an encoded texture atlas, an encoded depth atlas, and encoded atlas data, and an atlas data hash message, such as an atlas data hash SEI message, which may include any data corresponding to the encoder. Volumetric video decoder 114 decodes the bitstream portion to generate a decoded texture atlas, a decoded depth atlas, and decoded atlas data (e.g., an atlas data instance, file, or mapping). Volumetric video decoder 114 applies one or more hash operations (e.g., selected based on hash operation indicators indicated in the atlas data hash SEI message) to the decoded atlas data to generate a decoder-side hash value. Comparator 120 compares the encoder-side hash value obtained in the atlas data hash SEI message with the decoder-side hash value. When comparator 120 determines that the encoder-side hash value matches the decoder-side hash value (e.g., generated at hash generator 118), comparator 120 can output an indicator to data reconstructor 116 to release decoded atlas data for use in generating a view (e.g., a viewport representing a reconstructed multiview) using the decoded texture atlas, decoded depth atlas, and decoded atlas data, for presentation to the user. When comparator 120 determines that the encoder-side hash value does not match the decoder-side hash value, comparator 120 outputs an indicator (e.g., an error flag) to indicate the mismatch, and the decoder can prevent the rendering of the view, request a new copy of the relevant bitstream, attempt to revise the received bitstream decoding, discard the acquired data, send an indication to the user, etc.

[0069] Although Figure 1 The diagram illustrates the implementation. Figure 1 Example of encoding system 101 and / or decoding system 110, but Figure 1 One or more of the elements, processes, and / or devices illustrated in the diagram may be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other way. Additionally, example atlas generator 102, example volumetric video encoder 104, example hash generator 106, example multiplexer 108, and / or more generally, Figure 1 Example encoding system 101, and / or example demultiplexer 112, example volumetric video decoder 114, example hash generator 118, example data reconstructor 116, example comparator 120, and / or more generally, Figure 1 The example decoding system 110 can be implemented using hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Thus, for example, it could be an example atlas generator 102, an example volumetric video encoder 104, an example hash generator 106, an example multiplexer 108, and / or more generally... Figure 1Example encoding system 101, and / or example demultiplexer 112, example volumetric video decoder 114, example hash generator 118, example data reconstructor 116, example comparator 120, and / or more generally Figure 1 Any of the example decoding systems 110 may be implemented by one or more analog or digital circuits, logic circuits, one or more programmable processors, one or more programmable controllers, one or more graphics processing units (GPUs), one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more programmable logic devices (PLDs), and / or one or more field-programmable logic devices (FPLDs). When any device or system claim of this patent is read to cover purely software and / or firmware implementations, example atlas generator 102, example volumetric video encoder 104, example hash generator 106, example multiplexer 108, and / or more generally... Figure 1 Example encoding system 101, and / or example demultiplexer 112, example volumetric video decoder 114, example hash generator 118, example data reconstructor 116, example comparator 120, and / or more generally Figure 1 At least one of the example decoding systems 110 is explicitly defined herein as including a non-transitory computer-readable storage device or disk containing the software and / or firmware, such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disc, etc. Furthermore, the example encoding system 101 and / or the example decoding system 110 may include, in addition to Figure 1 Those other than or replacing those shown Figure 1 One or more of the elements, processes, and / or devices shown, and / or may include any or all of more than one of the elements, processes, and devices shown in the illustration. As used herein, the phrase “communicating with”—including its variations—covers direct communication and / or indirect communication via one or more intermediate components, without requiring direct physical (e.g., wired) communication and / or continuous communication, but also including selective communication at periodic intervals, scheduled intervals, non-periodic intervals, and / or one-off events.

[0070] Representative used for implementation Figure 1Flowcharts of example hardware logic, machine-readable instructions, hardware implementation state machines, and / or any combination thereof for encoding system 101 and / or decoding system 110 are shown in Figure 2 and Figure 3 As shown below. Machine-readable instructions can be one or more executable programs or portions thereof for execution by a computer processor, such as the one described below. Figure 4 and Figure 5 The processors 412 and 512 shown in the example processor platforms 400 and 500 discussed herein. The program may be embodied in software stored on a non-transitory computer-readable storage medium such as a CD-ROM, floppy disk, hard disk drive, DVD, Blu-ray disc, or memory associated with the processors 412 and 512, but the entire program and / or parts thereof may instead be executed by a device other than the processors 412 and 512 and / or embodied in firmware or dedicated hardware. Furthermore, although this is a reference... Figure 4 and Figure 5 The flowcharts shown illustrate the example program, but many other methods for implementing the example encoding system 101 and / or decoding system 110 may be used alternatively. For example, the execution order of the blocks may be changed, and / or some of the blocks described may be altered, eliminated, or combined. Additionally or alternatively, any or all blocks may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers, logic circuits, etc.) configured to perform the corresponding operations without executing software or firmware.

[0071] The machine-readable instructions described herein can be stored in one or more formats, including compressed formats, encrypted formats, segmented formats, compiled formats, executable formats, packaged formats, etc. The machine-readable instructions described herein can be stored as data (e.g., portions of instructions, code, representations of code, etc.) that can be used to create, manufacture, and / or produce machine-executable instructions. For example, machine-readable instructions can be segmented and stored on one or more storage devices and / or computing devices (e.g., servers). Machine-readable instructions can require installation, modification, adaptation, updating, combination, supplementation, configuration, decryption, decompression, unpacking, distribution, reassignment, compilation, etc., so that they are directly readable, interpretable, and / or executable by computing devices and / or other machines. For example, machine-readable instructions can be stored as multiple parts that are individually compressed, encrypted, and stored on separate computing devices, wherein these parts, when decrypted, decompressed, and combined, form a set of executable instructions that implement a program such as that described herein.

[0072] In another example, machine-readable instructions may be stored in a state in which they are readable by a computer, but require the addition of libraries (e.g., dynamic link libraries (DLLs)), software development kits (SDKs), application programming interfaces (APIs), etc., to execute these instructions on a specific computing device or other device. In another example, the machine-readable instructions and / or the corresponding program(s) may need to be configured (e.g., storage settings, input data, recording network addresses, etc.) before they can be executed in whole or in part. Thus, the disclosed machine-readable instructions and / or the corresponding program(s) are intended to encompass such machine-readable instructions and / or program(s), regardless of the specific format or state of these machine-readable instructions and / or program(s) at the time of storage or otherwise at rest or in transit.

[0073] The machine-readable instructions described in this article can be represented by any past, present, or future instruction language, scripting language, programming language, etc. For example, machine-readable instructions can be represented using any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, HyperText Markup Language (HTML), Structured Query Language (SQL), Swift, etc.

[0074] As described above, executable instructions (e.g., computer and / or machine-readable instructions) stored on a non-transitory computer and / or machine-readable medium can be used to implement... Figure 2 and Figure 3 The example process, wherein the medium is, for example, a hard disk drive, flash memory, read-only memory, compact disk, digital multifunction disk, cache, random access memory, and / or any other storage device or disk in which information may be stored for any duration (e.g., long-term storage, permanent storage, transient storage, temporary buffering, and / or caching for information). For the purposes of this document, the term nontransitory computer-readable medium is explicitly defined as including any type of computer-readable storage device and / or disk, and excludes propagating signals and transmission media.

[0075] "Comprising" and "including" (and all its forms and tenses) are used herein as introductory terms. Thus, whenever a claim uses any form of "comprising" or "including" (e.g., including, comprising, having, etc.) as a preamble or in any kind of claim statement, it is understood that additional elements, terms, etc., may exist without falling outside the scope of the corresponding claim or statement. For the purposes of this document, when the phrase "at least" is used as a transitional term in, for example, the preamble of a claim, it is introductory, just as the terms "comprising" and "including" are introductory. The term "and / or," when used, for example, in the form of, say, A, B, and / or C, refers to any combination or subset of A, B, and C, such as (1) A alone, (2) B alone, (3) C alone, (4) A and B, (5) A and C, (6) B and C, and (7) A and B and C. As used herein in the context of describing structures, components, items, objects, and / or things, the phrase “at least one of A and B” is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing structures, components, items, objects, and / or things, the phrase “at least one of A or B” is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. As used herein in the context of describing the execution or operation of processes, instructions, actions, activities, and / or steps, the phrase “at least one of A and B” is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, for the purposes of this document in the context of describing the execution or operation of processes, instructions, actions, activities and / or steps, the phrase “at least one of A or B” is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.

[0076] As used herein, singular references (e.g., “a,” “first,” “second,” etc.) do not exclude pluralism. For the purposes of this document, the term “a” refers to one or more of the same entity. The terms “a,” “one or more,” and “at least one” may be used interchangeably herein. Furthermore, although listed separately, multiple means, elements, or method actions may be implemented by, for example, a single unit or processor. Moreover, while individual features may be included in different examples or claims, they may be combined, and inclusion in different examples or claims does not imply that the combination of features is infeasible and / or not advantageous.

[0077] Figure 2 This is a flowchart representing example machine-readable instructions 200, which can be executed to implement example encoding system 101 to generate a bitstream with an SEI message, the SEI message including hash atlas data that can be used by encoding system 101 to ensure proper decoding. Although it is combined... Figure 2 The example encoding system 101 is used to describe instruction 200, but any type of encoding system can be used to describe instruction 200.

[0078] In block 202, atlas generator 102 determines whether input data has been received or acquired. Input data is video image data corresponding to one or more views of an image that will be displayed as part of a video. Input data can be acquired from a processor, storage device, sensor, video generation device, etc. If the example atlas generator 102 determines that no input data has been received (block 202, "No"), control returns to block 202 until input data is received (e.g., acquired). If the example atlas generator 102 determines that input data has been received (block 202, "Yes"), the example atlas generator 102 generates atlas data, texture atlas data, depth atlas data, etc., based on the received input data (e.g., using a V3C or MIV encoder) (block 204). For example, atlas generator 102 can generate atlas data using a 2D array variable (e.g., BlockToPatchMap[y][x] as defined above).

[0079] In block 206, example hash generator 106 hashes the atlas data and includes the hashed atlas data in an SEI message. In some examples, hash generator 106 also includes a hash protocol in the SEI message (e.g., so that decoding system 110 can hash the bitstream using the same hash protocol). As described above, hash generator 106 can generate SEI messages and / or obtain SEI messages from another component, and include (e.g., embed) the hashed atlas data in the SEI message. Hash generator 106 can generate hashes of the atlas data using 16-byte MD5 hashes, cyclic redundancy check (CRC) hashes, checksum hashes, and / or any other type of hash protocol. In block 208, example volumetric video encoder 104 generates an encoded bitstream (e.g., encoded bitstream data) based on atlas data, texture atlas data, and / or depth atlas data by encoding and / or compressing the atlas data, texture atlas data, and / or depth atlas data. The volumetric video encoder 104 can use any type of atlas encoding, video encoding technology, atlas compression and / or video compression technology.

[0080] In block 210, example multiplexer 108 generates bitstream data by combining the encoded bitstream from example volumetric video encoder 104 with an SEI message including hash atlas data from example hash generator 106. Example multiplexer 108 can use any type of multiplexing protocol to combine these two signals. In block 212, example multiplexer 108 transmits the bitstream data to decoding system 110. For example, multiplexer 108 can output the bitstream data signal to an interface (e.g., ...). Figure 4 Example interface 420) is used to output to the decoding system 110 via a wired or wireless connection.

[0081] Figure 3 This is a flowchart representing example machine-readable instructions 300, which can be executed to implement example decoding system 110 to generate a bitstream with an SEI message including hash set data, which the decoding system 110 can use to ensure that the decoding system 110 properly decodes and / or that the bitstream does not contain any errors. Although it is combined Figure 1 The example decoding system 110 is used to describe instruction 300, but instruction 300 can be described using any type of decoding system.

[0082] In block 302, demultiplexer 112 determines whether bitstream data has been received or obtained from encoding system 101. As described above, the bitstream data includes encoded video data and corresponding SEI messages. Demultiplexer 112 can obtain data from an interface (e.g., Figure 5 The example demultiplexer 112 receives bitstream data via an interface 520, which receives the bitstream data via a wired or wireless connection. If the example demultiplexer 112 determines that no bitstream data has been received (block 302, "No"), control returns to block 302 until bitstream data is received (e.g., acquired). If the example demultiplexer 112 determines that bitstream data has been received (block 302, "Yes"), the example demultiplexer 112 demultiplexes (e.g., separates and / or parses) the bitstream data to separate the data bitstream and SEI message from the bitstream data (block 304).

[0083] In block 306, the example volumetric video decoder 114 decodes bitstream data to generate decoded atlas data, decoded texture atlas data, decoded depth atlas data, and so on, using the bitstream data (e.g., using a V3C or MIV decoder). The example volumetric video decoder 114 uses a decoding technique corresponding to the technique used to encode the data (e.g., the technique used by the volumetric video encoder 104) to decode the data. In block 308, the example hash generator 118 hashes the decoded atlas data to generate decoder-side atlas hash values. The example hash generator 118 uses the same technique used at encoding system 101 to hash the decoded atlas data. In this way, if the decoder-side hash value matches the encoding-side hash value (e.g., from an acquired SEI message), the decoding system 110 determines that a decoding error has occurred. In some examples, the hash generator 118 extracts the hash protocol from the SEI message.

[0084] In block 310, example comparator 120 extracts or otherwise obtains the encoder-side graph set hash value from the SEI message. As described above, encoding system 101 performs one or more hash operations on the graph set data during the encoding process and transmits the encoder-side hash graph set as part of the SEI message. In block 312, example comparator 120 compares the encoder-side graph set hash value (e.g., determined at encoding system 101 and extracted from the SEI message) with the decoder-side graph set hash value (e.g., a hash value generated by hash generator 118 at decoding system 110). As described above, example comparator 120 compares the encoder-side graph set hash value with the decoder-side graph set value to determine whether accurate transmission and / or accurate decoding of the bitstream was performed.

[0085] In block 314, example comparator 120 determines whether the encoder-side map set hash value differs from the decoder-side map set hash value. Since the encoder-side map set hash value and the decoder-side map set hash value are hashed using the same hash protocol, and the map set data should be identical, the encoder-side hash value and the decoder-side hash value will be the same unless a transmission error or decoding error exists. Therefore, if example comparator 120 determines that the encoder-side map set hash value differs from the decoder-side map set hash value (block 314: "Yes"), example comparator 120 flags a mismatch (e.g., outputs a value indicating a mismatch) and / or executes a mismatch-based instruction (block 316) because the decoding system 110 determines that a transmission and / or decoding error exists when the two hash map set values ​​differ. For example, the output of comparator 120 could cause the mismatch to be reflected at the user interface to notify the user of a decoding error. The error flag could be sent to another processor or component to perform an action corresponding to the mismatch. In some examples, an error flag may be output to data reconstructor 116 and / or another device so that the corresponding data is discarded. In some examples, comparator 120, data reconstructor 115, and / or another device may determine whether the acquired bitstream data and / or SEI message includes a value indicating persistence. If, in example, comparator 120, data reconstructor 115, and / or another device determine that the acquired bitstream data includes a value indicating persistence, comparator 120 will mark the corresponding access unit corresponding to that bitstream as having an error.

[0086] If the example comparator 120 determines that the encoder side map set hash value is no different from the decoder side map set hash value (block 314: "No"), then control ends because the decoding system 110 determines that the bitstream has been accurately decoded. In some examples, the output of comparator 120 is sent to data reconstructor 116. In this way, data reconstructor 116 can determine whether to discard the decoded map set data and / or reconstruct the decoded map set data into a reconstructed view based on the result of comparator 120 (e.g., initiating a match or a mismatch).

[0087] Figure 4 It is constructed to execute Figure 2 Instructions to achieve Figure 1 A block diagram of an example processor platform 400 for the encoding system 101. The processor platform 400 can be, for example, a server, a personal computer, a workstation, a self-learning machine (e.g., a neural network), or a mobile device (e.g., a cellular phone, a smartphone, or an iPad). TMTablet devices, personal digital assistants (PDAs), internet-connected appliances, DVD players, CD players, digital video recorders, Blu-ray players, game consoles, personal video recorders, set-top boxes, headphones or other wearable devices, or any other type of computing device.

[0088] The illustrated processor platform 400 includes a processor 412. The illustrated processor 412 is hardware. For example, the processor 412 may be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor may be a semiconductor-based (e.g., silicon-based) device. In this example, the processor implements... Figure 1 The atlas generator 102, volumetric video encoder 104, hash generator 106, and multiplexer 108 are included.

[0089] The illustrated processor 412 includes local memory 413 (e.g., cache). The illustrated processor 412 communicates via bus 418 with main memory, which includes volatile memory 414 and non-volatile memory 416. The volatile memory 414 may be Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), etc. Dynamic Random Access Memory (DRAM) Dynamic Random Access Memory, This can be implemented using flash memory and / or any other type of random access memory device. Non-volatile memory 416 can be implemented using flash memory and / or any other desired type of memory device. Access to main memory 414, 416 is controlled by the memory controller.

[0090] The processor platform 400 illustrated also includes interface circuitry 420. Interface circuitry 420 can be implemented using any type of interface standard, such as an Ethernet interface, Universal Serial Bus (USB), etc. Interfaces include near field communication (NFC) interfaces and / or PCI fast interfaces.

[0091] In the illustrated example, one or more input devices 422 are connected to interface circuitry 420. The input devices 422 allow users to input data and / or commands into processor 412. The input devices may be implemented as, for example, audio sensors, microphones, cameras (still or video), keyboards, buttons, mice, touchscreens, touchpads, trackballs, isopoints, and / or voice recognition systems.

[0092] One or more output devices 424 are also connected to the interface circuitry 420 of the illustrated example. The output devices 424 may be implemented, for example, by display devices (e.g., light-emitting diodes (LEDs), organic light-emitting diodes (OLEDs), liquid crystal displays (LCDs), cathode ray tube displays (CRTs), in-place switching (IPS) displays, touchscreens, etc.), haptic output devices, printers, and / or speakers. The interface circuitry 420 of the illustrated example thus typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0093] The interface circuit 420 illustrated also includes communication devices, such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces, to facilitate data exchange with external machines (e.g., any type of computing device) via network 426. Communication may be via, for example, Ethernet connections, digital subscriber line (DSL) connections, telephone line connections, coaxial cable systems, satellite systems, line-to-line wireless systems, cellular telephone systems, and so on.

[0094] The illustrated processor platform 400 also includes one or more mass storage devices 428 for storing software and / or data. Examples of such mass storage devices 428 include floppy disk drives, hard disk drives, compact disk drives, Blu-ray disc drives, redundant array of independent disks (RAID) systems, and digital versatile disk (DVD) drives.

[0095] Figure 2 The machine-executable instructions 432 may be stored in mass storage device 428, volatile memory 414, non-volatile memory 416, and / or on a removable non-transitory computer-readable storage medium such as a CD or DVD.

[0096] Figure 5 It is constructed to execute Figure 3 Instructions to achieve Figure 1 A block diagram of an example processor platform 500 for the decoding system 110. The processor platform 500 can be, for example, a server, personal computer, workstation, self-learning machine (e.g., neural network), mobile device (e.g., cellular phone, smartphone, such as iPad). TM Tablet devices, personal digital assistants (PDAs), internet-connected appliances, DVD players, CD players, digital video recorders, Blu-ray players, game consoles, personal video recorders, set-top boxes, headphones or other wearable devices, or any other type of computing device.

[0097] The illustrated processor platform 500 includes a processor 512. The illustrated processor 512 is hardware. For example, the processor 512 may be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor may be a semiconductor-based (e.g., silicon-based) device. In this example, the processor implements... Figure 1 Example demultiplexer 112, example volumetric video decoder 114, example data reconstructor 116, example hash generator 118, and example comparator 120.

[0098] The illustrated processor 512 includes local memory 513 (e.g., cache). The illustrated processor 512 communicates via bus 518 with main memory, which includes volatile memory 514 and non-volatile memory 516. The volatile memory 514 may be Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), etc. Dynamic Random Access Memory (DRAM) Dynamic Random Access Memory, This can be implemented using flash memory and / or any other type of random access memory device. The non-volatile memory 516 can be implemented using flash memory and / or any other desired type of memory device. Access to the main memory 514, 516 is controlled by the memory controller.

[0099] The processor platform 500 illustrated also includes interface circuitry 520. Interface circuitry 520 can be implemented using any type of interface standard, such as an Ethernet interface, universal serial bus (USB), etc. Interfaces include near field communication (NFC) interfaces and / or PCI fast interfaces.

[0100] In the illustrated example, one or more input devices 522 are connected to interface circuitry 520. The input devices 522 allow a user to input data and / or commands into processor 512. The input devices may be implemented as, for example, audio sensors, microphones, cameras (still or video), keyboards, buttons, mice, touchscreens, touchpads, trackballs, isopoint devices, and / or voice recognition systems.

[0101] One or more output devices 524 are also connected to the interface circuitry 520 illustrated in the figure. The output devices 524 may be implemented, for example, by display devices (e.g., light-emitting diodes (LEDs), organic light-emitting diodes (OLEDs), liquid crystal displays (LCDs), cathode ray tube displays (CRTs), in-place switching (IPS) displays, touchscreens, etc.), haptic output devices, printers, and / or speakers. The interface circuitry 520 illustrated in the figure thus typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0102] The interface circuit 520 illustrated also includes communication devices, such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces, to facilitate data exchange with external machines (e.g., any type of computing device) via network 526. Communication may be via, for example, Ethernet connections, digital subscriber line (DSL) connections, telephone line connections, coaxial cable systems, satellite systems, line-to-line wireless systems, cellular telephone systems, and so on.

[0103] The processor platform 500 illustrated also includes one or more mass storage devices 528 for storing software and / or data. Examples of such mass storage devices 528 include floppy disk drives, hard disk drives, compact disk drives, Blu-ray disc drives, redundant array of independent disks (RAID) systems, and digital versatile disk (DVD) drives.

[0104] Figure 3 The machine-executable instructions 532 may be stored in mass storage device 528, volatile memory 514, non-volatile memory 516, and / or on a removable non-transitory computer-readable storage medium such as a CD or DVD.

[0105] This document discloses example methods, apparatuses, systems, and articles of art for identifying video decoding errors. Further examples and combinations thereof include the following: Example 1 includes a video encoding apparatus comprising an atlas generator for generating atlas data for one or more atlases generated from an input view of a video; a hash generator for performing a hash operation on the atlas data to generate a hash value and including the hash value in a message; and a multiplexer for combining the one or more atlases, encoded atlas data corresponding to the atlas data, and the message to generate a video bitstream.

[0106] Example 2 includes the apparatus as described in Example 1, and further includes an interface for transmitting the bitstream data to a decoding device.

[0107] Example 3 includes the apparatus as described in Example 1, wherein the atlas data comprises a two-dimensional array of variables corresponding to a block-to-pattern mapping atlas of the input view.

[0108] Example 4 includes an apparatus as described in Example 1, wherein the atlas data corresponds to a mapping from block to corresponding patch.

[0109] Example 5 includes the apparatus as described in Example 1, wherein the atlas generator is configured to perform the following operations: converting an input view of the video into at least one of a texture atlas or a depth atlas, and encoding at least one of the texture atlas, the depth atlas, or the atlas data to generate encoded bitstream data.

[0110] Example 6 includes an apparatus as described in Example 1, wherein the hash generator is configured to perform at least one of the following operations: generating the message, or obtaining the message from another device.

[0111] Example 7 includes the apparatus as described in Example 1, wherein the hash generator uses a 16-byte message digest algorithm 5 (MD5) sum to hash the atlas data.

[0112] Example 8 includes a non-transitory computer-readable storage medium comprising instructions that, when executed, cause one or more processors to perform at least the following operations: generating atlas data for one or more atlases generated from an input view of video; performing a hash operation on the atlas data to generate a hash value; including the hash value in a message; and combining the one or more atlases, coded atlas data corresponding to the atlas data, and the message to generate a video bitstream.

[0113] Example 9 includes a computer-readable storage medium as described in Example 8, wherein the instructions cause the one or more processors to transmit the bitstream data to a decoding device.

[0114] Example 10 includes a computer-readable storage medium as described in Example 8, wherein the atlas data is a variable two-dimensional array corresponding to a block-to-pattern mapping atlas of the input view.

[0115] Example 11 includes a computer-readable storage medium as described in Example 8, wherein the atlas data corresponds to a mapping from blocks to corresponding patches in the atlas space.

[0116] Example 12 includes a computer-readable storage medium as described in Example 8, wherein the instructions cause the one or more processors to perform the following operations: converting an input view of the video into at least one of a texture atlas or a depth atlas, and encoding at least one of the texture atlas, the depth atlas, or the atlas data to generate encoded bitstream data.

[0117] Example 13 includes a computer-readable storage medium as described in Example 8, wherein the instructions cause the one or more processors to perform at least one of the following operations: generating the message or obtaining the message from another device.

[0118] Example 14 includes a computer-readable storage medium as described in Example 8, wherein the instructions cause the one or more processors to hash the atlas data using a 16-byte message digest algorithm 5 (MD5) sum.

[0119] Example 15 includes a video encoding method comprising: generating atlas data for one or more atlases generated from an input view of a video by executing instructions using a processor; performing a hash operation on the atlas data to generate a hash value by executing instructions using the processor; including the hash value in a message by executing instructions using the processor; and combining the one or more atlases, encoded atlas data corresponding to the atlas data, and the message by executing instructions using the processor to generate a video bitstream.

[0120] Example 16 includes the method as described in Example 15, further comprising: transmitting the bitstream data to a decoding device.

[0121] Example 17 includes the method as described in Example 15, wherein the atlas data is a variable two-dimensional array corresponding to a block-to-pattern mapping atlas of the input view.

[0122] Example 18 includes the method as described in Example 15, wherein the atlas data corresponds to a mapping from block to corresponding patch.

[0123] Example 19 includes the method as described in Example 15, further comprising: converting an input view of the video into at least one of a texture atlas or a depth atlas, and encoding at least one of the texture atlas, the depth atlas, or the atlas data to generate encoded bitstream data.

[0124] Example 20 includes the method as described in Example 15, and further includes performing at least one of the following operations: generating the message, or obtaining the message from another device.

[0125] Example 21 includes the method described in Example 15, further comprising: hashing the atlas data using a 16-byte message digest algorithm 5 (MD5) sum.

[0126] Example 22 includes a video decoding apparatus comprising a hash generator for performing a hash operation on atlas data to determine a first hash value of the atlas data, the atlas data being decoded from a volumetric video bitstream; and a comparator for performing the following operations: comparing the first hash value with a second hash value of the atlas data, the second hash value being included in the video bitstream; and generating an error flag when the first hash value and the second hash value do not match.

[0127] Example 23 includes the apparatus as described in Example 22, and further includes a demultiplexer for performing the following operations: obtaining the video bitstream, and separating the atlas data and messages from the bitstream.

[0128] Example 24 includes the apparatus as described in Example 23, wherein the message includes the second hash value.

[0129] Example 25 includes an apparatus as described in Example 22, wherein the hash generator uses the same hash operation used to generate the second hash value to generate the first hash value.

[0130] Example 26 includes an apparatus as described in Example 22, wherein the demultiplexer is used to obtain the bitstream from the encoder via a network interface.

[0131] Example 27 includes the apparatus as described in Example 22, wherein the error flag is used to identify that an error has occurred in at least one of the transmission of the video bitstream or the decoding of the bitstream.

[0132] Example 28 includes the apparatus as described in Example 22, wherein the error flag is a first error flag corresponding to a first access unit, and the comparator is used to generate a second error flag for a second access unit in the bitstream when the message indicates persistence.

[0133] Example 29 includes the apparatus as described in Example 22, wherein the error flag is used to cause at least one of the following: an error is displayed on a user interface, the video bitstream is discarded, or a new copy of the video bitstream is requested.

[0134] Example 30 includes an apparatus as described in Example 22, wherein the atlas data includes patch indexes of data blocks.

[0135] Example 31 includes a non-transitory computer-readable storage medium comprising instructions that, when executed, cause one or more processors to perform at least the following operations: perform a hash operation on atlas data to determine a first hash value of the atlas data, the atlas data being decoded from a video bitstream; compare the first hash value with a second hash value of the atlas data, the second hash value being included in the video bitstream; and generate an error flag when the first hash value and the second hash value do not match.

[0136] Example 32 includes a computer-readable storage medium as described in Example 31, wherein the instructions cause the one or more processors to perform at least the following operations: obtaining the video bitstream, and separating the atlas data and messages from the bitstream.

[0137] Example 33 includes a computer-readable storage medium as described in Example 32, wherein the message includes the second hash value.

[0138] Example 34 includes a computer-readable storage medium as described in Example 31, wherein the instructions cause the one or more processors to generate the first hash value using at least the same hash operation used to generate the second hash value.

[0139] Example 35 includes a computer-readable storage medium as described in Example 31, wherein the instructions cause the one or more processors to obtain the bitstream from the encoder at least via a network interface.

[0140] Example 36 includes a computer-readable storage medium as described in Example 31, wherein the error flag is used to identify that an error has occurred in at least one of the transmission of the video bitstream or the decoding of the bitstream.

[0141] Example 37 includes a computer-readable storage medium as described in Example 31, wherein the error flag is a first error flag corresponding to a first access unit, and the instruction causes the one or more processors to generate a second error flag for a second access unit in the bitstream when the message indicates persistence.

[0142] Example 38 includes a computer-readable storage medium as described in Example 31, wherein the error flag is used to cause at least one of the following: an error is displayed on a user interface, the video bitstream is discarded, or a new copy of the video bitstream is requested.

[0143] Example 39 includes a computer-readable storage medium as described in Example 31, wherein the atlas data includes patch indexes of data blocks.

[0144] Example 40 includes a video decoding method comprising: performing a hash operation on atlas data to determine a first hash value of the atlas data, the atlas data being decoded from a video bitstream, by executing instructions using a processor; comparing the first hash value with a second hash value of the atlas data, the second hash value being included in the video bitstream, by executing instructions using the processor; and generating an error flag when the first hash value and the second hash value do not match, by executing instructions using the processor.

[0145] Example 41 includes the method as described in Example 40, further comprising: obtaining the video bitstream, and separating the atlas data and messages from the bitstream.

[0146] Example 42 includes the method as described in Example 41, wherein the message includes the second hash value.

[0147] Example 43 includes the method described in Example 40, further comprising: generating the first hash value using the same hash operation used to generate the second hash value.

[0148] Example 44 includes the method as described in Example 40, further comprising: obtaining the bitstream from an encoder via a network interface.

[0149] Example 45 includes the method as described in Example 40, wherein the error flag is used to identify that an error has occurred in at least one of the transmission of the video bitstream or the decoding of the bitstream.

[0150] Example 46 includes the method as described in Example 40, wherein the error flag is a first error flag corresponding to the first access unit, and further includes generating a second error flag for the second access unit in the bitstream when the message indicates persistence.

[0151] Example 47 includes the method as described in Example 40, wherein the error flag is used to cause at least one of the following: an error is displayed on a user interface, the video bitstream is discarded, or a new copy of the video bitstream is requested.

[0152] Example 48 includes the method as described in Example 40, wherein the atlas data includes patch indexes of data blocks.

[0153] As will be clear from the foregoing, example methods, apparatus, and articles for identifying video decoding errors have been disclosed. The disclosed methods, apparatus, and articles improve the efficiency of using computing devices by identifying transmission and / or decoding errors corresponding to the transmission and / or decoding of video bitstream data. In this way, errors can be addressed based on user preferences. Therefore, the disclosed methods, apparatus, and articles target one or more improvements to the functionality of a computer.

[0154] While certain example methods, apparatuses, and articles of manufacture are disclosed herein, the scope of this patent is not limited thereto. Rather, this patent covers all methods, apparatuses, and articles of manufacture that fairly fall within the scope of the claims of this patent.

Claims

1. An encoder, comprising: instruction; as well as At least one programmable circuit, said at least one programmable circuit being programmed based on the instructions to: Generate tile data that maps tiles to one or more patches represented by the tiles, the tile data being based on at least one syntax element associated with at least one of the one or more patches; The tile data is hashed to generate a hash value; The hash value is included in the Supplemental Enhancement Information (SEI) message; as well as A bitstream is generated based on the tile and the SEI message.

2. The encoder according to claim 1, wherein, The bitstream is a visual volumetric video encoded (V3C) bitstream.

3. The encoder according to claim 1, wherein, One or more of the at least one programmable circuits include information used to identify the type of the hash in the SEI message.

4. The encoder according to claim 3, wherein, The type of hash corresponds to one of several hash types, including Message Digest Algorithm 5 (MD5) hash, Cyclic Redundancy Check (CRC) hash, and Checksum hash.

5. The encoder according to claim 1, wherein, The tiles in question are from a tile catalog.

6. The encoder according to claim 1, wherein, The at least one syntax element includes the BlockToPatchMap syntax element.

7. The encoder according to claim 1, wherein, The at least one syntax element includes the Patch2dPosX syntax element and the Patch2dPosY syntax element.

8. A method comprising: Generate tile data that maps tiles to one or more patches represented by the tiles, the tile data being based on at least one syntax element associated with at least one of the one or more patches; The tile data is hashed to generate a hash value; The hash value is included in the Supplemental Enhancement Information (SEI) message; as well as A bitstream is generated based on the tile and the SEI message.

9. The method according to claim 8, wherein, The bitstream is a visual volumetric video encoded (V3C) bitstream.

10. The method of claim 8, further comprising: Information used to identify the type of the hash is included in the SEI message.

11. The method according to claim 10, wherein, The type of hash corresponds to one of several hash types, including Message Digest Algorithm 5 (MD5) hash, Cyclic Redundancy Check (CRC) hash, and Checksum hash.

12. The method according to claim 8, wherein, The tiles in question are from a tile catalog.

13. The method according to claim 8, wherein, The at least one syntax element includes the BlockToPatchMap syntax element.

14. The method according to claim 8, wherein, The at least one syntax element includes the Patch2dPosX syntax element and the Patch2dPosY syntax element.

15. One or more computer-readable media storing instructions that, in response to execution by one or more processors, cause the one or more processors to perform the method according to any one of claims 8 to 14.

16. A computing device comprising means for performing the method according to any one of claims 8 to 14.

17. A computer program product comprising instructions that, in response to execution by one or more processors, cause the one or more processors to perform the method according to any one of claims 8 to 14.

Images