Data transmission method, device and equipment and computer readable storage medium

Abstract

The invention discloses a data transmission method, device and equipment and a computer readable storage medium, and is applied to the field of data transmission, and the method comprises the steps that a receiving end analyzes an extended three-dimensional Gaussian detail hierarchical descriptor in a transmission signaling to obtain first-level information metadata, requesting media resources corresponding to a target layer from a transmission end according to the first-level information metadata; the first hierarchy information metadata at least comprises hierarchy information and media resource information corresponding to the hierarchy information; the target layer is determined according to receiving end performance; analyzing expanded second-level information metadata in the media resources to obtain three-dimensional Gaussian data corresponding to a target layer; the second hierarchy information metadata at least comprises hierarchy information and three-dimensional Gaussian data information corresponding to the hierarchy; the three-dimensional Gaussian data is six-degree-of-freedom space three-dimensional Gaussian reconstruction data under a VR or AR real-time interaction scene; and carrying out decoding and rendering presentation on the three-dimensional Gaussian data. According to the invention, adaptive transmission of three-dimensional Gaussian data based on hierarchical information is supported.

Description

Technical Field

[0001] This invention relates to the field of data transmission, and in particular to a data transmission method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] When representing 3D scenes using 3D Gaussian data, millions of 3D Gaussian primitives are typically required to ensure a realistic reconstruction of the scene. Therefore, representing a 3D scene using 3D Gaussian data often involves a massive amount of stored data. Based on this, hierarchical partitioning of the N primitives of a 3D Gaussian scene is currently an effective solution. However, existing hierarchical encoding, transmission, and decoding schemes for 3D Gaussian data cannot efficiently and accurately achieve adaptive transmission and decoding based on hierarchical information at the receiving end, making it difficult to balance transmission efficiency, terminal computing power adaptability, and scene presentation quality.

[0003] Therefore, how to provide an adaptive transmission method for 3D Gaussian data based on hierarchical information is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a data transmission method, apparatus, device and computer-readable storage medium, which solves the problem that the prior art cannot support adaptive transmission of three-dimensional Gaussian data based on hierarchical information.

[0005] To solve the above-mentioned technical problems, the present invention provides a data transmission method, comprising:

[0006] The extended 3D Gaussian detail layered descriptor in the transmission signaling is parsed to obtain the first-level information metadata, and the media resources corresponding to the target layer are requested from the transmission end based on the first-level information metadata; the first-level information metadata includes at least the layer information and the media resource information corresponding to the layer information; the target layer is determined based on the performance of the receiving end;

[0007] The extended second-level information metadata in the media resource is parsed, and the three-dimensional Gaussian data corresponding to the target layer is extracted based on the parsed information; the second-level information metadata includes at least the level information and the corresponding three-dimensional Gaussian data information; the three-dimensional Gaussian data is six-degree-of-freedom three-dimensional Gaussian reconstruction data in VR or AR real-time interactive scenarios;

[0008] The three-dimensional Gaussian data is decoded and rendered.

[0009] Optionally, if the media resource is a file, the file is decapsulated to obtain a three-dimensional Gaussian data track, and the data in the three-dimensional Gaussian data track is decoded to obtain the three-dimensional Gaussian data corresponding to the target level, including:

[0010] The file is decapsulated to separate the three-dimensional Gaussian detail level orbitals;

[0011] Analyze the three-dimensional Gaussian detail level orbit to obtain the level identifier corresponding to the target level;

[0012] Based on the level identifier corresponding to the target level, locate the encoded data segment in the 3D Gaussian detail level orbit that matches the target level, and decode the encoded data segment to obtain the 3D Gaussian data corresponding to the target level.

[0013] Optionally, the 3D Gaussian detail layer descriptor includes at least an identifier for 3D Gaussian detail layering, wherein a larger value of the identifier indicates a higher level of detail layering.

[0014] Optionally, the first-level information metadata and the second-level information metadata may further include: the association and dependency relationships between different levels, as well as the quality differences and presentation differences between different levels.

[0015] Optionally, the first-level information metadata and the second-level information metadata may further include: the association and dependency relationships between different levels, as well as the quality differences and presentation differences between different levels.

[0016] The present invention also provides a data transmission device, applied at a receiving end, comprising:

[0017] The first parsing module is used to parse the extended three-dimensional Gaussian detail layered descriptor in the transmission signaling, obtain the first-level information metadata, and request the media resources corresponding to the target layer from the transmission end based on the first-level information metadata; the first-level information metadata includes at least the layer information and the media resource information corresponding to the layer information; the target layer is determined based on the performance of the receiving end;

[0018] The second parsing module is used to parse the extended second-level information metadata in the media resource, and extract the three-dimensional Gaussian data corresponding to the target layer based on the parsed information; the second-level information metadata includes at least the level information and the corresponding three-dimensional Gaussian data information; the three-dimensional Gaussian data is six-degree-of-freedom three-dimensional Gaussian reconstruction data in VR or AR real-time interactive scenarios.

[0019] The decoding and rendering module is used to decode and render the three-dimensional Gaussian data.

[0020] The present invention also provides a data transmission device, comprising:

[0021] Memory, used to store computer programs;

[0022] A processor for implementing the data transmission method as described above when executing the computer program.

[0023] The present invention also provides a computer-readable storage medium storing computer-executable instructions, which, when loaded and executed by a processor, implement the data transmission method described above.

[0024] As can be seen, this invention obtains first-level information metadata by parsing the extended 3D Gaussian detail layered descriptor in the transmission signaling, and requests the media resources corresponding to the target layer from the transmitting end based on the first-level information metadata. The first-level information metadata includes at least the layer information and the media resource information corresponding to the layer information. The target layer is determined based on the performance of the receiving end. The extended second-level information metadata in the media resources is parsed, and the 3D Gaussian data corresponding to the target layer is extracted based on the parsed information. The second-level information metadata includes at least the layer information and the 3D Gaussian data information corresponding to the layer. The 3D Gaussian data is six-degree-of-freedom 3D Gaussian reconstruction data in VR or AR real-time interactive scenarios. The 3D Gaussian data is decoded and rendered. This invention combines the layered 3D Gaussian data with encoding and transmission technologies, expands the metadata system of transmission signaling and media resources, and supports adaptive transmission and decoding based on layer information by setting standardized layer metadata indication information. Standardized metadata extensions across the entire chain can effectively ensure the consistency of hierarchical information in each stage of encoding, encapsulation, transmission, and decoding, avoiding hierarchical alignment errors caused by signaling fragmentation, improving decoding robustness and interoperability between different devices, and reducing technology deployment costs. The receiving end can accurately obtain the target level of 3D Gaussian data based on the metadata without having to decode the entire data, significantly reducing terminal computing power overhead. This meets the stringent requirements of VR / AR six-degrees-of-freedom real-time interactive scenarios for 3D Gaussian data in terms of transmission efficiency, terminal adaptability, and immersive presentation quality.

[0025] In addition, the present invention also provides a data transmission device, apparatus, and computer-readable storage medium, which also have the above-mentioned beneficial effects. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0027] Figure 1 A three-dimensional Gaussian example diagram provided for an embodiment of the present invention;

[0028] Figure 2 A flowchart of a data transmission method provided in an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of the parameter structure and sorting mesh of a three-dimensional Gaussian element provided in an embodiment of the present invention;

[0030] Figure 4 A comparison diagram of frame storage structures for 3DGS parameters and YUV format data provided in an embodiment of the present invention;

[0031] Figure 5 A flowchart illustrating a data transmission method provided in an embodiment of the present invention;

[0032] Figure 6 This is a schematic diagram of the structure of a data transmission device provided in an embodiment of the present invention;

[0033] Figure 7 This is a schematic diagram of a data transmission device provided in an embodiment of the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] First, let me explain some of the terms used in this application:

[0036] ISOBMFF: Short for ISO Base Media File Format, it is an ISO-based media file format. It does not define the encoding methods for audio and video (such as H.264, AAC), but is only responsible for "packaging" these encoded data streams, similar to the role of a "folder"—placing different content such as video, audio, subtitles, and cover art into corresponding "subfolders" (boxes) according to rules.

[0037] ISOBMFF's core positioning: (1) Universality: Supports all media types, including video, audio, images, subtitles, metadata, etc. (2) Scalability: Allows for the customization of new "box" types to adapt to new media needs (such as VR (Virtual Reality) and Dolby Atmos). (3) Cross-platform compatibility: It is the "mother" of mainstream file formats such as MP4 (Moving Picture Experts Group 4 format), MOV (QuickTime video format), F4V (Flash video format), and 3GP (3rd Generation Partner Program format), and is compatible with devices such as computers, mobile phones, and televisions.

[0038] ISOBMFF Core Structure: Nested "Boxes" Logic. All ISOBMFF data is encapsulated within "boxes," which can nest sub-boxes to form a tree structure. The top-level box is called the File Type Box (ftyp), used to identify the file type (e.g., the "ftyp" value for MP4 is "isom" or "mp42"). Key Top-Level Boxes and Their Functions: The top-level boxes are the "first-level directories" of the file. The core includes the following three types, through which all audio and video data is organized: ftyp (File Type Box): The file type box, which must be placed at the very beginning of the file. It declares the ISOBMFF sub-standards the file follows (e.g., MP4, MOV), ensuring the player can recognize the file format. moov (Movie Box): The movie box, the "index core" of the entire file, containing descriptive information for all media. It records metadata such as video duration, resolution, and audio sampling rate, as well as the index of the media stream (e.g., the "trak" box). mdat (Media Data Box): The media data box, storing the encoded raw audio and video data streams (e.g., H.264 NALU units, AAC ADTS frames). It is the "data warehouse" of files, and its volume usually accounts for more than 90% of the file size.

[0039] Typical applications of ISOBMFF: association with mainstream file formats. ISOBMFF is not a specific file format, but a "standard template." Mainstream audio and video formats are all extensions of it, differing only in the "box type" and "extension rules": MP4's most basic extension, with the core boxes being "ftyp(isom)," "moov," and "mdat" for general audio and video storage (download, on-demand); MOV, an extension defined by Apple, supports more metadata (such as editing information) for professional video editing (Final Cut); a simplified 3GP extension, small in size, supporting teleconferencing, short videos, and low-bitrate videos on mobile devices; and F4V, an extension defined by Adobe, optimizing streaming media transmission and supporting H.264 Flash players and online live streaming.

[0040] SEI messages, short for Supplemental Enhancement Information, are special units in mainstream video coding standards such as H.264 and H.265. Their core function is to carry auxiliary information in the video stream that is not essential for decoding, like attaching "notes" to the video data. Even if discarded, it does not affect the basic video decoding and playback, but it can significantly enhance the performance of video transmission, decoding, and display. The structure of an SEI message mainly consists of three parts: payload type, payload size, and payload data, as detailed below:

[0041] The payloadType is used to distinguish the specific type of SEI message, such as timing information, error recovery information, etc. Its number of bytes is not fixed. During parsing, bytes need to be read continuously until they are not 0xFF, and the final type value is obtained by accumulating the read values.

[0042] payloadSize identifies the length of the payload data in bytes. The parsing logic is the same as payloadType, and it is obtained by accumulating the byte values ​​before non-0xFF, thus supporting payload data of arbitrary length.

[0043] The auxiliary information actually carried by the payload varies in format depending on the payloadType. For example, user-defined data types may contain a 16-byte UUID and specific user data, while timing types may contain parameters such as frame delay and display structure.

[0044] 3D Gaussian data is a special data structure used to represent 3D scenes. A 3D scene consists of N 3D Gaussian primitives, and each 3D Gaussian primitive can be viewed as a 3D Gaussian ellipsoid, such as... Figure 1 As shown, Figure 1This invention provides a 3D Gaussian example image. Each 3D Gaussian primitive is represented by a series of parameters, including: center point coordinates (3D vector), size (3D vector), rotation information (4D vector), color spherical harmonic coefficients (the dimension of the coefficients depends on the order of the spherical harmonic function SHdegree, a 3*[(SHdegree+1)^2] dimensional vector), and opacity (1-dimensional scalar). The generation and presentation process of the 3D Gaussian data is as follows: A camera captures photos / videos from multiple angles; an algorithm reconstructs the original point cloud data corresponding to the current scene from the captured photos / videos; the original point cloud data is represented as M 3D Gaussian primitives; through machine learning training, the initial M 3D Gaussian primitives are continuously optimized, and finally N 3D Gaussian primitives are obtained, which is the 3D Gaussian data representation of the current scene; based on the user's viewing / operation behavior, K (K<=N) 3D Gaussian primitives corresponding to the current viewpoint are obtained; a projection algorithm is used to obtain a 2D image corresponding to the K 3D Gaussian primitives; and the 2D image is presented to the user, realizing 6DoF (Six Degrees of Freedom) real-time interactive volumetric video presentation.

[0045] When representing 3D scenes using 3D Gaussian data, millions of 3D Gaussian primitives are typically required to ensure a realistic 3D scene representation. Therefore, representing 3D scenes using 3D Gaussian data often implies a massive amount of stored data. To address this challenge, it is necessary to continuously explore methods to reduce the data volume of 3D Gaussian scenes, including reducing the number of 3D Gaussian primitives and compressing the data. One such method is to hierarchically divide the N primitives of a 3D Gaussian scene. By dividing the N primitives into L levels, each level corresponds to M... i This invention proposes a three-dimensional Gaussian data transmission method for scenarios involving hierarchical partitioning of three-dimensional Gaussian data, to support adaptive transmission of such data. The method allows clients to transmit / decode corresponding levels of three-dimensional Gaussian Gaussian primitives according to their needs, reducing transmission bandwidth or decoding resource consumption.

[0046] Please refer to the details. Figure 2 , Figure 2 A flowchart illustrating a data transmission method provided in an embodiment of the present invention. The method may include:

[0047] S101: Parse the extended three-dimensional Gaussian detail layered descriptor in the transmission signaling, obtain the first-level information metadata, and request the media resources corresponding to the target layer from the transmission end based on the first-level information metadata.

[0048] In this embodiment, the execution entity is the receiving end, which can also be understood as the decoding end. The first-level information metadata can at least include the level information and the corresponding media resource information; the target level can be determined based on the performance of the receiving end. A descriptor is a structured metadata carrier used to define and transmit information on core technical features such as the level of detail of 3D Gaussian data, media resources, and bitstream / file attributes.

[0049] In this embodiment, the transmission signaling can be DASH signaling or SMT signaling. DASH (Dynamic Adaptive Streaming over HTTP) is an adaptive streaming media transmission technology standard. Its core is to divide media content into slices with different bitrates and resolutions. The terminal can dynamically select and adapt the slices according to the real-time network and hardware performance to achieve smooth playback. SMT (Streaming Media Transport) is responsible for transmitting metadata of media resources (such as layering information, dependencies, etc.) during transmission to guide the receiving end in resource selection and parsing.

[0050] In the DASH standard, descriptors are typically used to describe the characteristics of media resources, including media type, resolution, bitrate, and quality level. Descriptors can describe media resources at different levels. In DASH, the most basic media resource is a representation, such as the data of a video track or audio track. One or more representations of the same content can form an adaptation set, such as representations of the same video content at different resolutions. For 3D Gaussian splashed representation, its different detail levels can exist as different representations or adaptation sets. Therefore, this invention adds a 3D Gaussian detail level descriptor to indicate information about different detail levels in the 3D Gaussian splashed representation.

[0051] Furthermore, the aforementioned 3D Gaussian detail level descriptor includes at least an identifier (i.e., gslod@lodId) that can represent the 3D Gaussian detail level; gslod@lodId is used to indicate the identifier of the 3D Gaussian detail level, and a larger value indicates a higher level of detail. For example, in DASH, please refer to Table 1, which is a table of attributes and semantic examples for the 3D Gaussian detail level descriptor.

[0052] Table 1. Examples of attributes and semantics of 3D Gaussian detail layering descriptors

[0053]

[0054] Similar to DASH, SMT can use the above descriptors to describe the corresponding media assets or asset groups.

[0055] Furthermore, the aforementioned first-level information metadata may also include the associations and dependencies between different levels, as well as the quality and presentation differences between different levels. The hierarchical associations and dependencies in this embodiment clearly define the hierarchical attribution and data reuse logic between different detail levels, preventing decoding failures or screen tearing caused by the receiver mistakenly selecting incompatible level combinations, thus improving the robustness of transmission and decoding. The level quality and presentation difference information (such as the number of Gaussian points, texture fidelity, and rendering frame rate corresponding to different levels) allows the receiver to accurately match the target level with the optimal performance and experience based on its own hardware performance and network bandwidth, avoiding both the waste of computing power caused by selecting an over-configured level and the experience degradation caused by selecting an under-configured level.

[0056] In the HTTP-based Dynamic Adaptive Streaming Media (DASH / SMT) signaling system, the representation is the core carrier of layered 3D Gaussian volumetric video data. Addressing the needs of VR / AR (Augmented Reality) six-DOF real-time interactive scenarios, the 3D Gaussian volumetric video is divided into multiple different representations according to detail granularity. Each representation corresponds to an independent Level of Detail (LOD), which encapsulates the 3D Gaussian parameter data, the mapping information between the data and 2D plane frames, and the hierarchical dependencies within that level. The receiving end, by parsing the extended 3D Gaussian LOD descriptor in the DASH / SMT signaling, can accurately identify the hierarchical attributes corresponding to each representation. Combining its own hardware performance and network bandwidth, it autonomously selects the media resources corresponding to the target representation for request, without needing to acquire the full level of data. This significantly reduces transmission bandwidth consumption and terminal decoding computational overhead, ensuring smooth interaction and high-quality presentation of volumetric video in VR / AR scenarios.

[0057] S102: Parse the extended second-level information metadata in the media resources, and extract the three-dimensional Gaussian data corresponding to the target layer based on the parsed information.

[0058] In this embodiment, the media resources refer to a structured data set that is encoded and encapsulated, corresponding one-to-one with a specific Level of Detail (LOD) of the 3D Gaussian volumetric video. The media resources in this embodiment carry extended second-level information metadata, which includes at least the level information and the corresponding 3D Gaussian data information; it may also include the association and dependency relationships between this level and other levels, and the quality and presentation differences information corresponding to the level, supporting the receiving end to accurately extract the 3D Gaussian data of the target level without processing the full data. It should be noted that the 3D Gaussian data in this embodiment can be six-degree-of-freedom 3D Gaussian reconstruction data in VR or AR real-time interactive scenarios. This embodiment can be specifically applied to VR or AR real-time interactive scenarios, or digital twin scenarios modeled using 3D Gaussian, etc.

[0059] Furthermore, the extended second-level information metadata in the aforementioned parsed media resources, based on the parsed information, extracts the three-dimensional Gaussian data corresponding to the target layer. Specifically, this may include: if the media resource is a file, decapsulating the file to obtain a three-dimensional Gaussian data track, decoding the data in the three-dimensional Gaussian data track to obtain the three-dimensional Gaussian data corresponding to the target layer; if the media resource is a bitstream, decoding the bitstream to obtain the three-dimensional Gaussian data of the target layer. In this embodiment, when the media resource exists in the form of a packaged file, the receiving end first performs a decapsulation operation on the file, stripping away redundant information in the packaged format and separating an independent track dedicated to carrying 3D Gaussian data. Then, based on the hierarchical identifier and data mapping relationship recorded in the second-level information metadata, the receiving end decodes the specified data segment within the track, ultimately extracting the 3D Gaussian data corresponding to the target detail level. When the media resource is transmitted in the form of a streaming bitstream, the receiving end does not need a decapsulation step. It can directly decode the data packet corresponding to the target level in the bitstream according to the guidance of the second-level information metadata to quickly obtain the target 3D Gaussian data. This design is compatible with both file and streaming transmission formats, improving the adaptability and practicality of the solution.

[0060] Furthermore, if the media resource is a file, the file is decapsulated to obtain a 3D Gaussian data track. The data in the 3D Gaussian data track is decoded to obtain the 3D Gaussian data corresponding to the target level. Specifically, this may include: decapsulating the file to separate and obtain the 3D Gaussian detail level track; parsing the 3D Gaussian detail level track to obtain the level identifier corresponding to the target level; locating the encoded data segment in the 3D Gaussian detail level track that matches the target level based on the level identifier corresponding to the target level; and decoding the encoded data segment to obtain the 3D Gaussian data corresponding to the target level.

[0061] In this embodiment, the file is a file with a newly added 3D Gaussian data track via an extended data box. When 3D Gaussian data is encapsulated into a file using an ISOBMFF data box, the 3D Gaussian data can be encapsulated into one or more 3D Gaussian detail level tracks according to detail layers. In this embodiment, the aforementioned 3D Gaussian data track is a 3D Gaussian detail level track. When the media resource is in file format, the file encapsulating the 3D Gaussian data is first decapsulated to remove redundant information from the container format and separate the 3D Gaussian detail level track dedicated to carrying the data of each detail layer; then, the preset layer description information in the track is parsed to filter out the target layer identifier that matches the requirements of the receiving end; finally, using the layer identifier as an index, the encoded data segment corresponding to the target layer is accurately located in the track, and only the data segment is decoded to obtain the 3D Gaussian data of the target layer. The entire process does not require processing of the full amount of data, effectively improving decoding efficiency and resource utilization.

[0062] Furthermore, steps for validating layer validity and associating dependent data can be added to avoid invalid parsing and ensure data integrity. Specifically, this can be achieved by decapsulating the media resource file to separate the 3D Gaussian detail level tracks; parsing the sample entry syntax structure of the 3D Gaussian detail level tracks to read the total number of detail layers corresponding to the current track and the layer identifier corresponding to the target level; comparing the target level identifier with the total number of detail layers, if the target level identifier exceeds the total number of layers, it is determined to be an invalid level and the parsing is terminated; if it does not exceed the total number of layers, the subsequent steps continue; if the sample entry syntax structure indicates that the current track has data dependencies, it is associated with the dependent lower-level 3D Gaussian detail level tracks and the corresponding basic data is obtained; based on the layer identifier of the target level, the encoded data segment matching the target level in the 3D Gaussian detail level track is located, and the encoded data segment is decoded to obtain the 3D Gaussian data corresponding to the target level. As can be seen, this embodiment first decapsulates the media resource file to separate the detail-level tracks storing the 3D Gaussian layered data; then it parses the sample entry syntax structure of the track to read the total number of detail layers and the identifier of the target layer; by comparing the target layer identifier with the total number of layers, invalid layer requests that exceed the range are filtered out; if the track has data dependencies, it first associates and obtains the low-level basic data that it depends on; finally, it locates and decodes the corresponding encoded data segment according to the target layer identifier to obtain the usable target-level 3D Gaussian data.

[0063] The following explains the syntax for entering a 3D Gaussian detail level orbit sample:

[0064] aligned(8) class 3DGSLoDSampleEntry extends SampleEntry('gspe') {

[0065] unsigned int(8) gsi_num_lod

[0066] unsigned int(4) gsi_lod_dependency_idc

[0067] for(i = 0; i < gsi_num_lod; i++ ){

[0068] unsigned int(8) gsi_lod_id[i]

[0069] unsigned int(32) gsi_mapping_width_max[i]

[0070] unsigned int(32) gsi_mapping_height_max[i]

[0071] unsigned int(32) gsi_mapping_width_min[i]

[0072] unsigned int(32) gsi_mapping_height_min[i]

[0073] unsigned int(1) gsi_rcmd_viewspace_flag[i]

[0074] if(gsi_rcmd_viewspace[i] == 1){

[0075] signed int(32) camera_pos_x_min

[0076] signed int(32) camera_pos_y_min

[0077] signed int(32) camera_pos_z_min

[0078] signed int(32) camera_pos_x_max

[0079] signed int(32) camera_pos_y_max

[0080] signed int(32) camera_pos_z_max

[0081] signed int(32) min_distance

[0082] signed int(32) max_distance

[0083] }

[0084] }

[0085] }

[0086] Among them, gsi_num_lod indicates the number of detail layers in the three-dimensional Gaussian data corresponding to the current orbit.

[0087] The semantics of gsi_lod_dependency_idc are the same as in SEI signaling. When the value of gsi_lod_dependency_idc is not 0, the 3D Gaussian detail level orbit needs to be associated with other lower-level 3D Gaussian detail level orbits it depends on through orbit references.

[0088] gsi_lod_id[i] indicates the identifier of the detail layer of the i-th 3D Gaussian splash representation. The larger the value of this field, the higher the detail layer.

[0089] gsi_mapping_width_max[i] and gsi_mapping_height_max[i] indicate the maximum resolution width and height when the detail layers of the i-th 3D Gaussian splash representation are projected onto a 2D plane.

[0090] gsi_mapping_width_min[i] and gsi_mapping_height_min[i] indicate the minimum resolution width and height when the detail layer of the i-th 3D Gaussian splash representation is projected onto a 2D plane.

[0091] When each LOD is rendered independently, the i-th LOD refers to the i-th LOD alone; when each LOD is not rendered independently, the i-th LOD refers to the set of all LODs accumulated up to the i-th LOD.

[0092] When gsi_rcmd_viewspace_flag[i] is 1, it indicates the recommended viewing space range for the i-th LOD; when gsi_rcmd_viewspace_flag[i] is 0, it does not indicate the recommended viewing space range for the i-th LOD, where i is the detail layer index.

[0093] `camera_pos_x_min`, `camera_pos_y_min`, and `camera_pos_z_min` represent the minimum values ​​of the x, y, and z coordinates in the view space within the global reference coordinate system, respectively. These values ​​should be defined as follows: The unit is meters.

[0094] `camera_pos_x_max`, `camera_pos_y_max`, and `camera_pos_z_max` represent the maximum values ​​of the x, y, and z coordinates in the view space within the global reference coordinate system, respectively. These values ​​should be based on... The unit is meters.

[0095] `min_distance` and `max_distance` indicate the minimum and maximum viewing distances, respectively. These values ​​should be set to... The unit is meters.

[0096] Furthermore, if the media resource is a bitstream, the bitstream is decoded to obtain the target level's 3D Gaussian data. Specifically, this may include: parsing the 3D Gaussian level information message carried in the bitstream; determining the validity of the 3D Gaussian data in the bitstream and the level function's enabled status based on the cancellation flag, persistence flag, and LOD enable flag in the 3D Gaussian level information message; when the LOD enable flag is 1, locating the encoded data segment in the bitstream corresponding to the target level based on the total number of LODs, the target level ID, and the mapping unit ID in the 3D Gaussian level information message; and decoding the encoded data segment to obtain the target level's 3D Gaussian data. This embodiment first parses the 3D Gaussian hierarchy information message carried in the bitstream. By using the cancellation flag, persistence flag, and LOD enable flag in the message, it determines whether the 3D Gaussian data in the bitstream is valid, whether it needs to be persistently stored, and whether the layering function is enabled. When the LOD enable flag is 1 (i.e., the layering function is enabled), it accurately locates the encoded data segment corresponding to the target hierarchy in the bitstream based on the total number of layers, the target hierarchy ID, and the mapping unit ID in the message. Finally, only this data segment is decoded to obtain the 3D Gaussian data of the target hierarchy, avoiding invalid processing of the entire bitstream.

[0097] In this embodiment, Supplemental Enhancement Information (SEI) signaling messages, after grammatical expansion, are embedded in the bitstream to carry 3D Gaussian hierarchy information. SEI signaling is auxiliary information appended to the encoded bitstream. By expanding the grammatical structure of SEI signaling, dedicated description fields for the 3D Gaussian Level of Detail (LOD) (such as gsi_num_lod, gsi_lod_dependency_idc, etc.) are added. This allows the decoding end to obtain key information such as the number of layers, hierarchy dependencies, and resolution range before parsing the core 3D Gaussian data, thereby achieving accurate extraction of the target hierarchy data without processing the full dataset. In this embodiment, the syntax structure of the 3D Gaussian hierarchy information message may specifically include: cancellation flag gsi_cancel_flag, persistence flag gsi_persistence_flag, and LOD enable flag gsi_lod_enable_flag; when the LOD enable flag is 1, it also includes LOD mapping type gsi_lod_mapping_idc, LOD dependency type gsi_lod_dependency_idc, and total number of LODs gsi_num_lod; for each LOD hierarchy, it also includes hierarchy ID gsi_lod_id[i] and the number of mapping units in the hierarchy gsi_num_mapping_unit[i]; for each mapping unit, it also includes mapping unit ID gsi_mapping_unit_id [i][j], padding flag gsi_mapping_unit_padding_flag[i][j]; when the padding flag is 1, it also includes the number of padding samples gsi_num_padding_sample[i][j]; when the LOD dependency type is 2, it also includes the number of dependent units of the mapping unit gsi_num_depended_unit[i][j] and the corresponding dependent unit ID gsi_depended_unit_id[i][j][k]. i is the LOD level index, j is the mapping unit index, and k is the dependent unit index; the cancellation flag, persistence flag, LOD enable flag, and padding flag are all 1-bit unsigned integers, the LOD mapping type and LOD dependency type are both 2-bit unsigned integers, the number of mapping units, the number of dependent units, the mapping unit ID (Identifier), and the dependent unit ID are all 8-bit unsigned integers, and the total number of LODs, level IDs, and the number of padding samples all use exponential Golomb encoding. For details, please refer to Table 2. Table 2 is the syntax structure table of the three-dimensional Gaussian hierarchical information message.

[0098] Table 2 Syntax Structure of 3D Gaussian Hierarchical Information Messages

[0099]

[0100] The 3D Gaussian hierarchy message is used to indicate the hierarchy information of the 3D Gaussian splash representation.

[0101] When `gsi_cancel_flag` is set to 1, it means that the current SEI message cancels the continued effect of any 3D Gaussian level messages that precede it in the output order. A value of 0 indicates the specific information of the current 3D Gaussian level message.

[0102] `gsi_persistence_flag` indicates the duration of the current 3D Gaussian level message's validity. A value of 0 indicates that the current 3D Gaussian level message only applies to the currently accessed unit. A value of 1 indicates that the current 3D Gaussian level message applies to the currently accessed unit and remains valid until one of the following conditions is met:

[0103] A new video sequence begins; the current bitstream ends; the access unit output after the current access unit contains a new 3D Gaussian hierarchy message. An access unit is a set of Picture Units (PUs) belonging to different layers. This set of Picture Units contains several interrelated encoded frames that need to be output simultaneously from the Decode Picture Buffer (DPB). Essentially, it is a basic unit encapsulating the encoded image and associated data that need to be processed collaboratively at a given moment. In simple terms, one access unit is equivalent to one video frame.

[0104] When gsi_lod_enable_flag is 1, it means that within the current SEI message's effective range, 3D Gaussian splashing indicates the presence of detail layering; when it is 0, it means that within the current SEI message's effective range, 3D Gaussian splashing indicates the absence of detail layering.

[0105] `gsi_lod_mapping_idc` indicates how different detail layers of the 3D Gaussian sputtering representation are mapped to the bitstream structure. A value of 0 indicates that a detail layer of the 3D Gaussian sputtering representation is mapped to one or more sub-pictures of the bitstream; a value of 1 indicates that a detail layer of the 3D Gaussian sputtering representation is mapped to one or more slices of the bitstream; and a value of 2 indicates that a detail layer of the 3D Gaussian sputtering representation is mapped to one or more tiles of the bitstream.

[0106] `gsi_lod_dependency_idc` indicates the dependencies between different detail layers in a 3D Gaussian splash representation. A value of 0 indicates that there are neither encoding / decoding dependencies nor rendering dependencies between different detail layers in a 3D Gaussian splash representation (i.e., different detail layers can be rendered independently); a value of 1 indicates that there are no encoding / decoding dependencies between different detail layers in a 3D Gaussian splash representation, but there are rendering dependencies (i.e., higher detail layers must be rendered simultaneously with all detail layers with lower values); a value of 2 indicates that there are both encoding / decoding dependencies and rendering dependencies between different detail layers in a 3D Gaussian splash representation.

[0107] gsi_num_lod indicates the number of detail layers in a three-dimensional Gaussian splash representation.

[0108] gsi_lod_id[i] indicates the identifier of the detail layer of the i-th 3D Gaussian splash representation. The larger the value of this field, the higher the detail layer.

[0109] gsi_num_mapping_unit[i] indicates the number of bitstream structure units corresponding to the detail layer of the i-th 3D Gaussian splash representation.

[0110] `gsi_mapping_unit_id[i][j]` indicates the identifier of the `j`-th bitstream structure unit corresponding to the detail layer of the `i`-th 3D Gaussian splash representation. The meaning of this field depends on the value of `gsi_lod_mapping_idc`. When `gsi_lod_mapping_idc` is 0, this field corresponds to the sub-picture id; when `gsi_lod_mapping_idc` is 1, this field corresponds to the slice id; and when `gsi_lod_mapping_idc` is 2, this field corresponds to the tile id.

[0111] When gsi_mapping_unit_padding_flag[i][j] is 1, it means that there is an invalid 3D Gaussian primitive component in the j-th bitstream structure unit corresponding to the detail layer of the i-th 3D Gaussian splash representation; when it is 0, it means that there is no invalid 3D Gaussian primitive component in the j-th bitstream structure unit corresponding to the detail layer of the i-th 3D Gaussian splash representation.

[0112] `gsi_num_padding_sample[i][j]` indicates the number of invalid 3D Gaussian primitive components that fill the j-th bitstream structure unit corresponding to the detail layer of the i-th 3D Gaussian splash representation. Furthermore, the samples corresponding to the invalid 3D Gaussian primitive components are fixed at the last `gsi_num_padding_sample[i][j]` samples in the corresponding bitstream structure unit during the raster scan sequence.

[0113] gsi_num_depended_unit[i][j] indicates the number of bitstream structure units that the j-th bitstream structure unit, corresponding to the detail layer of the i-th 3D Gaussian splash representation, depends on during decoding.

[0114] gsi_depended_unit_id[i][j][k] indicates the identifier of the k-th bitstream structure unit that the j-th bitstream structure unit corresponding to the detail layer of the i-th 3D Gaussian splash representation depends on during decoding.

[0115] S103: Decodes and renders 3D Gaussian data.

[0116] In this step, decoding involves inversely processing the extracted target-level 3D Gaussian data to reconstruct a set of 3D Gaussian primitives containing core attributes such as 3D coordinates, color, covariance, and opacity. This eliminates redundant information introduced during encoding and provides directly processable raw Gaussian parameter data for subsequent rendering. Rendering, on the other hand, addresses the needs of VR / AR six-DOF real-time interactive scenarios. Based on the user's current perspective and interactive commands, primitives within the visible range are selected from the decoded Gaussian primitive set and mapped to a 2D planar image using a projection algorithm. The final output is a volumetric video image with free-viewpoint interaction capabilities that the user can view.

[0117] For example, this embodiment also provides a specific method for mapping three-dimensional Gaussian elements to planar frames: each attribute component of a three-dimensional Gaussian element is considered as a pixel, and the data of N three-dimensional Gaussian elements is mapped to K planar frames, each planar frame containing M pixels (M>=N). K depends on the number of attribute components of each Gaussian element and the representation of the attributes, so as to... Figure 3 For example, Figure 3 This is a schematic diagram of the parameter structure and sorting mesh of a three-dimensional Gaussian element provided in an embodiment of the present invention. K = 4 + 3 + 3 + 1 + 48 = 59. M may be greater than N because some padding may be needed to ensure that the planar frame is rectangular. Rotation is the rotation parameter. `x` represents the rotational component of the Gaussian element in 3D space, used to describe its spatial orientation; `position` is the position parameter, and `x / y / z` are the spatial coordinates of the Gaussian element in the 3D coordinate system; `scale` is the scaling parameter. is the scaling factor of the Gaussian primitive in three dimensions, determining its shape and size; Opacity is the opacity parameter (represented by 'o'), controlling the transparency of the Gaussian primitive during rendering; Color is the color parameter. The corresponding RGB three-channel color values; spherical harmonics are the parameters of the spherical harmonic function. It is a high-order feature used to characterize the appearance of Gaussian primitives under different lighting conditions (such as light and shadow, reflection); sorted grids refers to the structured data set formed by sorting the above Gaussian primitive parameters according to rules; Gaussian Splat is the core technology of 3D Gaussian primitive rendering, which generates the final visualization image by projecting Gaussian primitives onto a 2D plane and mixing them.

[0118] Furthermore, one could consider rearranging the K planar frames corresponding to different attribute components into L planar frames (K>L), such as... Figure 4 As shown. Figure 4 This is a comparison diagram of the frame storage structure for 3DGS parameters and YUV format data provided in an embodiment of the present invention. It illustrates the planar and temporal storage methods of 3D Gaussian data (3DGS) under different YUV sampling formats: The upper Planar area shows the planar storage layout of 3DGS coordinate parameters (af) within a single frame, using YUV400 / 420 / 444 sampling formats as examples. Different formats correspond to different parameter arrangement densities. The lower Temporal area shows the temporal storage method of 3DGS parameters under multiple frames (e.g., 6 frames, 2 frames), reflecting the stacking logic of parameters over time. The examples clearly indicate the corresponding positions of 3DGS parameters (including coordinates, scaling, etc.) in the storage structure, intuitively presenting the storage design for 3DGS data adapted to traditional video formats. It can be seen that the arrangement can be spatial or temporal. Furthermore, a video frame contains N layers, each layer being a part of the data in the complete video frame. For example, K attribute components can be considered as K layers.

[0119] The data transmission method provided in this invention involves the following steps: S101: parsing the extended 3D Gaussian detail layered descriptor in the transmission signaling to obtain first-level information metadata, and requesting the media resource corresponding to the target layer from the transmitting end based on the first-level information metadata; S102: parsing the extended second-level information metadata in the media resource, and extracting the 3D Gaussian data corresponding to the target layer based on the parsed information; S103: decoding and rendering the 3D Gaussian data. This invention combines the layered 3D Gaussian data with encoding and transmission technologies, expanding the metadata system of transmission signaling and media resources. By setting standardized layered metadata indication information, it supports adaptive transmission and decoding based on layered information. Standardized metadata extensions across the entire chain can effectively ensure the consistency of hierarchical information in each stage of encoding, encapsulation, transmission, and decoding, avoiding hierarchical alignment errors caused by signaling fragmentation, improving decoding robustness and interoperability between different devices, and reducing technology deployment costs. The receiving end can accurately obtain the target level of 3D Gaussian data based on the metadata without having to decode the entire data, significantly reducing terminal computing power overhead. This meets the stringent requirements of VR / AR six-degrees-of-freedom real-time interactive scenarios for 3D Gaussian data in terms of transmission efficiency, terminal adaptability, and immersive presentation quality. Furthermore, it achieves on-demand transmission and efficient parsing of 3D Gaussian data. The receiving end does not need to acquire the full 3D Gaussian data; it only needs to accurately locate and request the media resources corresponding to the target layer by parsing the first-level metadata in the transmission signaling based on its own hardware performance, network bandwidth, and other conditions. This significantly reduces the bandwidth consumption and transmission time of data transmission. At the same time, with the guidance of the second-level metadata in the media resources, the 3D Gaussian data of the target layer can be quickly extracted, avoiding the terminal computing power consumption caused by full data decoding, shortening the decoding latency, and improving data processing efficiency. In addition, the standardized two-level metadata parsing logic unifies the transmission and parsing specifications of 3D Gaussian layered data, enhances the compatibility of the solution with existing streaming media transmission protocols and media encapsulation formats, and can be widely adapted to heterogeneous terminals such as mobile terminals, PC terminals, and VR devices, ensuring the smoothness and stability of 3D Gaussian data rendering in different scenarios.

[0120] For a clearer understanding of this invention, please refer to the following details. Figure 5 , Figure 5 A flowchart illustrating a data transmission method provided in an embodiment of the present invention may specifically include:

[0121] The receiving / decoding end parses the hierarchical information metadata in the transmission signaling. This metadata includes: hierarchical information and corresponding media resource information; the relationships and dependencies between different hierarchies; and difference information between different hierarchies, including quality and presentation differences. The receiving / decoding end requests the corresponding media resources based on the hierarchical information. The receiving / decoding end parses the hierarchical information metadata in the file / bitstream. This metadata includes: hierarchical information and corresponding 3D Gaussian data information: i.e., the mapping information between data and hierarchies in the file / bitstream; the relationships and dependencies between different hierarchies; and the difference information between different hierarchies, including quality and presentation differences. Based on the hierarchical information, the receiving / decoding end obtains the 3D Gaussian data corresponding to the target hierarchies: if the receiving / decoding end obtains a file, it needs to undergo decapsulation and decoding to obtain the 3D Gaussian data corresponding to the target hierarchies; if the receiving / decoding end obtains a bitstream, it needs to undergo decoding to obtain the 3D Gaussian data corresponding to the target hierarchies. The receiving / decoding end decodes and presents the 3D Gaussian data corresponding to the target hierarchies.

[0122] The data transmission device provided in the embodiments of the present invention will be described below. The data transmission device described below and the data transmission method described above can be referred to in correspondence.

[0123] Please refer to the details. Figure 6 , Figure 6 A schematic diagram of a data transmission device provided in an embodiment of the present invention may include:

[0124] The first parsing module 100 is used to parse the extended three-dimensional Gaussian detail layered descriptor in the transmission signaling, obtain the first-level information metadata, and request the media resources corresponding to the target layer from the transmission end based on the first-level information metadata; the first-level information metadata includes at least the layer information and the media resource information corresponding to the layer information; the target layer is determined based on the performance of the receiving end.

[0125] The second parsing module 200 is used to parse the extended second-level information metadata in the media resource and extract the three-dimensional Gaussian data corresponding to the target layer based on the parsed information; the second-level information metadata includes at least the level information and the three-dimensional Gaussian data information corresponding to the level; the three-dimensional Gaussian data is six-degree-of-freedom three-dimensional Gaussian reconstruction data in VR or AR real-time interactive scenarios.

[0126] The decoding and rendering module 300 is used to decode and render the three-dimensional Gaussian data.

[0127] Based on any of the above embodiments, the second parsing module 200 may include:

[0128] The first decoding unit is used to decapsulate the file if the media resource is a file to obtain a three-dimensional Gaussian data track, and to decode the data in the three-dimensional Gaussian data track to obtain the three-dimensional Gaussian data corresponding to the target level.

[0129] The second decoding unit is used to decode the bit stream if the media resource is a bit stream, to obtain the target level three-dimensional Gaussian data.

[0130] Based on any of the above embodiments, the file can be a file with added three-dimensional Gaussian data tracks through an extended data box; the bit stream is embedded with a supplementary enhancement information signaling message after syntax expansion to carry three-dimensional Gaussian hierarchy information messages.

[0131] Based on any of the above embodiments, the second decoding unit may include:

[0132] The first parsing subunit is used to parse the three-dimensional Gaussian hierarchy information message carried in the bit stream, and determine the validity of the three-dimensional Gaussian data in the bit stream and the hierarchical function enable status based on the cancellation flag, persistence flag and LOD enable flag in the three-dimensional Gaussian hierarchy information message.

[0133] The first positioning subunit is used to locate the encoded data segment in the bit stream corresponding to the target level according to the total number of LODs, the target level ID, and the mapping unit ID in the three-dimensional Gaussian hierarchy information message when the LOD enable flag is 1.

[0134] The first decoding subunit is used to decode the encoded data segment to obtain the target-level three-dimensional Gaussian data.

[0135] Based on any of the above embodiments, the first decoding unit may include:

[0136] The decapsulation processing subunit is used to decapsulate the file and separate it into three-dimensional Gaussian detail level orbits;

[0137] The second parsing subunit is used to parse the three-dimensional Gaussian detail level track and obtain the level identifier corresponding to the target level.

[0138] The second positioning subunit is used to locate the encoded data segment that matches the target level in the three-dimensional Gaussian detail level track based on the level identifier corresponding to the target level, and to decode the encoded data segment to obtain the three-dimensional Gaussian data corresponding to the target level.

[0139] Based on any of the above embodiments, the three-dimensional Gaussian detail layer descriptor includes at least an identifier that can include three-dimensional Gaussian detail layer, and the larger the value of the identifier, the higher the level of detail layer.

[0140] Based on any of the above embodiments, the first-level information metadata and the second-level information metadata may further include: the association and dependency relationships between different levels, as well as the quality differences and presentation differences between different levels.

[0141] It should be noted that the order of the modules and units in the aforementioned data transmission device can be changed without affecting the logic.

[0142] The data transmission apparatus provided in this embodiment of the invention comprises a first parsing module 100, used to parse the extended 3D Gaussian detail layering descriptor in the transmission signaling, obtain first-level information metadata, and request media resources corresponding to the target layer from the transmitting end based on the first-level information metadata; the first-level information metadata includes at least layer information and media resource information corresponding to the layer information; the target layer is determined based on the performance of the receiving end; a second parsing module 200, used to parse the extended second-level information metadata in the media resources, and extract the 3D Gaussian data corresponding to the target layer based on the parsed information; the second-level information metadata includes at least layer information and 3D Gaussian data information corresponding to the layer; the 3D Gaussian data is six-degree-of-freedom 3D Gaussian reconstruction data in VR or AR real-time interactive scenarios; and a decoding and rendering module 300, used to decode and render the 3D Gaussian data. This invention combines the layered 3D Gaussian data with encoding and transmission technologies, expanding the metadata system of transmission signaling and media resources, and supporting adaptive transmission and decoding based on layer information by setting standardized layer metadata indication information. Standardized metadata extensions across the entire chain can effectively ensure the consistency of hierarchical information in each stage of encoding, encapsulation, transmission, and decoding, avoiding hierarchical alignment errors caused by signaling fragmentation, improving decoding robustness and interoperability between different devices, and reducing technology deployment costs. The receiving end can accurately obtain the target level of 3D Gaussian data based on the metadata without having to decode the entire data, significantly reducing terminal computing power overhead. This meets the stringent requirements of VR / AR six-degrees-of-freedom real-time interactive scenarios for 3D Gaussian data in terms of transmission efficiency, terminal adaptability, and immersive presentation quality. Furthermore, it achieves on-demand transmission and efficient parsing of 3D Gaussian data. The receiving end does not need to acquire the full 3D Gaussian data; it only needs to accurately locate and request the media resources corresponding to the target layer by parsing the first-level metadata in the transmission signaling based on its own hardware performance, network bandwidth, and other conditions. This significantly reduces the bandwidth consumption and transmission time of data transmission. At the same time, with the guidance of the second-level metadata in the media resources, the 3D Gaussian data of the target layer can be quickly extracted, avoiding the terminal computing power consumption caused by full data decoding, shortening the decoding latency, and improving data processing efficiency. In addition, the standardized two-level metadata parsing logic unifies the transmission and parsing specifications of 3D Gaussian layered data, enhances the compatibility of the solution with existing streaming media transmission protocols and media encapsulation formats, and can be widely adapted to heterogeneous terminals such as mobile terminals, PC terminals, and VR devices, ensuring the smoothness and stability of 3D Gaussian data rendering in different scenarios.

[0143] The data transmission device provided in the embodiments of the present invention will be described below. The data transmission device described below and the data transmission method described above can be referred to in correspondence.

[0144] Please refer to Figure 7 , Figure 7 A schematic diagram of a data transmission device provided in an embodiment of the present invention may include:

[0145] Memory 10 is used to store computer programs;

[0146] Processor 20 is used to execute computer programs to implement the data transmission method described above.

[0147] The memory 10, processor 20, and communication interface 31 all communicate with each other through the communication bus 32.

[0148] In this embodiment of the invention, the memory 10 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment of the invention, the memory 10 may store programs for implementing the following functions:

[0149] The extended 3D Gaussian detail layered descriptor in the transmission signaling is parsed to obtain the first-level information metadata, and the media resources corresponding to the target layer are requested from the transmission end based on the first-level information metadata; the first-level information metadata includes at least the layer information and the media resource information corresponding to the layer information; the target layer is determined based on the performance of the receiving end;

[0150] Parse the extended second-level information metadata in the media resources, and extract the three-dimensional Gaussian data corresponding to the target layer based on the parsed information; the second-level information metadata includes at least the level information and the corresponding three-dimensional Gaussian data information; the three-dimensional Gaussian data is six-degree-of-freedom three-dimensional Gaussian reconstruction data in VR or AR real-time interactive scenarios;

[0151] Decode and render 3D Gaussian data.

[0152] In one possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0153] Furthermore, memory 10 may include read-only memory and random access memory, providing instructions and data to the processor. A portion of the memory may also include NVRAM. The memory stores operating systems and operating instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and handling hardware-based tasks.

[0154] Processor 20 can be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic device. Processor 20 can be a microprocessor or any conventional processor. Processor 20 can call programs stored in memory 10.

[0155] Communication interface 31 can be an interface for the communication module, used to connect with other devices or systems.

[0156] Of course, it should be noted that, Figure 7 The structure shown does not constitute a limitation on the data transmission device in the embodiments of the present invention. In practical applications, the data transmission device may include devices such as... Figure 7 More or fewer components as shown, or combinations of certain components.

[0157] The computer-readable storage medium provided in the embodiments of the present invention is described below. The computer-readable storage medium described below and the data transmission method described above can be referred to in correspondence.

[0158] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described data transmission method.

[0159] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0160] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0161] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0162] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0163] The present invention has provided a detailed description of a data transmission method, apparatus, device, and computer-readable storage medium. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data transmission method, characterized in that, Applied to the receiving end, including: The extended 3D Gaussian detail layered descriptor in the transmission signaling is parsed to obtain the first-level information metadata, and the media resources corresponding to the target layer are requested from the transmission end based on the first-level information metadata; the first-level information metadata includes at least the layer information and the media resource information corresponding to the layer information; the target layer is determined based on the performance of the receiving end; The extended second-level information metadata in the media resource is parsed, and the three-dimensional Gaussian data corresponding to the target layer is extracted based on the parsed information; the second-level information metadata includes at least the level information and the corresponding three-dimensional Gaussian data information; the three-dimensional Gaussian data is six-degree-of-freedom three-dimensional Gaussian reconstruction data in VR or AR real-time interactive scenarios; The three-dimensional Gaussian data is decoded and rendered.

2. The data transmission method according to claim 1, characterized in that, Parse the extended second-level information metadata in the media resource, and extract the three-dimensional Gaussian data corresponding to the target layer based on the parsed information, including: If the media resource is a file, the file is decapsulated to obtain a three-dimensional Gaussian data track, and the data in the three-dimensional Gaussian data track is decoded to obtain the three-dimensional Gaussian data corresponding to the target level. If the media resource is a bitstream, then the bitstream is decoded to obtain the target-level three-dimensional Gaussian data.

3. The data transmission method according to claim 2, characterized in that, The file is a file that adds a three-dimensional Gaussian data track through an extended data box; the bit stream embeds a supplementary enhancement information signaling message after syntax expansion to carry three-dimensional Gaussian level information messages.

4. The data transmission method according to claim 3, characterized in that, If the media resource is a bitstream, then the bitstream is decoded to obtain the target-level three-dimensional Gaussian data, including: The three-dimensional Gaussian hierarchy information message carried in the bit stream is parsed, and the validity of the three-dimensional Gaussian data in the bit stream and the hierarchical function enable status are determined based on the cancellation flag, persistence flag and LOD enable flag in the three-dimensional Gaussian hierarchy information message. When the LOD enable flag is 1, the encoded data segment corresponding to the target level in the bit stream is located according to the total number of LODs, the target level ID and the mapping unit ID in the three-dimensional Gaussian hierarchy information message. The encoded data segment is decoded to obtain the target-level three-dimensional Gaussian data.

5. The data transmission method according to claim 3, characterized in that, If the media resource is a file, the file is decapsulated to obtain a 3D Gaussian data track. The data in the 3D Gaussian data track is decoded to obtain the 3D Gaussian data corresponding to the target level, including: The file is decapsulated to separate the three-dimensional Gaussian detail level orbitals; Analyze the three-dimensional Gaussian detail level orbit to obtain the level identifier corresponding to the target level; Based on the level identifier corresponding to the target level, locate the encoded data segment in the 3D Gaussian detail level orbit that matches the target level, and decode the encoded data segment to obtain the 3D Gaussian data corresponding to the target level.

6. The data transmission method according to claim 1, characterized in that, The 3D Gaussian detail layer descriptor includes at least an identifier for 3D Gaussian detail layering, and the larger the value of the identifier, the higher the level of detail layering.

7. The data transmission method according to claim 1, characterized in that, The first-level information metadata and the second-level information metadata also include: the association and dependency relationships between different levels, as well as the quality differences and presentation differences between different levels.

8. A data transmission device, characterized in that, Applied to the receiving end, including: The first parsing module is used to parse the extended three-dimensional Gaussian detail layered descriptor in the transmission signaling, obtain the first-level information metadata, and request the media resources corresponding to the target layer from the transmission end based on the first-level information metadata; the first-level information metadata includes at least the layer information and the media resource information corresponding to the layer information; the target layer is determined based on the performance of the receiving end; The second parsing module is used to parse the extended second-level information metadata in the media resource, and extract the three-dimensional Gaussian data corresponding to the target layer based on the parsed information; the second-level information metadata includes at least the level information and the corresponding three-dimensional Gaussian data information; the three-dimensional Gaussian data is six-degree-of-freedom three-dimensional Gaussian reconstruction data in VR or AR real-time interactive scenarios. The decoding and rendering module is used to decode and render the three-dimensional Gaussian data.

9. A data transmission device, characterized in that, include: Memory, used to store computer programs; A processor for implementing the data transmission method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the data transmission method as described in any one of claims 1 to 7.

Images