A data decoding method, device, apparatus and computer readable storage medium

Abstract

The application discloses a data decoding method, device and equipment and a computer readable storage medium, and applies to the technical field of data processing, and comprises the following steps: obtaining signaling component information corresponding to a media resource by analyzing three-dimensional Gaussian media signaling; requesting and receiving a target media resource according to the signaling component information; obtaining a target three-dimensional Gaussian file track encapsulated based on an improved ISOBMFF file format by unpacking the target media resource and analyzing file component information; determining a target three-dimensional Gaussian component and a decoding relationship according to the file component information; and performing parallel decoding on multiple components which are independent of each other in the decoding. Compared with the current three-dimensional Gaussian encoding bit stream which lacks unified encapsulation and dependency description, the application encodes the three-dimensional Gaussian code stream through the improved ISOBMFF file format, determines a decoding relationship based on a decoding dependency relationship, and parallel decodes three-dimensional Gaussian components which are independent of each other based on the decoding relationship, so that the decoding and rendering presentation speed is improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data decoding method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] When representing a 3D scene using 3D Gaussian data, millions of 3D Gaussian primitives are typically required to ensure a realistic reconstruction of the scene. Therefore, representing a 3D scene using 3D Gaussian data often implies a huge amount of data storage and transmission bandwidth.

[0003] It is evident that how to encode 3D Gaussian data and improve decoding speed is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a data decoding method, apparatus, device and computer-readable storage medium, which solves the problems of lack of dedicated encapsulation technology for three-dimensional Gaussian code streams and slow decoding speed in the prior art.

[0005] To solve the above-mentioned technical problems, the present invention provides a data decoding method, comprising: Parse the three-dimensional Gaussian media signaling to obtain the signaling component information corresponding to the multiple media resources described by the three-dimensional Gaussian media signaling; the signaling component information includes component identification information, decoding dependency relationship and color level corresponding to spherical harmonic coefficient components, and the component identification information includes component type and component identifier; Based on the signaling component information, a resource request is sent to the server, and at least one target media resource is received from the server based on the resource request; wherein, the target media resource corresponds to a three-dimensional Gaussian file track, and the three-dimensional Gaussian file track is obtained by encapsulating the three-dimensional Gaussian components in the three-dimensional Gaussian encoded bitstream based on the improved ISOBMFF file format; The target media resource is unpacked to obtain the target 3D Gaussian file track corresponding to the target media resource, and the file component information in the target 3D Gaussian file track is parsed; wherein, the file component information is used to describe the component type, component identifier, decoding dependency and color level corresponding to the spherical harmonic coefficient component actually contained in the corresponding target 3D Gaussian file track; Based on the file component information, the target three-dimensional Gaussian component is determined from the target three-dimensional Gaussian file track, and the decoding relationship corresponding to the target three-dimensional Gaussian component is determined based on the decoding dependency relationship; The target three-dimensional Gaussian components are decoded according to the decoding relationship; wherein, when the decoding relationship indicates that the decoding of multiple target three-dimensional Gaussian components is independent of each other, the multiple target three-dimensional Gaussian components are decoded in parallel.

[0006] Optionally, the three-dimensional Gaussian media signaling is parsed to obtain the signaling component information corresponding to the multiple media resources described by the three-dimensional Gaussian media signaling, including: When the three-dimensional Gaussian media signaling is HTTP-based dynamic adaptive streaming media signaling, the three-dimensional Gaussian bitstream descriptor and three-dimensional Gaussian component descriptor of the media representation description file in the three-dimensional Gaussian media signaling are parsed to obtain the component identification information, decoding dependency relationship and color level corresponding to each media resource; When the three-dimensional Gaussian media signaling is intelligent media transmission signaling, the three-dimensional Gaussian bitstream description information and three-dimensional Gaussian component description information of the media resource descriptor in the three-dimensional Gaussian media signaling are parsed to obtain the component identification information, decoding dependency relationship and color level corresponding to each media resource.

[0007] Optionally, sending a resource request to the server based on the signaling component information includes: Based on the color level in the signaling component information, a target color level corresponding to the decoding or presentation requirements is determined, and a resource request for requesting the spherical harmonic coefficient data corresponding to the target color level is sent to the server, so that the server sends the target media resource carrying the spherical harmonic coefficient data; The request is to transmit multiple media resources simultaneously to identify at least two of the target media resources.

[0008] Optionally, after unpacking the target media resource to obtain the target 3D Gaussian file track corresponding to the target media resource, and parsing the file component information in the target 3D Gaussian file track, the method further includes: Obtain the color level enable flag, the level identifier of each color level, and the number of spherical harmonics contained in each color level from the file component information. When the color level enable flag indicates that the spherical harmonic coefficients are divided into different color levels according to their order, the target color level is determined from each of the color levels based on the presentation requirements. Based on the level identifier of the target color level and the corresponding number of spherical harmonic orders, the spherical harmonic coefficient components corresponding to the target color level are extracted from the target 3D Gaussian file track, and the extracted spherical harmonic coefficient components are decoded.

[0009] Optionally, the target media resource is unpacked to obtain the target 3D Gaussian file track corresponding to the target media resource, and the file component information in the target 3D Gaussian file track is parsed, including: Obtain the orbit component information recorded in the three-dimensional Gaussian component information data box in the target three-dimensional Gaussian file orbit; Based on the orbital component information, determine the actual component types and component identifiers contained in the target 3D Gaussian file orbit; When the target 3D Gaussian file track contains decoder configuration information, the decoder configuration information is obtained from the target 3D Gaussian file track; When the target 3D Gaussian file track does not contain the decoder configuration information, the track identifier recorded in the track reference data box in the target 3D Gaussian file track is obtained, the referenced 3D Gaussian file track is determined according to the track identifier, and the decoder configuration information is obtained from the referenced 3D Gaussian file track. Based on the decoder configuration information, the decoding dependency and the color level corresponding to the spherical harmonic coefficient components are determined, and the component types and component identifiers actually contained in the target three-dimensional Gaussian file track, the decoding dependency, and the color level are determined as the file component information.

[0010] Optionally, determining the decoding relationship corresponding to the target three-dimensional Gaussian component based on the decoding dependency includes: The decoding dependency is obtained from the file component information, and based on the decoding dependency, it is determined whether the decoding of the attribute data depends on the geometric data, and whether the decoding of the attribute data depends on other attribute data. When the decoding dependency indicates that the decoding of the attribute data depends on the geometric data, the target three-dimensional Gaussian component corresponding to the geometric data is determined as the dependent component of the target three-dimensional Gaussian component corresponding to the attribute data; When the decoding dependency indicates that the decoding of the attribute data depends on other attribute data, the target three-dimensional Gaussian component corresponding to the other attribute data is determined as the dependent component of the target three-dimensional Gaussian component corresponding to the attribute data. When the decoding dependency indicates that the decoding of the attribute data does not depend on the geometric data and the other attribute data, it is determined that the decoding of the target 3D Gaussian component corresponding to the attribute data is independent of the decoding of the geometric data and the target 3D Gaussian component corresponding to the other attribute data.

[0011] Optionally, determining the target three-dimensional Gaussian components from the target three-dimensional Gaussian file orbit based on the file component information includes: When the sample in the target 3D Gaussian file orbit is divided into multiple sub-samples, the sub-sample information data box in the target 3D Gaussian file orbit is parsed to obtain the sub-sample structure indication information corresponding to each sub-sample, as well as the component type or component identifier corresponding to the sub-sample structure indication information. Based on the signaling component information and the file component information, determine the target component type or target component identifier corresponding to the three-dimensional Gaussian component to be decoded; When the subsample structure indication information is 0 and the component type of the corresponding subsample is the same as the target component type, the corresponding subsample is determined as the target subsample. When the subsample structure indication information is 2, and the component identifier of the corresponding subsample is the same as the target component identifier, the corresponding subsample is determined as the target subsample; The three-dimensional Gaussian components contained in the target subsample are determined as the target three-dimensional Gaussian components.

[0012] The present invention also provides a data decoding device, comprising: The parsing module is used to parse the three-dimensional Gaussian media signaling to obtain the signaling component information corresponding to the multiple media resources described by the three-dimensional Gaussian media signaling; the signaling component information includes component identification information, decoding dependency relationship and color level corresponding to the spherical harmonic coefficient component, and the component identification information includes component type and component identifier. The request module is used to send a resource request to the server based on the signaling component information, and to receive at least one target media resource sent by the server based on the resource request; wherein the target media resource corresponds to a three-dimensional Gaussian file track, and the three-dimensional Gaussian file track is obtained by encapsulating the three-dimensional Gaussian components in the three-dimensional Gaussian encoded bitstream based on the improved ISOBMFF file format; The unpacking module is used to unpack the target media resource to obtain the target 3D Gaussian file track corresponding to the target media resource, and parse the file component information in the target 3D Gaussian file track; wherein, the file component information is used to describe the component type, component identifier, decoding dependency and color level corresponding to the spherical harmonic coefficient component actually contained in the corresponding target 3D Gaussian file track; The decoding relationship determination module is used to determine the target three-dimensional Gaussian component from the target three-dimensional Gaussian file track based on the file component information, and to determine the decoding relationship corresponding to the target three-dimensional Gaussian component based on the decoding dependency relationship; A decoding module is used to decode the target three-dimensional Gaussian components according to the decoding relationship; wherein, when the decoding relationship indicates that the decoding of multiple target three-dimensional Gaussian components is independent of each other, the multiple target three-dimensional Gaussian components are decoded in parallel.

[0013] The present invention also provides a data decoding device, comprising: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the aforementioned data decoding method.

[0014] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described data decoding method.

[0015] The present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-described method.

[0016] As can be seen, this invention obtains signaling component information corresponding to multiple media resources described by the 3D Gaussian media signaling by parsing the signaling component information. The signaling component information includes component identification information, decoding dependencies, and color levels corresponding to spherical harmonic coefficient components. The component identification information includes component type and component identifier. Based on the signaling component information, a resource request is sent to the server, and at least one target media resource is received from the server based on the resource request. Each target media resource corresponds to a 3D Gaussian file track, which is obtained by encapsulating the 3D Gaussian components in the 3D Gaussian encoded bitstream using an improved ISOBMFF file format. The target media resource is then unpacked. The process involves obtaining the target 3D Gaussian file track corresponding to the target media resource and parsing the file component information within the target 3D Gaussian file track. The file component information describes the actual component types, component identifiers, decoding dependencies, and color levels corresponding to the spherical harmonic coefficient components contained in the target 3D Gaussian file track. Based on the file component information, the target 3D Gaussian components are determined from the target 3D Gaussian file track, and the decoding dependencies are used to determine the corresponding decoding relationships. The target 3D Gaussian components are then decoded according to the decoding relationships. When the decoding relationships indicate that the decoding of multiple target 3D Gaussian components is independent, the multiple target 3D Gaussian components are decoded in parallel.

[0017] The beneficial effects of the present invention are as follows: the improved ISOBMFF file format in the present invention is a special encapsulation technology for three-dimensional Gaussian bitstream designed in this application, and the dependency relationship determines whether several can be parallelized, thereby performing parallel decoding and improving the decoding speed.

[0018] In addition, the present invention also provides a data decoding device, apparatus, and computer-readable storage medium, which also have the above-mentioned beneficial effects. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0020] Figure 1 A flowchart of a data decoding method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a data decoding device provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a data decoding device provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Some terms that appear in the description of the embodiments of this application are subject to the following interpretation: ISO Base Media File Format (ISOBMFF) is a universal media container format used for storing and transmitting time-based media data. Its underlying standard is ISO / IEC 14496-12. This format is primarily used to organize video, audio, subtitles, images, metadata, and timing, structural, and encoding / decoding information involved in media presentation.

[0023] Dynamic Adaptive Streaming over HTTP (DASH) is an HTTP-based adaptive streaming technology developed by MPEG, with its relevant standard being SO / IEC 23009.

[0024] Smart Media Transport (SMT) is a media transmission technology for heterogeneous packet-switched networks. The current relevant standard in my country is GB / T 33475.6-2024, "Information Technology - High-Efficiency Multimedia Coding - Part 6: Smart Media Transport".

[0025] Please refer to Figure 1 , Figure 1 A flowchart illustrating a data decoding method provided in an embodiment of the present invention. The method may include: S101, parse the three-dimensional Gaussian media signaling to obtain the signaling component information corresponding to the multiple media resources described by the three-dimensional Gaussian media signaling; the signaling component information includes component identification information, decoding dependency relationship and color level corresponding to the spherical harmonic coefficient component, and the component identification information includes component type and component identifier.

[0026] The execution entity in this embodiment is the client. In this embodiment, the 3D Gaussian media signaling is used to describe the attributes of the media resources before the terminal device actually requests the media resources. The 3D Gaussian media signaling in this embodiment describes the 3D Gaussian component information corresponding to the media resources. The component information (i.e., signaling component information) includes: the type and identifier of the component; the dependency information of the component data on other component data during decoding (i.e., decoding dependency); and the color level information of the spherical harmonic coefficients (for specific component types).

[0027] S102, send a resource request to the server based on signaling component information, and receive at least one target media resource sent by the server based on the resource request; wherein, the target media resource corresponds to a three-dimensional Gaussian file track, and the three-dimensional Gaussian file track is obtained by encapsulating the three-dimensional Gaussian components in the three-dimensional Gaussian encoded bitstream based on the improved ISOBMFF file format.

[0028] This embodiment requests corresponding media resources based on signaling component information and the client's own decoding or rendering needs. For example, when network bandwidth or device rendering capabilities are limited, only the spherical harmonic coefficient data corresponding to lower color levels can be requested based on color level information. When network bandwidth and decoding capabilities are not limited, multiple media resources can be requested to be transmitted simultaneously (one media resource corresponds to one file track) to support parallel decoding by the client and improve rendering speed. The improved ISOBMFF file format in this embodiment refers to adding data structures to describe 3D Gaussian bitstreams, 3D Gaussian components, decoding dependencies, and spherical harmonic coefficient color levels, based on the track, data item, data box, track group, entity group, track reference, data item reference, and subsample mechanism of the ISO basic media file format. The improved ISOBMFF file format in this embodiment is equivalent to the ISOBMFF file format, which is an extension for 3DGS. When the 3D Gaussian data of a scene is encoded into multiple independent 3D Gaussian bitstreams, the tracks corresponding to these 3D Gaussian bitstreams should be associated through track groups.

[0029] S103, unpack the target media resource to obtain the target 3D Gaussian file track corresponding to the target media resource, and parse the file component information in the target 3D Gaussian file track; wherein, the file component information is used to describe the component type, component identifier, decoding dependency and color level corresponding to the spherical harmonic coefficient component actually contained in the corresponding target 3D Gaussian file track.

[0030] In this embodiment, the server sends the media resources requested by the client to the client. Upon receiving the requested target media resources, the client unpacks them and restores them to a 3D Gaussian file track. The client parses the component information (i.e., file component information) in the 3D Gaussian file track level and sample level, including: component type and identifier; dependency information of component data on other component data during decoding; and color level information of spherical harmonic coefficients (for specific component types).

[0031] S104, determine the target three-dimensional Gaussian components from the target three-dimensional Gaussian file track based on the file component information, and determine the decoding relationship corresponding to the target three-dimensional Gaussian components based on the decoding dependency relationship.

[0032] This embodiment can determine the decoding relationship based on decoding dependencies, thereby determining whether to perform parallel decoding. In this embodiment, the client can decode the corresponding file tracks or sub-samples based on its own decoding or rendering needs. For example, when the device's rendering capabilities are limited, based on color level information, only the spherical harmonic coefficient data corresponding to the lower color levels can be decoded and rendered. Based on the 3D Gaussian component information and the component's encoding / decoding independence information, combined with the client's decoding capabilities, it is determined whether to perform parallel decoding. Decoding capability refers to the terminal device's ability to decode the 3D Gaussian components in the 3D Gaussian encoded bitstream using currently available decoding processing resources. Decoding capability meeting the parallel decoding requirement means that the terminal device can simultaneously decode at least two mutually independent 3D Gaussian components without having to decode all 3D Gaussian components serially.

[0033] S105, Decode the target three-dimensional Gaussian components according to the decoding relationship; wherein, when the decoding relationship indicates that the decoding of multiple target three-dimensional Gaussian components is independent of each other, the multiple target three-dimensional Gaussian components are decoded in parallel.

[0034] This embodiment can decode different three-dimensional Gaussian components independently of the encoding and decoding in parallel, thereby improving the rendering speed.

[0035] An embodiment of the present invention provides a data decoding method, which may include: S101, parse the 3D Gaussian media signaling to obtain the signaling component information corresponding to the multiple media resources described by the 3D Gaussian media signaling; the signaling component information includes component identification information, decoding dependency relationship, and color level corresponding to the spherical harmonic coefficient components. The component identification information includes component type and component identifier. S102, send a resource request to the server based on the signaling component information, and receive at least one target media resource sent by the server based on the resource request; wherein, the target media resource corresponds to a 3D Gaussian file track, and the 3D Gaussian file track is obtained by encapsulating the 3D Gaussian components in the 3D Gaussian encoded bitstream based on the improved ISOBMFF file format. S103, unpack the target media resource. The method involves obtaining the target 3D Gaussian file track corresponding to the target media resource and parsing the file component information within the target 3D Gaussian file track. The file component information describes the actual component types, component identifiers, decoding dependencies, and color levels corresponding to the spherical harmonic coefficient components contained in the corresponding target 3D Gaussian file track. In step S104, the method determines the target 3D Gaussian components from the target 3D Gaussian file track based on the file component information and determines the decoding relationship corresponding to the target 3D Gaussian components based on the decoding dependencies. In step S105, the method decodes the target 3D Gaussian components according to the decoding relationships. Wherein, when the decoding relationships indicate that the decoding of multiple target 3D Gaussian components is independent, the multiple target 3D Gaussian components are decoded in parallel. Based on the method of this application, the improved ISOBMFF file format can be used to encapsulate the 3D Gaussian bitstream and enables parallel decoding based on the decoding relationships determined by the decoding dependencies, thereby improving the decoding speed.

[0036] It should be further noted that, based on any of the above embodiments, the above-described parsing of three-dimensional Gaussian media signaling to obtain signaling component information corresponding to the multiple media resources described by the three-dimensional Gaussian media signaling may include: S1011, When the three-dimensional Gaussian media signaling is HTTP-based dynamic adaptive streaming media signaling, the three-dimensional Gaussian bitstream descriptor and three-dimensional Gaussian component descriptor of the media representation description file in the three-dimensional Gaussian media signaling are parsed to obtain the component identification information, decoding dependency relationship and color level corresponding to each media resource. In this embodiment, the HTTP-based dynamic adaptive streaming media signaling is DASH. In existing technologies, transmission signaling is used to describe the attributes of media resources so that clients can request the corresponding media resources. The decoder configuration information and 3DGSComponentInfoBox (3D Gaussian Component Information Data Box) proposed in this invention are used to describe the encoding and decoding related information of the 3D Gaussian bitstream and the 3D Gaussian component information in the 3D Gaussian file track / data item, respectively. When the 3D Gaussian file track / data item is sliced ​​and packaged into media resources, the decoder configuration information and the metadata information in 3DGSComponentInfoBox can also be used in the transmission signaling descriptor to describe the corresponding media resource. Specifically, according to the transmission signaling standard itself, if DASH is used to transmit 3D Gaussian file tracks / data items, the decoder configuration information corresponding to the track / data item should be included in the initialization fragment and sent to the client. The information in 3DGSComponentInfoBox should exist in the form of a Media Presentation Description (MPD) descriptor.

[0037] It should be noted that in component-based multitrack transmission mode, 3D Gaussian data is encapsulated in multiple tracks, with each track carrying a set of 3D Gaussian components. Each set of 3D Gaussian components should be represented as an independent adaptation set in DASH MPD, called a component adaptation set. The component adaptation set carrying geometric information should serve as the main adaptation set for the 3D Gaussian content. Except for the main adaptation set, each representation in other component adaptation sets should use the @dependencyId attribute specified in ISO / IEC 23009-1 to identify the corresponding representation in the main adaptation set.

[0038] The main adapter set should contain an initialization segment at the adapter set level, or an initialization segment for each representation at the representation level. Each initialization segment should contain decoder configuration information for initializing the decoder. For component adapter sets, appropriate descriptors should be used. One or more Gaussian splash component descriptors can be used to describe the component information of the corresponding component adapter set. Each component descriptor describes a specific type of component information.

[0039] The specific syntax and semantics are shown in Table 1, which is a schematic diagram of the syntax and semantics provided by an embodiment of the present invention.

[0040]

[0041] S1012, when the three-dimensional Gaussian media signaling is intelligent media transmission signaling, the three-dimensional Gaussian bitstream description information and three-dimensional Gaussian component description information of the media resource descriptor in the three-dimensional Gaussian media signaling are parsed to obtain the component identification information, decoding dependency relationship and color level corresponding to each media resource.

[0042] In this embodiment, if the intelligent media transmission signaling is SMT signaling, and SMT is used to transmit 3D Gaussian file tracks / data items, then the decoder configuration information corresponding to the track / data item and the information in the 3DGSComponentInfoBox should exist in the form of a media resource descriptor. SMT / DASH is an application-layer transmission protocol for audio and video media. It should be noted that this embodiment provides a method for identifying 3D Gaussian data components based on the SMT / DASH transmission protocol. When a 3D Gaussian file is transmitted via the SMT / DASH protocol, the transmission rules follow existing SMT / DASH technology, but the 3D Gaussian media file needs to be identified using 3DGS_bitstream_descriptor (3D Gaussian bitstream descriptor) and 3DGS_component_descriptor (3D Gaussian component descriptor). The fields in the identifier correspond to the fields in the file encapsulation mentioned above, as shown in Table 2 below.

[0043] Table 2. Schematic diagram of the first type of identifier field.

[0044] It should be further explained that, based on any of the above embodiments, sending a resource request to the server based on signaling component information may include: determining a target color level corresponding to the decoding or presentation requirements based on the color level in the signaling component information, and sending a resource request to the server for requesting the spherical harmonic coefficient data corresponding to the target color level, so that the server sends the target media resource carrying the spherical harmonic coefficient data; requesting the simultaneous transmission of multiple media resources to determine at least two target media resources. Specifically, this embodiment can request the server to transmit multiple media resources simultaneously to obtain at least two target media resources when network bandwidth and decoding capabilities are not limited. One media resource can correspond to one three-dimensional Gaussian file track, and different target media resources can carry different three-dimensional Gaussian components. After the terminal device obtains multiple media resources simultaneously, it can perform parallel decoding of multiple independent three-dimensional Gaussian components, thereby reducing serial waiting, or it can be unrestricted by this condition. This embodiment can adaptively select some file tracks for decoding according to the performance of the user device, which can reduce the decoding overhead of the device. That is, on the one hand, it improves the decoding speed and reduces decoding consumption (power), and on the other hand, it lowers the device performance threshold, enabling more devices to support it. The media resources in this embodiment may include media resources carrying spherical harmonic coefficient data and media resources that can be transmitted simultaneously, thereby improving the comprehensiveness of media resource requests.

[0045] It should be further explained that after unpacking the target media resource to obtain the target 3D Gaussian file track corresponding to the target media resource, and parsing the file component information in the target 3D Gaussian file track, the process may further include: obtaining the color level enable flag, the level identifier of each color level, and the number of spherical harmonic orders contained in each color level from the file component information; when the color level enable flag indicates that the spherical harmonic coefficients are divided into different color levels according to their orders, determining the target color level from each color level based on the presentation requirements; extracting the spherical harmonic coefficient components corresponding to the target color level from the target 3D Gaussian file track according to the level identifier and the corresponding number of spherical harmonic orders, and decoding the extracted spherical harmonic coefficient components. Specifically, this embodiment can be performed under the condition that the network bandwidth and decoding capability meet the processing requirements, or it can be performed without such limitations. In this embodiment, the network bandwidth and decoding capability meeting the processing requirements means that the currently available network bandwidth can support the simultaneous transmission of at least two target media resources, and the terminal device can perform parallel decoding of at least two 3D Gaussian components that are decoded independently. Limited network bandwidth in this embodiment refers to insufficient available network bandwidth of the terminal device to support the simultaneous transmission of multiple media resources, or insufficient bandwidth to support the transmission of spherical harmonic coefficients corresponding to higher color levels. When network bandwidth is limited, the terminal device can request only the spherical harmonic coefficient data corresponding to lower color levels based on color level information. Limited device rendering capability in this embodiment refers to insufficient processing capability of the terminal device for decoding and rendering 3D Gaussian data to support the decoding and rendering of spherical harmonic coefficient data corresponding to higher color levels. When device rendering capability is limited, the terminal device can decode and render only the spherical harmonic coefficient data corresponding to lower color levels based on color level information. It should be noted that the aforementioned limited network bandwidth and limited device rendering capability are relative to the currently transmitted or rendered 3D Gaussian data, and do not indicate a lack of data transmission or rendering capability in the network bandwidth or device rendering capability. In this embodiment, when device rendering capability is limited, only the spherical harmonic coefficient data corresponding to lower color levels can be decoded and rendered based on color level information. When decoding capability is not limited, different 3D Gaussian components that are independently encoded and decoded can be decoded in parallel to improve rendering speed.

[0046] It should be further explained that unpacking the target media resource to obtain the target 3D Gaussian file track corresponding to the target media resource, and parsing the file component information in the target 3D Gaussian file track, may include: obtaining the track component information recorded in the 3D Gaussian component information data box in the target 3D Gaussian file track; determining the component types and component identifiers actually contained in the target 3D Gaussian file track based on the track component information; when the target 3D Gaussian file track contains decoder configuration information, obtaining the decoder configuration information from the target 3D Gaussian file track; when the target 3D Gaussian file track does not contain decoder configuration information, obtaining the track identifier recorded in the track reference data box in the target 3D Gaussian file track, determining the referenced 3D Gaussian file track based on the track identifier, and obtaining the decoder configuration information from the referenced 3D Gaussian file track; based on the decoder configuration information, determining the decoding dependency and the color level corresponding to the spherical harmonic coefficient components, and determining the component types and component identifiers, decoding dependency, and color levels actually contained in the target 3D Gaussian file track as file component information. Compared to the existing ISOBMFF file format, the improved ISOBMFF file format in this embodiment extends the file track encapsulation method, data box expansion, subsample definition, and static 3DGS file format expansion.

[0047] Understandably, depending on the encoding and file encapsulation strategies, the 3D Gaussian data of a scene can be encoded into one or more independent 3D Gaussian bitstreams, and each independent 3D Gaussian bitstream can be encapsulated into one or more 3D Gaussian data tracks (i.e., 3D Gaussian file tracks). When the 3D Gaussian data of a scene is encoded into multiple independent 3D Gaussian bitstreams, the tracks corresponding to these 3D Gaussian bitstreams should be associated through track groups. Note: A 3DGSBitstreamTrackGroupBox can associate all tracks corresponding to N 3D Gaussian bitstreams, or it can associate only N tracks containing decoder configuration information corresponding to N 3D Gaussian bitstreams.

[0048] aligned(8) class 3DGSBitstreamTrackGroupBox extends TrackGroupTypeBox('gsbg') {

[0049] unsigned int(8) bitstream_id;

[0050] 3DGSBitstreamInfoBox();

[0051] }

[0052] `bitstream_id` indicates the identifier of a 3D Gaussian bitstream. Tracks with the same `bitstream_id` belong to the same 3D Gaussian bitstream. `3DGSBitstreamInfoBox` (3D Gaussian Bitstream Information Data Box) indicates information related to the 3D Gaussian data components in the 3D Gaussian bitstream. Note: This data box can be contained in either a `3DGSBitstreamTrackGroupBox` or a `3DGSDecoderConfigurationRecord` (decoder configuration information) corresponding to the bitstream; either option is acceptable. When a standalone 3D Gaussian bitstream is encapsulated into a single track, that track contains all the 3D Gaussian data components of the 3D Gaussian bitstream. When a standalone 3D Gaussian bitstream is encapsulated into multiple tracks, each track contains a portion of the 3D Gaussian data components of the 3D Gaussian bitstream, and the corresponding 3D Gaussian data component information is indicated by the `3DGSComponentInfoBox` data box. When a 3D Gaussian bitstream is encapsulated through a single track, the decoder configuration information corresponding to the 3D Gaussian bitstream should be included in that track. When a 3D Gaussian bitstream is encapsulated through multiple file tracks, the decoder configuration information corresponding to the 3D Gaussian bitstream should be included in one of the file tracks. Preferably, it can be included in the track encapsulating the 3D Gaussian geometric data (i.e., 3D Gaussian x, y, z coordinate information). When a single 3D Gaussian bitstream is encapsulated into multiple tracks, the tracks that do not contain decoder configuration information should use the TrackReferenceTypeBox specified in ISO / IEC 14496-12, with the reference index pointing to the track containing the decoder configuration information. Here, the reference_type in the TrackReferenceTypeBox is 'gsca' (3D Gaussian component track reference), and the track_IDs in the TrackReferenceTypeBox should correspond to the identifier of the track containing the decoder configuration information.

[0053] It should be noted that the decoder configuration information in this embodiment is used to indicate the decoding-related information of the three-dimensional Gaussian bitstream corresponding to the file track.

[0054] aligned(8) class 3DGDecoderConfigurationRecord {

[0055] unsigned int(8) configurationVersion = 1;

[0056] unsigned int(8) profile;

[0057] unsigned int(8) level; unsigned int(8) max_sh_degree; unsigned int(3) attr_data_dependency; unsigned int(1) color_lod_enable_flag; unsigned int(1) monochrome_flag unsigned int(1) multi_bitstream_flag; bit(2) reserved; if(color_lod_enable_flag == 1){ unsigned int(8) num_color_lod; for(int i=0; i<num_color_lod; i++){ unsigned int(8) color_lod_idx[i]; unsigned int(8) num_sh_degrees[i]; } } if(multi_bitstream_flag == 1){ 3DGSBitstreamInfoBox(); } }。

[0058] The `3DGSBitstreamInfoBox`, contained in the decoder configuration information, indicates specific information about the 3D Gaussian bitstream. This information is used to select different 3D Gaussian bitstreams. `configurationVersion` indicates the version information of the decoder configuration information; if the decoder is incompatible with a certain version, it should not decode the corresponding bitstream. `profile` specifies the profile of the bitstream corresponding to the decoder configuration information; each profile corresponds to a subset of the syntax, semantics, and algorithms specified for a bitstream. `level` specifies the level of the bitstream corresponding to the decoder configuration information; each level is a set of constraints on syntax elements and syntax element parameter values ​​within a given profile. `max_sh_degree` indicates the maximum order of the spherical harmonic coefficients contained in the 3D Gaussian bitstream (the order is derived from the original 3DGS data itself; the order is the same as the order included when the 3DGS was acquired and generated). `attr_data_dependency` indicates the encoding / decoding dependency information of the attribute data (data other than xyz coordinates). The three bits from the highest to the lowest represent: a value of 1 for the highest bit indicates that the decoding of the attribute data depends on the geometric data, and a value of 0 indicates that the decoding of the attribute data does not depend on the geometric data. When the second-highest bit is 1, it indicates that the decoding of the attribute data depends on other attribute data; when it is 0, it indicates that the decoding of the attribute data does not depend on other attribute data. When the least significant bit is 0, it indicates that the decoding of the attribute data depends only on attribute data within the same attribute group; when it is 1, it indicates that the decoding of the attribute data may depend on any other attribute data. When `color_lod_enable_flag` is 1, it indicates that the spherical harmonic coefficients are divided into different color levels according to their order; when it is 0, it indicates that the spherical harmonic coefficients are not divided into color levels. `num_color_lod` indicates the number of color levels. `color_lod_idx[i]` indicates the identifier of the i-th color level. Color levels are numbered starting from 0 and increasing sequentially; the higher the level, the richer the color information it contains.

[0059] `num_sh_degrees[i]` indicates the number of orders corresponding to the spherical harmonic coefficient components contained in the i-th color level. The starting spherical harmonic order corresponding to the i-th color level is `colorLoDSHStart[i]`, calculated as follows: when `color_lod_idx[i] = 0`, `colorLoDSHStart[i] = 0`; when `color_lod_idx[i] > 0`, `colorLoDSHStart[i] = colorLoDSHStart[i-1] + 1`; all spherical harmonic orders corresponding to the spherical harmonic coefficient components contained in the i-th color level are: `colorLoDSHStart[i] ~ colorLoDSHStart[i] + num_sh_degrees[i] - 1`. `monochrome_flag` with a value of 1 indicates that the 3D Gaussian data contains only monochrome data; with a value of 0, it indicates that the 3D Gaussian data contains three color channels. `multi_bitstream_flag` with a value of 1 indicates that the original 3D Gaussian data is encoded into multiple bitstreams; with a value of 0, it indicates that the original 3D Gaussian data is encoded into a single bitstream.

[0060] aligned(8) class 3DGSBitstreamInfoBox extends FullBox('gbif', version= 0, 0) {

[0061] unsigned int(4) num_component_types;

[0062] bit(4) reserved;

[0063] for(int i=0; i <num_component_types; i++){

[0064] unsigned int(4) component_type[i];

[0065] unsigned bit(1) component_idx_flag[i];

[0066] bit(3) reserved;

[0067] if(component_idx_flag[i] == 1){

[0068] unsigned int(8) num_components[i];

[0069] for(int j=0; j< num_components[i]; j++){

[0070] unsigned int(8) component_idx[i][j];

[0071] }

[0072] }

[0073] }

[0074] }

[0075] `num_component_types` indicates the number of 3D Gaussian data types contained in the bitstream corresponding to the current decoder configuration information. `component_type[i]` indicates the type of the i-th 3D Gaussian data contained in the bitstream corresponding to the current decoder configuration information. When `component_idx_flag[i]` is 0, it means that the bitstream corresponding to the current decoder configuration information contains all 3D Gaussian data components of type `component_type[i]`. When it is 1, it means that the bitstream corresponding to the current decoder configuration information contains some 3D Gaussian data components of type `component_type[i]`. `num_components[i]` indicates the number of 3D Gaussian data components of type `component_type[i]` contained in the bitstream corresponding to the current decoder configuration information. `component_idx[i][j]` indicates the index of the j-th 3D Gaussian data component of type `component_type[i]` contained in the bitstream corresponding to the current decoder configuration information. The correspondence between the index value and the three-dimensional Gaussian data components is shown in Table 3. Table 3. Schematic diagram of a correspondence relationship

[0076] Note: The table above is a classification method. Geometric data can also be classified into only one type without distinguishing between the highest and lowest bits. SH coefficients can also be extended to 4th order, 5th order, etc.

[0077] It should be noted that the decoder configuration information indicates the characteristics of the bitstream. The 3D Gaussian component information data box indicates the characteristics of the data in a track. A bitstream can be encapsulated into multiple tracks.

[0078] aligned(8) class 3DGSComponentInfoBox extends FullBox('gcif', version= 0, 0) {

[0079] unsigned int(4) num_component_types;

[0080] unsigned int(2) component_group_idc;

[0081] bit(2) reserved;

[0082] if(component_group_idc == 1){

[0083] unsigned int(8) component_group_idx;

[0084] }

[0085] for(int i=0; i<num_component_types; i++){

[0086] unsigned int(4) component_type[i];

[0087] unsigned bit(1) component_idx_flag[i];

[0088] bit(3) reserved;

[0089] if(component_group_idc == 2){

[0090] unsigned int(8) component_group_idx[i];

[0091] }

[0092] if(component_idx_flag[i] == 1){

[0093] unsigned int(8) num_components[i];

[0094] for(int j=0; j< num_components[i]; j++){

[0095] unsigned int(8) component_idx[i][j];

[0096] if(component_group_idc == 3){

[0097] unsigned int(8) component_group_idx[i][j];

[0098] }

[0099] }

[0100] }

[0101] }

[0102] }

[0103] `num_component_types` indicates the number of component types contained in the 3D Gaussian data corresponding to the current track. `component_group_idc` indicates the component group type to which the 3D Gaussian data corresponding to the current track belongs. When this field is 0, it means that the 3D Gaussian data corresponding to the current track does not belong to any 3D Gaussian component group; when this field is 1, it means that the 3D Gaussian data corresponding to the current track belongs to the same 3D Gaussian component group; when this field is 2, it means that each type of 3D Gaussian component in the 3D Gaussian data corresponding to the current track belongs to a 3D Gaussian component group; when this field is 3, it means that each 3D Gaussian component of each type in the 3D Gaussian data corresponding to the current track belongs to a 3D Gaussian component group. `component_group_idx` indicates the identifier of the 3D Gaussian component group. There are encoding / decoding dependencies between 3D Gaussian components in the same 3D Gaussian component group. `component_type[i]` indicates the type of the i-th 3D Gaussian data contained in the 3D Gaussian data corresponding to the current track. When `component_idx_flag[i]` is 0, it indicates that the current track's 3D Gaussian data contains all 3D Gaussian data components of type `component_type[i]`. When it is 1, it indicates that the current track's 3D Gaussian data contains some 3D Gaussian data components of type `component_type[i]`. `num_components[i]` indicates the number of 3D Gaussian data components of type `component_type[i]` contained in the current track's 3D Gaussian data. `component_idx[i][j]` indicates the index of the j-th 3D Gaussian data component of type `component_type[i]` contained in the current track's 3D Gaussian data. Refer to the table above for the correspondence between index values ​​and 3D Gaussian data components.

[0104] This embodiment demonstrates how to parse the target 3D Gaussian file trajectory after encapsulating the 3D Gaussian components in a 3D Gaussian encoded bitstream based on the improved ISOBMFF file format, thereby improving the accuracy of determining component information.

[0105] It should be further noted that, based on any of the above embodiments, the decoding relationship for determining the target three-dimensional Gaussian components based on decoding dependencies may include: S1041, Obtain the decoding dependency from the file component information, and based on the decoding dependency, determine whether the decoding of attribute data depends on geometric data, and whether the decoding of attribute data depends on other attribute data; S1042, when the decoding dependency indicates that the attribute data depends on the geometric data, the target three-dimensional Gaussian component corresponding to the geometric data is determined as the dependent component of the target three-dimensional Gaussian component corresponding to the attribute data. S1043, when the decoding dependency indicates that the decoding of attribute data depends on other attribute data, the target three-dimensional Gaussian component corresponding to the other attribute data is determined as the dependent component of the target three-dimensional Gaussian component corresponding to the attribute data. S1044, when the decoding dependency indicates that the decoding of the attribute data does not depend on the geometric data and other attribute data, the decoding of the target three-dimensional Gaussian component corresponding to the attribute data is determined to be independent of the decoding of the target three-dimensional Gaussian component corresponding to the geometric data and other attribute data.

[0106] This embodiment uses `attr_data_dependency` to indicate attribute data (data other than xyz coordinates) to determine encoding / decoding dependency information (i.e., the decoding dependency relationship of this application). The three bits from high to low represent the following: The highest bit is 1 when the attribute data decoding depends on geometric data, and 0 when the attribute data decoding does not depend on geometric data. The second highest bit is 1 when the attribute data decoding depends on other attribute data, and 0 when the attribute data decoding does not depend on other attribute data. The lowest bit is 0 when the attribute data decoding depends only on attribute data within the same attribute group, and 1 when the attribute data decoding may depend on any other attribute data. This embodiment provides a method for determining dependencies, improving the accuracy of dependency determination.

[0107] It should be further explained that, based on any of the above embodiments, the determination of the target 3D Gaussian component from the target 3D Gaussian file track based on file component information may include: when the samples in the target 3D Gaussian file track are divided into multiple sub-samples, parsing the sub-sample information data boxes in the target 3D Gaussian file track to obtain the sub-sample structure indication information corresponding to each sub-sample, and the component type or component identifier corresponding to the sub-sample structure indication information; determining the target component type or target component identifier corresponding to the 3D Gaussian component to be decoded based on the signaling component information and file component information; when the sub-sample structure indication information is 0 and the component type of the corresponding sub-sample is the same as the target component type, the corresponding sub-sample is determined as the target sub-sample; when the sub-sample structure indication information is 2 and the component identifier of the corresponding sub-sample is the same as the target component identifier, the corresponding sub-sample is determined as the target sub-sample; and the 3D Gaussian component contained in the target sub-sample is determined as the target 3D Gaussian component. This embodiment explains the sub-sample definition part, that is, the sample in the 3D Gaussian file track, which corresponds to a frame of encoded 3D Gaussian data in the 3D Gaussian bitstream. A sample of a 3D Gaussian file track can be further divided into one or more subsamples. The definition and use of subsamples follow the standard ISO / IEC 14496-12:2022. The values ​​and meanings of the flags field in the SubSampleInformationBox of the 3D Gaussian file track are as follows: a) When the flags field is 0, it means that each subsample corresponds to one or more three-dimensional Gaussian data components; b) All other values ​​are retained.

[0108] The values ​​and meanings of the `codec_specific_parameters` field in the SubSampleInformationBox are as follows: if (flags == 0) { unsigned int(2) sub_sample_idc; if(sub_sample_idc == 0){ unsigned int(4) component_type; bit(4) reserved; } else if(sub_sample_idc == 1){ unsigned int(8) component_group_idx; } else if(sub_sample_idc == 2){ unsigned int(8) component_idx; } bit(22) reserved = 0; }

[0109] `sub_sample_idc` indicates the composition structure of the 3D Gaussian data components in each subsample. A value of 0 indicates that each subsample contains 3D Gaussian data components of type `component_type`, and includes all data components of that type. A value of 1 indicates that each subsample contains all 3D Gaussian data components corresponding to the 3D Gaussian component group with the identifier `component_group_idx`. A value of 2 indicates that each subsample contains one 3D Gaussian data component with the identifier `component_group_idx`. `component_type` indicates the type of the 3D Gaussian data components contained in the subsample. `component_group_idx` indicates the identifier of the 3D Gaussian component group to which the 3D Gaussian data components contained in the subsample belong. `component_group_idx` indicates the identifier of the 3D Gaussian data components contained in the subsample. `reserved` is a reserved field.

[0110] This embodiment explains the static 3DGS file format extension. In the existing ISOBMFF technology, dynamic media (such as video) is encapsulated using tracks, while static media (such as images) is encapsulated using items. The file encapsulation method and item extension used in this invention can also be used for static media encapsulation. Depending on different encoding and file encapsulation strategies, the 3D Gaussian data of a scene can be encoded into one or more independent 3D Gaussian bitstreams, and an independent 3D Gaussian bitstream can be encapsulated into one or more 3D Gaussian data items. When the 3D Gaussian data of a scene is encoded into multiple independent 3D Gaussian bitstreams, the data items corresponding to these 3D Gaussian bitstreams should be associated through entity groups.

[0111] Note: 3DGSBitstreamEntityGroupBox can associate all data items corresponding to N 3D Gaussian bitstreams, or it can associate only N data items containing decoder configuration information corresponding to N 3D Gaussian bitstreams.

[0112] aligned(8) class 3DGSBitstreamEntityGroupBox extendsEntityGroupTypeBox('gsbe') {

[0113] unsigned int(8) bitstream_id;

[0114] 3DGSBitstreamInfoBox();

[0115] }

[0116] bitstream_id indicates the identifier of a three-dimensional Gaussian bitstream.

[0117] The 3DGSBitstreamInfoBox indicates information related to the three-dimensional Gaussian data components in the three-dimensional Gaussian bitstream.

[0118] Note: This data box can be contained within a 3DGSBitstreamEntityGroupBox or within the corresponding 3DGSDecoderConfigurationRecord decoder configuration information; either method is acceptable. When a standalone 3D Gaussian bitstream is encapsulated into a single data item, that data item contains all the 3D Gaussian data components of the bitstream. When a standalone 3D Gaussian bitstream is encapsulated into multiple data items, each data item contains a portion of the 3D Gaussian data components of the bitstream, with the corresponding 3D Gaussian data component information indicated by a 3DGSComponentInfoBox data box. The 3DGSComponentInfoBox (3D Gaussian component information data box) should exist as an ItemProperty attribute in the file. When a 3D Gaussian bitstream is encapsulated through a single data item, the decoder configuration information corresponding to the 3D Gaussian bitstream should be included in the data item's data item attribute. When a 3D Gaussian bitstream is encapsulated through multiple data items, the decoder configuration information corresponding to the 3D Gaussian bitstream should be included in one of the data items. Preferably, it can be included in the attributes of the data item encapsulating the 3D Gaussian geometric data (i.e., 3D Gaussian x, y, z coordinate information). When a single 3D Gaussian bitstream is encapsulated into multiple data items, the data items that do not contain decoder configuration information should use the ItemReferenceBox track reference data box specified in ISO / IEC 14496-12, with the reference index pointing to the data item containing decoder configuration information. Specifically, the reference_type in the ItemReferenceBox is 'gica', the from_item_ID in the ItemReferenceBox should correspond to the identifier of the data item that does not contain decoder configuration information, and the to_item_ID in the ItemReferenceBox should correspond to the identifier of the data item that contains decoder configuration information.

[0119] The technical solutions provided in this invention support multi-track encapsulation of 3D Gaussian coded bitstreams according to encoding / decoding independence; partial access within 3D Gaussian data frames according to encoding / decoding independence; hierarchical and extended spherical harmonic coefficients of 3D Gaussian data; and partial or multi-path transmission of 3D Gaussian data based on presentation requirements / transmission bandwidth, etc. Through the encapsulation technology of this invention, partial file tracks can be adaptively selected for transmission based on network bandwidth, reducing transmission bandwidth; and partial file tracks can be adaptively selected for decoding based on user device performance, reducing device decoding overhead.

[0120] To make the present invention easier to understand, the data decoding method provided in the embodiments of the present invention may include: (1) The client parses the three-dimensional Gaussian media signaling. The media signaling describes the signaling component information corresponding to the media resources. The signaling component information includes: the type and identifier of the component; the dependency information of the component data on other component data during decoding; and the color level information of the spherical harmonic coefficients (specific component types).

[0121] (2) Based on the signaling component information and the client's own decoding or rendering requirements, request the corresponding media resources. For example, when network bandwidth or device rendering capabilities are limited, based on color level information, only the spherical harmonic coefficient data corresponding to the lower color level can be requested. When network bandwidth and device decoding capabilities are not limited, multiple media resources can be requested to be transmitted simultaneously (one media resource corresponds to one file track) to support parallel decoding by the client and improve rendering speed.

[0122] (3) The server sends the media resources requested by the client to the client.

[0123] (4) After receiving the requested media resources, the client unpacks them and restores them to a 3D Gaussian file track. The client parses the component information in the 3D Gaussian file track level and sample level, including: the type and identifier of the components; the dependency information of the component data on other component data during decoding; and the color level information of the spherical harmonic coefficients (specific component types).

[0124] In this embodiment, the media resources are media resources obtained based on the improved ISOBMFF file format, DASH signaling, and SMT signaling encoding.

[0125] (5) Based on the parsed file component information, the client can decode the corresponding file tracks or sub-samples according to its own decoding or rendering requirements. For example, when the device's rendering capability is limited, based on the color level information, only the spherical harmonic coefficient data corresponding to the low color level can be decoded and rendered. When the device's decoding capability is not limited, different three-dimensional Gaussian components that are independently encoded and decoded can be decoded in parallel to improve the rendering speed.

[0126] The data decoding device provided in the embodiments of the present invention will be described below. The data decoding device described below and the data decoding method described above can be referred to in correspondence.

[0127] Please refer to the details. Figure 2 , Figure 2 A schematic diagram of a data decoding device provided in an embodiment of the present invention may include: The parsing module 100 is used to parse the three-dimensional Gaussian media signaling to obtain the signaling component information corresponding to the multiple media resources described by the three-dimensional Gaussian media signaling; the signaling component information includes component identification information, decoding dependency relationship and color level corresponding to the spherical harmonic coefficient component, and the component identification information includes component type and component identifier. The request module 200 is used to send a resource request to the server based on the signaling component information, and to receive at least one target media resource sent by the server based on the resource request; wherein the target media resource corresponds to a three-dimensional Gaussian file track, and the three-dimensional Gaussian file track is obtained by encapsulating the three-dimensional Gaussian components in the three-dimensional Gaussian encoded bitstream based on the improved ISOBMFF file format; The unpacking module 300 is used to unpack the target media resource to obtain the target three-dimensional Gaussian file track corresponding to the target media resource, and parse the file component information in the target three-dimensional Gaussian file track; wherein, the file component information is used to describe the component type, component identifier, decoding dependency and color level corresponding to the spherical harmonic coefficient component actually contained in the corresponding target three-dimensional Gaussian file track; The decoding relationship determination module 400 is used to determine the target three-dimensional Gaussian component from the target three-dimensional Gaussian file track based on the file component information, and to determine the decoding relationship corresponding to the target three-dimensional Gaussian component based on the decoding dependency relationship; The decoding module 500 is used to decode the target three-dimensional Gaussian components according to the decoding relationship; wherein, when the decoding relationship indicates that the decoding of multiple target three-dimensional Gaussian components is independent of each other, the multiple target three-dimensional Gaussian components are decoded in parallel.

[0128] Furthermore, based on any of the above embodiments, the parsing module 100 may include: The first parsing unit is used to parse the three-dimensional Gaussian bitstream descriptor and three-dimensional Gaussian component descriptor of the media representation description file in the three-dimensional Gaussian media signaling when the three-dimensional Gaussian media signaling is HTTP-based dynamic adaptive streaming media signaling, so as to obtain the component identification information, decoding dependency relationship and color level corresponding to each media resource. The second parsing unit is used to parse the three-dimensional Gaussian bitstream description information and three-dimensional Gaussian component description information of the media resource descriptor in the three-dimensional Gaussian media signaling when the three-dimensional Gaussian media signaling is intelligent media transmission signaling, so as to obtain the component identification information, decoding dependency relationship and color level corresponding to each media resource.

[0129] Furthermore, based on any of the above embodiments, the request module 200 may include: The first request unit is configured to determine the target color level corresponding to the decoding or presentation requirements based on the color level in the signaling component information, and send a resource request to the server for requesting the spherical harmonic coefficient data corresponding to the target color level, so that the server sends the target media resource carrying the spherical harmonic coefficient data; The second request unit is used to request the simultaneous transmission of multiple media resources to identify at least two of the target media resources.

[0130] Furthermore, based on any of the above embodiments, the data decoding apparatus may further include: The spherical harmonic order number determination module is used to obtain the color level enable flag, the level identifier of each color level, and the number of spherical harmonic orders contained in each color level from the file component information. The target color level determination module is used to determine the target color level from each of the color levels based on the presentation requirements when the color level enable flag indicates that the spherical harmonic coefficients are divided into different color levels according to their order. The harmonic coefficient component determination module is used to extract the spherical harmonic coefficient components corresponding to the target color level from the target three-dimensional Gaussian file track according to the level identifier of the target color level and the corresponding number of spherical harmonic orders, and to decode the extracted spherical harmonic coefficient components.

[0131] Furthermore, based on any of the above embodiments, the unpacking module 300 may include: The orbit component information determination unit is used to obtain the orbit component information recorded in the three-dimensional Gaussian component information data box in the target three-dimensional Gaussian file orbit; The component type and component identifier determination unit is used to determine the component type and component identifier actually contained in the target three-dimensional Gaussian file orbit based on the orbit component information. The first decoder configuration information acquisition unit is used to acquire the decoder configuration information from the target three-dimensional Gaussian file track when the target three-dimensional Gaussian file track contains decoder configuration information; The second decoder configuration information unit is used to obtain the track identifier recorded in the track reference data box in the target three-dimensional Gaussian file track when the target three-dimensional Gaussian file track does not contain the decoder configuration information, determine the referenced three-dimensional Gaussian file track according to the track identifier, and obtain the decoder configuration information from the referenced three-dimensional Gaussian file track. The file component information determination unit is used to determine the decoding dependency and the color level corresponding to the spherical harmonic coefficient component based on the decoder configuration information, and to determine the component type and component identifier actually contained in the target three-dimensional Gaussian file track, the decoding dependency, and the color level as the file component information.

[0132] Furthermore, based on any of the above embodiments, the decoding relationship determination module 400 may include: The first dependency determination unit is used to obtain the decoding dependency relationship from the file component information, and based on the decoding dependency relationship, determine whether the decoding of attribute data depends on geometric data, and whether the decoding of attribute data depends on other attribute data; The second dependency determination unit is used to determine the target three-dimensional Gaussian component corresponding to the geometric data as the dependent component of the target three-dimensional Gaussian component corresponding to the attribute data when the decoding dependency relationship indicates that the decoding of the attribute data depends on the geometric data. The third dependency determination unit is used to determine the target three-dimensional Gaussian component corresponding to the other attribute data as the dependent component of the target three-dimensional Gaussian component corresponding to the attribute data when the decoding dependency relationship indicates that the decoding of the attribute data depends on other attribute data. The fourth dependency determination unit is used to determine that the decoding of the target three-dimensional Gaussian component corresponding to the attribute data is independent of the decoding of the geometric data and the target three-dimensional Gaussian component corresponding to the other attribute data when the decoding dependency relationship indicates that the decoding of the attribute data does not depend on the geometric data and does not depend on the other attribute data.

[0133] Furthermore, based on any of the above embodiments, the decoding relationship determination module 400 may include: The subsample determination unit is used to parse the subsample information data box in the target three-dimensional Gaussian file track when the sample in the target three-dimensional Gaussian file track is divided into multiple subsamples, to obtain the subsample structure indication information corresponding to each subsample, and the component type or component identifier corresponding to the subsample structure indication information. The target component type or target component identifier determination unit is used to determine the target component type or target component identifier corresponding to the three-dimensional Gaussian component to be decoded based on the signaling component information and the file component information. The first target subsample determination unit is used to determine the corresponding subsample as the target subsample when the subsample structure indication information is 0 and the component type of the corresponding subsample is the same as the target component type. The second target subsample determination unit is used to determine the corresponding subsample as the target subsample when the subsample structure indication information is 2 and the component identifier of the corresponding subsample is the same as the target component identifier. The target three-dimensional Gaussian component determination unit is used to determine the three-dimensional Gaussian components contained in the target sub-sample as the target three-dimensional Gaussian components.

[0134] It should be noted that the order of the modules and units in the above-mentioned data decoding device can be changed without affecting the logic.

[0135] An embodiment of the present invention provides a data decoding device, which may include: a parsing module 100, used to parse three-dimensional Gaussian media signaling to obtain signaling component information corresponding to multiple media resources described by the three-dimensional Gaussian media signaling; the signaling component information includes component identification information, decoding dependency relationship and color level corresponding to spherical harmonic coefficient components, and the component identification information includes component type and component identifier; a request module 200, used to send a resource request to a server based on the signaling component information, and receive at least one target media resource sent by the server based on the resource request; wherein, the target media resource corresponds to a three-dimensional Gaussian file track, and the three-dimensional Gaussian file track is obtained by encapsulating the three-dimensional Gaussian components in the three-dimensional Gaussian encoded bitstream based on the improved ISOBMFF file format; and an unpacking module 300, used to unpack the target... The media resource is unpacked to obtain the target 3D Gaussian file track corresponding to the target media resource, and the file component information in the target 3D Gaussian file track is parsed. The file component information describes the component types, component identifiers, decoding dependencies, and color levels corresponding to the spherical harmonic coefficients in the corresponding target 3D Gaussian file track. A decoding relationship determination module 400 is used to determine the target 3D Gaussian components from the target 3D Gaussian file track based on the file component information, and to determine the decoding relationship corresponding to the target 3D Gaussian components based on the decoding dependencies. A decoding module 500 is used to decode the target 3D Gaussian components according to the decoding relationships. When the decoding relationships indicate that the decoding of multiple target 3D Gaussian components is independent, the multiple target 3D Gaussian components are decoded in parallel. This invention can encapsulate 3D Gaussian bitstreams through an improved ISOBMFF file format and can perform parallel decoding based on decoding relationships determined by decoding dependencies, thereby improving decoding speed.

[0136] The following describes a data decoding device provided by an embodiment of the present invention. The data decoding device described below and the data decoding method described above can be referred to in correspondence.

[0137] Please refer to Figure 3 , Figure 3 A schematic diagram of a data decoding device provided in an embodiment of the present invention may include: Memory 10 is used to store computer programs; Processor 20 is used to execute computer programs to implement the data decoding method described above.

[0138] The memory 10, processor 20, and communication interface 30 all communicate with each other through the communication bus 40.

[0139] In this embodiment of the invention, the memory 10 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment of the invention, the memory 10 may store programs for implementing the above-mentioned functions.

[0140] In one possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0141] Furthermore, memory 10 may include read-only memory and random access memory, providing instructions and data to the processor. A portion of the memory may also include NVRAM. The memory stores operating systems and operating instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and handling hardware-based tasks.

[0142] Processor 20 can be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic device. Processor 20 can be a microprocessor or any conventional processor. Processor 20 can call programs stored in memory 10.

[0143] The communication interface 30 can be an interface for the communication module, used to connect with other devices or systems.

[0144] Of course, it should be noted that, Figure 3 The structure shown does not constitute a limitation on the data decoding device in the embodiments of the present invention. In practical applications, the data decoding device may include more than Figure 3 More or fewer components as shown, or combinations of certain components.

[0145] The following describes the computer-readable storage medium provided in the embodiments of the present invention. The computer-readable storage medium described below and the data decoding method described above can be referred to in correspondence.

[0146] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described data decoding method.

[0147] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0148] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0149] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0150] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0151] The above provides a detailed description of a data decoding method, apparatus, device, and computer-readable storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data decoding method, characterized in that, include: The three-dimensional Gaussian media signaling is parsed to obtain the signaling component information corresponding to the multiple media resources described by the three-dimensional Gaussian media signaling. The signaling component information includes component identification information, decoding dependency relationship and color level corresponding to spherical harmonic coefficient components. The component identification information includes component type and component identifier. Based on the signaling component information, a resource request is sent to the server, and at least one target media resource is received from the server based on the resource request; wherein, the target media resource corresponds to a three-dimensional Gaussian file track, and the three-dimensional Gaussian file track is obtained by encapsulating the three-dimensional Gaussian components in the three-dimensional Gaussian encoded bitstream based on the improved ISOBMFF file format; The target media resource is unpacked to obtain the target 3D Gaussian file track corresponding to the target media resource, and the file component information in the target 3D Gaussian file track is parsed; wherein, the file component information is used to describe the component type, component identifier, decoding dependency and color level corresponding to the spherical harmonic coefficient component actually contained in the corresponding target 3D Gaussian file track; Based on the file component information, the target three-dimensional Gaussian component is determined from the target three-dimensional Gaussian file track, and the decoding relationship corresponding to the target three-dimensional Gaussian component is determined based on the decoding dependency relationship; The target three-dimensional Gaussian components are decoded according to the decoding relationship; wherein, when the decoding relationship indicates that the decoding of multiple target three-dimensional Gaussian components is independent of each other, the multiple target three-dimensional Gaussian components are decoded in parallel.

2. The data decoding method according to claim 1, characterized in that, Parsing the three-dimensional Gaussian media signaling yields signaling component information corresponding to the multiple media resources described by the three-dimensional Gaussian media signaling, including: When the three-dimensional Gaussian media signaling is HTTP-based dynamic adaptive streaming media signaling, the three-dimensional Gaussian bitstream descriptor and three-dimensional Gaussian component descriptor of the media representation description file in the three-dimensional Gaussian media signaling are parsed to obtain the component identification information, decoding dependency relationship and color level corresponding to each media resource; When the three-dimensional Gaussian media signaling is intelligent media transmission signaling, the three-dimensional Gaussian bitstream description information and three-dimensional Gaussian component description information of the media resource descriptor in the three-dimensional Gaussian media signaling are parsed to obtain the component identification information, decoding dependency relationship and color level corresponding to each media resource.

3. The data decoding method according to claim 1, characterized in that, Sending a resource request to the server based on the signaling component information includes: Based on the color level in the signaling component information, a target color level corresponding to the decoding or presentation requirements is determined, and a resource request for requesting the spherical harmonic coefficient data corresponding to the target color level is sent to the server, so that the server sends the target media resource carrying the spherical harmonic coefficient data; The request is to transmit multiple media resources simultaneously to identify at least two of the target media resources.

4. The data decoding method according to claim 3, characterized in that, After unpacking the target media resource to obtain the target 3D Gaussian file track corresponding to the target media resource, and parsing the file component information in the target 3D Gaussian file track, the process further includes: Obtain the color level enable flag, the level identifier of each color level, and the number of spherical harmonics contained in each color level from the file component information. When the color level enable flag indicates that the spherical harmonic coefficients are divided into different color levels according to their order, the target color level is determined from each of the color levels based on the presentation requirements. Based on the level identifier of the target color level and the corresponding number of spherical harmonic orders, the spherical harmonic coefficient components corresponding to the target color level are extracted from the target 3D Gaussian file track, and the extracted spherical harmonic coefficient components are decoded.

5. The data decoding method according to any one of claims 1 to 4, characterized in that, The target media resource is unpacked to obtain the target 3D Gaussian file track corresponding to the target media resource, and the file component information in the target 3D Gaussian file track is parsed, including: Obtain the orbit component information recorded in the three-dimensional Gaussian component information data box in the target three-dimensional Gaussian file orbit; Based on the orbital component information, determine the actual component types and component identifiers contained in the target 3D Gaussian file orbit; When the target 3D Gaussian file track contains decoder configuration information, the decoder configuration information is obtained from the target 3D Gaussian file track; When the target 3D Gaussian file track does not contain the decoder configuration information, the track identifier recorded in the track reference data box in the target 3D Gaussian file track is obtained, the referenced 3D Gaussian file track is determined according to the track identifier, and the decoder configuration information is obtained from the referenced 3D Gaussian file track. Based on the decoder configuration information, the decoding dependency and the color level corresponding to the spherical harmonic coefficient components are determined, and the component types and component identifiers actually contained in the target three-dimensional Gaussian file track, the decoding dependency, and the color level are determined as the file component information.

6. The data decoding method according to claim 4, characterized in that, Determining the decoding relationship corresponding to the target 3D Gaussian component based on the decoding dependency includes: The decoding dependency is obtained from the file component information, and based on the decoding dependency, it is determined whether the decoding of the attribute data depends on the geometric data, and whether the decoding of the attribute data depends on other attribute data. When the decoding dependency indicates that the decoding of the attribute data depends on the geometric data, the target three-dimensional Gaussian component corresponding to the geometric data is determined as the dependent component of the target three-dimensional Gaussian component corresponding to the attribute data; When the decoding dependency indicates that the decoding of the attribute data depends on other attribute data, the target three-dimensional Gaussian component corresponding to the other attribute data is determined as the dependent component of the target three-dimensional Gaussian component corresponding to the attribute data. When the decoding dependency indicates that the decoding of the attribute data does not depend on the geometric data and the other attribute data, it is determined that the decoding of the target 3D Gaussian component corresponding to the attribute data is independent of the decoding of the geometric data and the target 3D Gaussian component corresponding to the other attribute data.

7. The data decoding method according to claim 1, characterized in that, Determining the target 3D Gaussian components from the target 3D Gaussian file orbit based on the file component information includes: When the sample in the target 3D Gaussian file orbit is divided into multiple sub-samples, the sub-sample information data box in the target 3D Gaussian file orbit is parsed to obtain the sub-sample structure indication information corresponding to each sub-sample, as well as the component type or component identifier corresponding to the sub-sample structure indication information. Based on the signaling component information and the file component information, determine the target component type or target component identifier corresponding to the three-dimensional Gaussian component to be decoded; When the subsample structure indication information is 0 and the component type of the corresponding subsample is the same as the target component type, the corresponding subsample is determined as the target subsample. When the subsample structure indication information is 2, and the component identifier of the corresponding subsample is the same as the target component identifier, the corresponding subsample is determined as the target subsample; The three-dimensional Gaussian components contained in the target subsample are determined as the target three-dimensional Gaussian components.

8. A data decoding method, characterized in that, include: The parsing module is used to parse the three-dimensional Gaussian media signaling to obtain the signaling component information corresponding to the multiple media resources described by the three-dimensional Gaussian media signaling. The signaling component information includes component identification information, decoding dependency relationship and color level corresponding to spherical harmonic coefficient components. The component identification information includes component type and component identifier. The request module is used to send a resource request to the server based on the signaling component information, and to receive at least one target media resource sent by the server based on the resource request; wherein the target media resource corresponds to a three-dimensional Gaussian file track, and the three-dimensional Gaussian file track is obtained by encapsulating the three-dimensional Gaussian components in the three-dimensional Gaussian encoded bitstream based on the improved ISOBMFF file format; The unpacking module is used to unpack the target media resource to obtain the target 3D Gaussian file track corresponding to the target media resource, and parse the file component information in the target 3D Gaussian file track; wherein, the file component information is used to describe the component type, component identifier, decoding dependency and color level corresponding to the spherical harmonic coefficient component actually contained in the corresponding target 3D Gaussian file track; The decoding relationship determination module is used to determine the target three-dimensional Gaussian component from the target three-dimensional Gaussian file track based on the file component information, and to determine the decoding relationship corresponding to the target three-dimensional Gaussian component based on the decoding dependency relationship; A decoding module is used to decode the target three-dimensional Gaussian components according to the decoding relationship; wherein, when the decoding relationship indicates that the decoding of multiple target three-dimensional Gaussian components is independent of each other, the multiple target three-dimensional Gaussian components are decoded in parallel.

9. A data decoding device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the data decoding method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the data decoding method as described in any one of claims 1 to 7.

Images