Data processing method, device, computer and readable storage medium

Abstract

The present application discloses a data processing method, device, computer, and readable storage medium. The method includes: obtaining k component tracks and atlas slices corresponding to each component track; the component track includes a component track data box, and the component track data box includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer; and the k component tracks and the atlas slices corresponding to each component track are encapsulated according to the component track data box to obtain a target file. Using this application, partial decapsulation and partial decoding can be achieved, thereby improving the accuracy of data encapsulation and decapsulation.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device, computer, and readable storage medium. Background Art

[0002] Immersive media refers to media content that provides users with an immersive experience. A point cloud is a set of irregularly distributed discrete points in space that represent the spatial structure and surface properties of a three-dimensional object or scene. Generally, point clouds can be acquired through computer generation, 3D laser scanning, or 3D photogrammetry. With the evolution of point cloud acquisition methods, the acquisition of large amounts of point cloud data has become possible. With the continuous accumulation of large-scale point cloud data, the efficient storage, transmission, publication, sharing, and standardization of point cloud data have become key to point cloud applications. After encoding the point cloud data, the encoded data stream needs to be packaged and transmitted to the user. Currently, this is typically done directly by packaging the encoded data stream and transmitting the packaged data to the user. This results in data loss during the encoding and packaging process. After receiving the packaged data, users are unable to obtain more accurate data from it, resulting in poor data processing efficiency and effectiveness. Summary of the Invention

[0003] The embodiments of the present application provide a data processing method, device, computer, and readable storage medium, which can implement partial decapsulation and partial decoding, thereby improving the accuracy of data encapsulation and decapsulation.

[0004] On the one hand, an embodiment of the present application provides a data processing method, including:

[0005] Get k component tracks and the atlas slices corresponding to each component track; the component track includes a component track data box, which includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer;

[0006] According to the component track data box, the k component tracks and the atlas slices corresponding to each component track are encapsulated to obtain the target file.

[0007] The component track data box includes a track group data box; the track group data box includes an atlas identifier and an atlas slice identifier; the atlas identifier indicates the identifier of the atlas of the volumetric video to which the atlas slice belongs;

[0008] When the atlas slice identifier is associated with the atlas identifier, the atlas slice corresponding to the atlas slice identifier belongs to the atlas of the volumetric video corresponding to the atlas identifier.

[0009] The component track data box includes a track group data box; the track group data box includes an atlas identifier and an atlas slice identifier; the atlas identifier indicates the identifier of the atlas to which the atlas slice belongs; the atlas slice corresponding to each component track includes a first atlas slice and a second atlas slice;

[0010] When the atlas identifier associated with the first atlas fragment identifier of the first atlas fragment is the same as the atlas identifier associated with the second atlas fragment identifier of the second atlas fragment, the first atlas fragment and the second atlas fragment belong to the same fragment track group.

[0011] The component track further includes a component track type; the k component tracks include a first component track and a second component track;

[0012] When the first component track type of the first component track is the same as the second component track type of the second component track, the atlas fragments corresponding to the first component track and the atlas fragments corresponding to the second component track belong to the same fragment track group.

[0013] The component track data box includes a component information data box, the component information data box includes a data box type, and the data box type indicates information about the component track to which the component information data box belongs; the k component tracks include a target component track;

[0014] When the component information data box is encapsulated in the target component track, the component information data box indicates the atlas slice and spatial information corresponding to the target component track.

[0015] Among them, according to the component track data box, k component tracks and the atlas slices corresponding to each component track are encapsulated to obtain the target file, including:

[0016] Acquire track group division information, and acquire component track information of the k component tracks from the component track data boxes respectively included in the k component tracks based on the track group division information; the track group division information indicates a method for grouping the k component tracks;

[0017] Based on the track group division information and component track information, k component tracks are divided into M shard track groups; M is a positive integer;

[0018] Encapsulate the M fragment track groups and the atlas fragments associated with each fragment track group to obtain the target file.

[0019] The track group division information includes atlas division information, and the component track information of the k component tracks includes atlas identifiers associated with atlas slices corresponding to the k component tracks; the atlas identifier represents an identifier of the atlas of the volumetric video to which the atlas slice belongs;

[0020] Based on the track group division information and component track information, k component tracks are divided into M shard track groups, including:

[0021] Among the k component tracks, based on the atlas partitioning information, the component tracks corresponding to the atlas slices associated with the same atlas identifier are grouped to obtain M slice track groups.

[0022] The track group division information includes atlas component division information, and the component track information of the k component tracks includes atlas identifiers associated with the atlas slices corresponding to the k component tracks, and component track types corresponding to the k component tracks; the atlas identifier indicates the identifier of the atlas of the volumetric video to which the atlas slice belongs;

[0023] Based on the track group division information and component track information, k component tracks are divided into M shard track groups, including:

[0024] Among the k component tracks, based on the atlas component partitioning information, the component tracks with the same component track type and associated with the same atlas identifier are grouped to obtain M fragment track groups.

[0025] The k component tracks are divided based on the track group division information and the component track information to obtain M shard track groups, including:

[0026] Determine, based on the component track information, component track types corresponding to the k component tracks, and atlas identifiers to which the atlas slices corresponding to the k component tracks belong; the atlas identifier represents an identifier of the atlas of the volumetric video to which the atlas slices belong;

[0027] If there is a component track to be divided among the k component tracks, then based on the atlas identifier to which the atlas fragment corresponding to the component track to be divided belongs, the component track to be divided is split into at least two component sub-tracks; the component track to be divided refers to a component track corresponding to at least two atlas fragments, and the atlas identifiers corresponding to at least two atlas fragments are different; the atlas fragments corresponding to the component sub-tracks are associated with the same atlas identifier;

[0028] Determine at least two component sub-tracks and a regular component track as candidate component tracks; a regular component track refers to a component track among the k component tracks excluding the component track to be divided;

[0029] The candidate component tracks with the same component track type and associated with the same atlas identifier are divided to obtain M fragment track groups.

[0030] The M fragment track groups and the atlas fragments associated with each fragment track group are encapsulated to obtain a target file, including:

[0031] Get the track index i of the component track type information corresponding to the i-th fragment track group among the M fragment track groups; i is a positive integer less than or equal to M;

[0032] Add track index i to the i-th shard track group, and based on track index i, associate the i-th shard track group with the atlas shards corresponding to the component tracks included in the i-th shard track group;

[0033] According to the association result, the M fragment track groups and the atlas fragments associated with each fragment track group are encapsulated to obtain the target file.

[0034] On the one hand, an embodiment of the present application provides a data processing method, which includes:

[0035] Get k component tracks in the target file; the component track includes a component track data box, which includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer;

[0036] Decapsulate the target file according to the component track data box to obtain the fragmented display content.

[0037] The target file is decapsulated according to the component track data box to obtain the fragmented display content, including:

[0038] When responding to a display request for a target component track, obtaining a target component track data box included in the target component track from the target file, and obtaining a target number of shards corresponding to the target component track from the target component track data box;

[0039] Based on the target number of shards, the target file is iteratively decapsulated h times to obtain the shard display content corresponding to the target component track; h is the target number of shards.

[0040] The target file is decapsulated according to the component track data box to obtain the fragmented display content, including:

[0041] When responding to a display request for a pending atlas slice, obtaining a pending atlas slice identifier of the pending atlas slice;

[0042] Obtain component track data boxes respectively included in k component tracks from the target file, and traverse and search the component track data boxes respectively included in the k component tracks based on the number of slices in the component track data box and the slice identifier of the atlas to be processed;

[0043] The component track to which the component track data box with the atlas fragment identifier to be processed belongs is determined as the component track to be processed;

[0044] Decapsulate the component track to be processed and obtain the fragment display content corresponding to the atlas fragment to be processed.

[0045] The target file is decapsulated according to the component track data box to obtain the fragmented display content, including:

[0046] Decapsulate the target file according to the component track data box to obtain k component tracks and the atlas slice corresponding to each component track;

[0047] Determining, based on spatial region information in component track data boxes respectively included in the k component tracks, spatial regions where the k component tracks are located in the volumetric video;

[0048] Sliced ​​display content is generated based on the spatial regions where the k component tracks are located in the volumetric video and the atlas slices corresponding to each component track.

[0049] On the one hand, an embodiment of the present application provides a data processing device, the device comprising:

[0050] The data acquisition module is used to obtain k component tracks and the atlas slices corresponding to each component track; the component track includes a component track data box, which includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer;

[0051] The data encapsulation module is used to encapsulate k component tracks and the atlas slices corresponding to each component track according to the component track data box to obtain the target file.

[0052] The component track data box includes a track group data box; the track group data box includes an atlas identifier and an atlas slice identifier; the atlas identifier indicates the identifier of the atlas of the volumetric video to which the atlas slice belongs;

[0053] When the atlas slice identifier is associated with the atlas identifier, the atlas slice corresponding to the atlas slice identifier belongs to the atlas of the volumetric video corresponding to the atlas identifier.

[0054] The component track data box includes a track group data box; the track group data box includes an atlas identifier and an atlas slice identifier; the atlas identifier indicates the identifier of the atlas to which the atlas slice belongs; the atlas slice corresponding to each component track includes a first atlas slice and a second atlas slice;

[0055] When the atlas identifier associated with the first atlas fragment identifier of the first atlas fragment is the same as the atlas identifier associated with the second atlas fragment identifier of the second atlas fragment, the first atlas fragment and the second atlas fragment belong to the same fragment track group.

[0056] The component track further includes a component track type; the k component tracks include a first component track and a second component track;

[0057] When the first component track type of the first component track is the same as the second component track type of the second component track, the atlas fragments corresponding to the first component track and the atlas fragments corresponding to the second component track belong to the same fragment track group.

[0058] The component track data box includes a component information data box, the component information data box includes a data box type, and the data box type indicates information about the component track to which the component information data box belongs; the k component tracks include a target component track;

[0059] When the component information data box is encapsulated in the target component track, the component information data box indicates the atlas slice and spatial information corresponding to the target component track.

[0060] The data encapsulation module includes:

[0061] an information acquisition unit, configured to acquire track group division information, and acquire component track information of the k component tracks from the component track data boxes respectively included in the k component tracks based on the track group division information; the track group division information indicates a method for grouping the k component tracks;

[0062] The track grouping unit is used to divide the k component tracks based on the track group division information and the component track information to obtain M fragment track groups; M is a positive integer;

[0063] The track encapsulation unit is used to encapsulate the M fragment track groups and the atlas fragments associated with each fragment track group to obtain the target file.

[0064] The track group division information includes atlas division information, and the component track information of the k component tracks includes atlas identifiers associated with atlas slices corresponding to the k component tracks; the atlas identifier represents an identifier of the atlas of the volumetric video to which the atlas slice belongs;

[0065] The track grouping unit comprises:

[0066] The atlas grouping subunit is used to group the component tracks corresponding to atlas slices associated with the same atlas identifier in the k component tracks based on the atlas partitioning information to obtain M slice track groups.

[0067] The track group division information includes atlas component division information, and the component track information of the k component tracks includes atlas identifiers associated with the atlas slices corresponding to the k component tracks, and component track types corresponding to the k component tracks; the atlas identifier indicates the identifier of the atlas of the volumetric video to which the atlas slice belongs;

[0068] The track grouping unit comprises:

[0069] The type grouping subunit is used to group the component tracks with the same component track type and associated with the same atlas identifier among the k component tracks based on the atlas component partitioning information to obtain M fragment track groups.

[0070] The track grouping unit includes:

[0071] a type determination subunit, configured to determine, based on the component track information, component track types corresponding to the k component tracks, and atlas identifiers to which the atlas slices corresponding to the k component tracks belong; the atlas identifier represents an identifier of an atlas of the volumetric video to which the atlas slices belong;

[0072] The track splitting subunit is configured to split the component track to be divided into at least two component sub-tracks based on the atlas identifier to which the atlas fragment corresponding to the component track to be divided belongs, if there is a component track to be divided among the k component tracks; the component track to be divided is a component track corresponding to at least two atlas fragments, and the atlas identifiers corresponding to the atlas fragments are different; the atlas fragments corresponding to the component sub-tracks are associated with the same atlas identifier;

[0073] The track determination subunit is used to determine at least two component sub-tracks and a regular component track as candidate component tracks; the regular component track refers to the component tracks other than the component track to be divided among the k component tracks;

[0074] The track grouping subunit is used to divide candidate component tracks having the same component track type and associated with the same atlas identifier into M fragment track groups.

[0075] The track packaging unit includes:

[0076] The index acquisition subunit is used to obtain the track index i of the component track type information corresponding to the i-th fragment track group among the M fragment track groups; i is a positive integer less than or equal to M;

[0077] The index association subunit is configured to add a track index i to the i-th fragment track group and associate the i-th fragment track group with the atlas fragments corresponding to the component tracks included in the i-th fragment track group based on the track index i.

[0078] The file generation subunit is used to encapsulate the M fragment track groups and the atlas fragments associated with each fragment track group according to the association result to obtain the target file.

[0079] On the one hand, an embodiment of the present application provides a data processing device, the device comprising:

[0080] The track acquisition module is used to obtain k component tracks in the target file; the component track includes a component track data box, which includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer;

[0081] The data decapsulation module is used to decapsulate the target file according to the component track data box to obtain the fragmented display content.

[0082] The data decapsulation module includes:

[0083] a quantity obtaining unit, configured to, when responding to a display request for a target component track, obtain a target component track data box included in the target component track from the target file, and obtain the target shard quantity corresponding to the target component track from the target component track data box;

[0084] The decapsulation iteration unit is used to iteratively decapsulate the target file h times based on the target number of fragments to obtain the fragment display content corresponding to the target component track; h is the target number of fragments.

[0085] The data decapsulation module includes:

[0086] an identification obtaining unit, configured to obtain an identification of the atlas fragment to be processed when responding to a display request for the atlas fragment to be processed;

[0087] a data box search unit, configured to obtain component track data boxes respectively included in k component tracks from a target file, and traverse and search the component track data boxes respectively included in the k component tracks based on the number of shards in the component track data box and the shard identifier of the atlas to be processed;

[0088] A track selection unit, configured to determine the component track to which the component track data box with the atlas fragment identifier to be processed belongs as the component track to be processed;

[0089] The track decapsulation unit is used to decapsulate the track of the component to be processed to obtain the fragment display content corresponding to the fragment of the atlas to be processed.

[0090] The data decapsulation module includes:

[0091] A file decapsulation unit is used to decapsulate the target file according to the component track data box to obtain k component tracks and the atlas slice corresponding to each component track;

[0092] a space determination unit, configured to determine the spatial regions in which the k component tracks are located in the volumetric video based on spatial region information in component track data boxes respectively included in the k component tracks;

[0093] The content generation unit is configured to generate slice display content according to the spatial regions where the k component tracks are located in the volumetric video and the atlas slice corresponding to each component track.

[0094] On the one hand, an embodiment of the present application provides a computer device, including a processor, a memory, and an input and output interface;

[0095] The above-mentioned processor is respectively connected to the above-mentioned memory and the above-mentioned input and output interface, wherein the above-mentioned input and output interface is used to receive data and output data, the above-mentioned memory is used to store program code, and the above-mentioned processor is used to call the above-mentioned program code to execute the above-mentioned data processing method in one aspect of the embodiment of the present application.

[0096] On one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, the data processing method described above in one aspect of the embodiment of the present application is executed.

[0097] Implementing the embodiments of this application will have the following beneficial effects:

[0098] The embodiment of the present application can obtain k component tracks and the atlas slices corresponding to each component track; the component track includes a component track data box, and the component track data box includes the number of slices and spatial area information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial area information refers to the spatial area of ​​the component track to which the component track data box belongs; k is a positive integer; the k component tracks and the atlas slices corresponding to each component track are encapsulated according to the component track data box to obtain a target file. Through the above process, a component track data box can be encapsulated in each component track, and the component track data box indicates the relevant information of the component track where it is located. The association between the atlas slice information and the component track (i.e., the atlas slice and the component track) can be refined to the granularity of a single component track, so that the information of the component track can be retained as comprehensively as possible, and the information of the component track in the encapsulated target file is also more organized and refined. Furthermore, after the target file is sent to the user device, the user device can accurately select the required component tracks to achieve the purpose of partial decapsulation and partial decoding, thereby eliminating the need to fully decapsulate and decode the target file, saving time and resources consumed in data processing, and thereby improving the accuracy and efficiency of data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0100] Figure 1 This is a data processing architecture diagram provided by an embodiment of the present application;

[0101] Figure 2 This is a schematic diagram of a component track generation scenario provided by an embodiment of the present application;

[0102] Figure 3 This is a schematic diagram of a data processing scenario provided by an embodiment of the present application;

[0103] Figure 4 This is a flow chart of a data encapsulation method provided in an embodiment of the present application;

[0104] Figure 5 This is a schematic diagram of track group division provided in an embodiment of the present application;

[0105] Figure 6 This is a flow chart of a data decapsulation method provided in an embodiment of the present application;

[0106] Figure 7 This is a schematic diagram of a data processing scenario provided by an embodiment of the present application;

[0107] Figure 8 is a schematic diagram of a data processing device provided in an embodiment of the present application;

[0108] Figure 9 This is a schematic diagram of another data processing device provided in an embodiment of the present application;

[0109] Figure 10 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0110] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0111] Embodiments of the present application relate to data processing technology, which is a data encapsulation and decapsulation technology for an atlas. The atlas is used to indicate regional information on a 2D plane frame, regional information in a 3D presentation space, a mapping relationship between the two, and necessary parameter information required for the mapping. The atlas may refer to an atlas corresponding to a volumetric video. Volumetric video refers to immersive media that captures visual content from a three-dimensional space and provides a viewing experience such as 3 degrees of freedom plus (3DoF+) or 6 degrees of freedom (6DoF), and includes a volumetric video type track in the file encapsulation, including multi-view video and video encoding point cloud. Among them, the multi-view / multi-viewpoint video refers to a video with depth information shot from multiple angles using multiple camera arrays. The multi-view / multi-viewpoint video is also called free-view / free-viewpoint video, which is an immersive media that provides a six-degree-of-freedom experience; the point cloud is a set of discrete points in space that are irregularly distributed and express the spatial structure and surface properties of a three-dimensional object or scene. Each point in the point cloud has at least three-dimensional position information, and may also have color, material or other information depending on the application scenario. Usually, each point in the point cloud has the same number of additional attributes.

[0112] Furthermore, Degree of Freedom (DoF) refers to the freedom of movement supported by users when watching immersive media and generating content interaction; 3DoF refers to the three degrees of freedom of the user's head rotating around the coordinate axes of three-dimensional space (x-axis, y-axis and z-axis); 3DoF+ means that on the basis of three degrees of freedom, the user also has the freedom of limited movement along the x-axis, y-axis and z-axis; 6DoF means that on the basis of three degrees of freedom, the user also has the freedom of free movement along the x-axis, y-axis and z-axis.

[0113] Optionally, the present application is applicable to the encapsulation and decapsulation of volumetric videos such as multi-view / multi-viewpoint videos and point cloud compression (Video-based Point Cloud Compression, V-PCC) point cloud media based on traditional video coding.

[0114] For details, see Figure 1 , Figure 1 A data processing architecture diagram provided in an embodiment of the present application is as follows: Figure 1 As shown, the embodiment of the present application is implemented by a computer device 101, and the computer device 101 may be composed of a server and a terminal device; the computer device 101 may also be a server or a terminal device, which is not limited here.

[0115] For example, the computer device 101 may obtain volumetric video data, including an atlas corresponding to the volumetric video, where the atlas consists of N atlas slices, where N is a positive integer. Alternatively, the computer device 101 may obtain one or at least two volumetric videos, where one volumetric video may correspond to one atlas, or one volumetric video may correspond to at least two atlases, or at least two volumetric videos may correspond to one atlas. Alternatively, if one volumetric video corresponds to at least two atlases, the computer device 101 may add an atlas identifier to each atlas, and based on the atlas identifier, the volumetric video to which the atlas belongs may be determined. Specifically, the computer device 101 may obtain k component tracks to be packaged and the atlas slices corresponding to each component track, where the component tracks include component track data boxes. Based on the component track data boxes included in each of the k component tracks, the computer device 101 may package the k component tracks and the atlas slices corresponding to each component track to obtain a target file, where k is a positive integer. Computer device 101 may send the target file to a user device, such as user device 102a, user device 102b, or user device 102c. Specifically, computer device 101 may receive a file acquisition request from the user device for the target file and, based on the file acquisition request, send the target file to the user device. Alternatively, computer device 101 may obtain a data acquisition request from the user device for a volumetric video and, based on the data acquisition request, obtain k component tracks associated with the volumetric video and atlas slices corresponding to each component track. Based on the component track data box included in each component track, the k component tracks and atlas slices corresponding to each component track are packaged to obtain the target file. Alternatively, computer device 101 may obtain an atlas corresponding to the volumetric video and slice the atlas to obtain the atlas slices included in the atlas. Furthermore, when slicing the atlas, the component tracks associated with each atlas slice may be obtained based on video information of the volumetric video. Based on the association between the component tracks and the atlas slices, k component tracks and atlas slices corresponding to each component track are obtained.

[0116] In other words, upon receiving a request to obtain target data, the computer device 101 can obtain k component tracks associated with the target data and the atlas slices corresponding to each component track. Based on the component track data box included in each component track, the computer device 101 can encapsulate the k component tracks and the atlas slices corresponding to each component track to obtain a target file, and send the target file to the user device that initiated the request to obtain the target data. The target data can be a target file, in which case the target file can be generated in advance; the target data can be a volumetric video, in which case the target file can be generated in advance or after receiving the request to obtain the target data; the target data can also be a component track, that is, the target data includes k component tracks, and the computer device can obtain the atlas slices corresponding to the target data, etc., without limitation herein.

[0117] Optionally, the data in this application can be stored in a computer device, or can be stored based on blockchain technology or cloud storage technology, that is, the data in this application can also be stored in a blockchain network or cloud space, etc., without limitation. For example, the computer device 101 can store k component tracks and the atlas slices corresponding to each component track, as well as the component association information of each component track, the target file obtained by encapsulating the k component tracks and the atlas slices corresponding to each component track, etc., in one or any multiple (at least two) locations in the computer device 101, blockchain network or cloud space, etc. The computer device 101 can store the data in this application in one location to save resources; it can also be stored in multiple locations to ensure the security of the data.

[0118] The embodiment of the present application encapsulates a component track data box in each component track. The component track data box can be used to represent the component association information of the corresponding component track, so that the component association information of each component track can be obtained, and the information of the component track can be retained as comprehensively as possible. The information of the component track in the encapsulated target file is also more organized and refined, so that the user device that receives the target file can decapsulate and obtain the required component track based on the component track data box of each component track, saving time and resources consumed in data processing.

[0119] Further, see Figure 2 , Figure 2 This is a schematic diagram of a component track generation scenario provided by an embodiment of the present application. Figure 2As shown, taking a volumetric video corresponding to an atlas as an example, embodiments of the present application are implemented using a computer device. The computer device can obtain a volumetric video 201, parse and extract information from the volumetric video 201 to obtain an atlas 202 for the volumetric video 201, and then slice the atlas 202 of the volumetric video 201 to obtain N atlas slices 203 corresponding to the atlas 202. Specifically, the atlas 202 of the volumetric video 201 can be sliced ​​by region. Based on the volumetric video 201, component tracks corresponding to the N atlas slices 203 are obtained. Based on the association between the atlas slices and the component tracks, an atlas track 204 is obtained. Specifically, the computer device obtains the component tracks corresponding to the N atlas slices 203 by obtaining slice information for each atlas slice, determining the component track type corresponding to each atlas slice based on the slice information type corresponding to the slice information, and then obtaining the component track corresponding to each atlas slice based on the component track type corresponding to each atlas slice. For example, if the slice information type corresponding to the slice information is a color information type, the computer device can determine the component track type corresponding to the atlas slice associated with the slice information as an attribute component type; if the slice information type corresponding to the slice information is a position information type, the computer device can determine the component track type corresponding to the atlas slice associated with the slice information as a geometric component type, etc. The atlas track 204 can be a track including a group of component tracks, that is, the atlas track 204 can include at least two component tracks and atlas slices corresponding to each component track. One atlas slice can correspond to one component track, or to at least two component tracks, or one component track can correspond to at least two atlas slices, etc., which are not limited here. Optionally, if the computer device generates a target file corresponding to the volumetric video, the number of the at least two component tracks is k.

[0120] Further, see Figure 3 , Figure 3 This is a schematic diagram of a data processing scenario provided by an embodiment of the present application. Figure 3As shown, computer device 301 can obtain component association information 302 corresponding to k component tracks, including k component tracks and the atlas slices corresponding to each component track, such as component track 1 and the atlas slice corresponding to component track 1, component track 2 and the atlas slice corresponding to component track 2, and component track k and the atlas slice corresponding to component track k. Each component track includes a component track data box. Computer device 301 can encapsulate the k component tracks and the component association information 302 of each component track to obtain a target file, and send the target file to user device 303. User device 303 can obtain the required component track from the target file, decapsulate the component association information of the component track, and obtain slice display content 304. Furthermore, user device 303 can display the slice display content 304.

[0121] It is understood that computer devices include but are not limited to terminal devices or servers, and can be considered as user terminals (UE), and user devices can include but are not limited to terminal devices or servers. In other words, a computer device can be a server or a terminal device, or a system composed of a server and a terminal device. Among them, the terminal device mentioned above can be an electronic device, including but not limited to mobile phones, tablet computers, desktop computers, laptop computers, PDAs, vehicle-mounted devices, augmented reality / virtual reality (AR / VR) devices, helmet displays, smart TVs, wearable devices, smart speakers, digital cameras, cameras and other mobile internet devices (MID) with network access capabilities, or terminal devices in scenarios such as trains, ships, and airplanes. Among them, the servers mentioned above can be independent physical servers, or they can be server clusters or distributed systems composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, vehicle-road collaboration, content delivery networks (CDNs), as well as big data and artificial intelligence platforms.

[0122] In this application, "or" refers to any one or any multiple of a plurality of items. For example, "A or B" refers to "A", "B" and "A and B".

[0123] Further, see Figure 4 , Figure 4 This is a flow chart of a data encapsulation method provided by an embodiment of the present application. Figure 4As shown, the data encapsulation process includes the following steps:

[0124] Step S401: Obtain k component tracks and the atlas slice corresponding to each component track.

[0125] Specifically, each component track includes a component track data box, which contains the number of tiles (which can be expressed as num_tiles) and spatial region information (which can be expressed as 3D_spatial_region_flag). The number of tiles refers to the number of atlas tiles corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs. k is a positive integer. Optionally, the component track data box may also include an atlas identifier and an atlas tile identifier.

[0126] Optionally, there may be one or at least two track group division information, each track group division information corresponding to a track group division method. The track group division information may be modified as needed, such as adding track group division information or deleting track group division information.

[0127] In a track group division method, the atlas slice corresponding to each component track includes a first atlas slice and a second atlas slice, that is, the first atlas slice included in the first component track and the second atlas slice included in the second component track. When the atlas identifier associated with the first atlas slice identifier of the first atlas slice is the same as the atlas identifier associated with the second atlas slice identifier of the second atlas slice, the first atlas slice and the second atlas slice belong to the same slice track group, that is, the first component track corresponding to the first atlas slice and the second component track corresponding to the second atlas slice can be added to the same slice track group. At this time, it can be considered that each slice track group is used to represent the information of the component track corresponding to an atlas, that is, the atlas slices corresponding to the component tracks in a slice track group belong to the same atlas.

[0128] In a track group partitioning method, component tracks also include component track types. Assuming that k component tracks include a first component track and a second component track, when the first component track type of the first component track is the same as the second component track type of the second component track, and the atlas identifier associated with the atlas fragment corresponding to the first component track is the same as the atlas identifier associated with the atlas fragment corresponding to the second component track, the atlas fragment corresponding to the first component track and the atlas fragment corresponding to the second component track belong to the same fragment track group. In other words, the component tracks included in the same fragment track group have the same component track type, and the atlas fragments corresponding to the component tracks belong to the same atlas.

[0129] The component track data box may include a track group data box, which includes an atlas identifier and an atlas slice identifier. The atlas identifier represents the identifier of the atlas of the volumetric video to which the atlas slice belongs. Optionally, when the atlas slice identifier is associated with the atlas identifier, the atlas slice corresponding to the atlas slice identifier belongs to the atlas of the volumetric video corresponding to the atlas identifier. The track group data box may also include information such as the number of slices and spatial region.

[0130] Optionally, the component track data box may include a component information data box, which may include a data box type. The data box type indicates that the component information data box indicates information about the component track to which the component information data box belongs. The k component tracks include the target component track. Specifically, the component information data box may include a data box type, a parent data box to which the component information data box belongs (such as the SchemeInformationBox data box), an execution state (including a mandatory execution state and an optional execution state), and the number of data boxes. The component information data box may also include the number of shards, spatial region information, and atlas identifiers. When the component information data box is encapsulated in the target component track, the component information data box indicates the atlas shards and spatial information corresponding to the target component track.

[0131] Optionally, the component track data box may include one or at least two of a track group data box and a component information data box, wherein the track group data box and the component information data box are located at different positions in the component track.

[0132] For example, see Figure 5 , Figure 5 This is a schematic diagram of track group division provided by an embodiment of the present application. Figure 5 As shown, the computer device obtains k component tracks and the atlas slices corresponding to each component track, for example, component track 1 (denoted as track11), component track 2 (denoted as track12), component track 3 (denoted as track13) and component track 4 (denoted as track14), wherein the component track type corresponding to track11 is a geometric component type, corresponding to atlas slice 1 (denoted as tile1); the component track type corresponding to track12 is a geometric component type, corresponding to atlas slice 2 (denoted as tile2); the component track type corresponding to track13 is an attribute component type, corresponding to atlas slice 3 (denoted as tile3); the component track type corresponding to track14 is an attribute component type, corresponding to atlas slice 4 (denoted as tile4). Optionally, the component track type may include but is not limited to a placeholder component type, a geometric component type, an attribute component type and an encapsulation component type.

[0133] Step S402 : encapsulate k component tracks and the atlas slice corresponding to each component track according to the component track data box included in the component track to obtain a target file.

[0134] Specifically, the computer device can obtain track group division information, and based on the track group division information, obtain component track information of the k component tracks from the component track data boxes respectively included in the k component tracks; the track group division information indicates the method of grouping the k component tracks. For example, if the track group division information is used to indicate a track group division method based on an atlas, the component track information obtained by the computer device may include the atlas to which the atlas fragments corresponding to the k component tracks belong; if the track group division information is used to indicate a track group division method based on an atlas and component track type, the component track information obtained by the computer device may include the atlas to which the atlas fragments corresponding to the k component tracks belong, as well as the component track type of each component track. The k component tracks are divided based on the track group division information and the component track information to obtain M fragment track groups; M is a positive integer. The M fragment track groups and the atlas fragments associated with each fragment track group are encapsulated to obtain the target file.

[0135] Specifically, in a track group division method, the track group division information includes atlas division information, and the component track information of k component tracks includes atlas identifiers associated with atlas slices corresponding to the k component tracks; the atlas identifier represents the identifier of the atlas of the volumetric video to which the atlas slice belongs. When the k component tracks are divided based on the track group division information and the component track information to obtain M slice track groups, the computer device can group the component tracks corresponding to atlas slices associated with the same atlas identifier within the k component tracks based on the atlas division information to obtain the M slice track groups.

[0136] In a track group division method, track group division information includes atlas component division information. The component track information of k component tracks includes atlas identifiers associated with atlas slices corresponding to the k component tracks, and component track types corresponding to the k component tracks. The atlas identifier indicates the identifier of the atlas of the volumetric video to which the atlas slice belongs. When the k component tracks are divided based on the track group division information and the component track information to obtain M slice track groups, the computer device can group component tracks with the same component track type and associated with the same atlas identifier within the k component tracks based on the atlas component division information to obtain the M slice track groups.

[0137] Optionally, the computer device can determine the component track types corresponding to the k component tracks respectively, and the atlas identifiers to which the atlas slices corresponding to the k component tracks respectively belong based on the component track information; the atlas identifier represents the identifier of the atlas of the volumetric video to which the atlas slice belongs. If there is a component track to be divided among the k component tracks, then based on the atlas identifier to which the atlas slice corresponding to the component track to be divided belongs, the component track to be divided is split into at least two component sub-tracks; the component track to be divided refers to a component track corresponding to at least two atlas slices, and the atlas identifiers corresponding to at least two atlas slices are different; the atlas slices corresponding to the component sub-tracks are associated with the same atlas identifier. For example, there is a component track A corresponding to atlas slice 1 and atlas slice 2, wherein atlas slice 1 corresponds to atlas identifier 1, and atlas slice 2 corresponds to atlas identifier 2, then the component track A can be determined as the component track to be divided, and the component track to be divided is split into at least two component sub-tracks, including component sub-track 1 (corresponding to atlas slice 1) and component sub-track 2 (corresponding to atlas slice 2). Furthermore, at least two component sub-tracks and a regular component track are determined as candidate component tracks; a regular component track refers to a component track among the k component tracks excluding the component track to be divided. Candidate component tracks with the same component track type and associated with the same atlas identifier are divided to obtain M fragment track groups.

[0138] Among them, obtain the track index i of the component track type information corresponding to the i-th shard track group in the M shard track groups; i is a positive integer less than or equal to M. Add track index i to the i-th shard track group, and based on the track index i, associate the i-th shard track group with the atlas shards corresponding to the component tracks included in the i-th shard track group. According to the association result, encapsulate the M shard track groups and the atlas shards associated with each shard track group to obtain the target file. Among them, the track index can be represented by a string, and the string can be generated according to the generation order of the track index, that is, it can represent the generation order of the track index; it can also be a randomly generated unique index; or it can be determined based on the data type carried by the component track, etc., and there is no limitation here.

[0139] For example, the track index may be shown in Table 1:

[0140]

[0141] As shown in Table 1, the track index includes but is not limited to v3vo, v3vg, v3va and v3vp, and the component track type of the component track included in the corresponding shard track group can be represented by the track index. Among them, when determining the shard track group based on the component track type, the component track type of the component track included in each shard track group is the same, and the component track type of the component track included in the shard track group can be represented based on the track index to reduce the amount of data that needs to be processed. For example, there are component tracks 1 to 10, all of which are geometric component types. The computer device can divide component tracks 1 to 10 into a shard track group and mark the shard track group based on the track index "v3vg", so that there is no need to carry the component track type for each component track, thereby reducing the amount of data that needs to be processed.

[0142] For example, if Figure 5 As shown, it can be seen that component track 1 (track11) and component track 2 (track12) both correspond to the geometry component type, and component track 3 (track13) and component track 4 (track14) both correspond to the attribute component type. Based on the component track type, the computer device can put track11 and track12 into a track group, namely, shard track group 1, and put track13 and track14 into a track group, namely, shard track group 2. Optionally, based on Table 1, the track index "v3vg" can be used to mark the shard track group 1, and the atlas shards corresponding to track11 and track12 can be associated with the shard track group 1. The shard track group 1 can also be associated with the atlas track (such as Figure 2 As shown); the track index "v3va" can be used to mark the fragment track group 2, and the atlas fragments corresponding to track13 and track14 can be associated with the fragment track group 2. The fragment track group 2 can also be associated with the atlas track (as shown). Figure 2 shown).

[0143] The embodiment of the present application can obtain k component tracks and the atlas slices corresponding to each component track; the component track includes a component track data box, and the component track data box includes the number of slices and spatial area information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial area information refers to the spatial area of ​​the component track to which the component track data box belongs; k is a positive integer; the k component tracks and the atlas slices corresponding to each component track are encapsulated according to the component track data box to obtain a target file. Through the above process, a component track data box can be encapsulated in each component track, and the component track data box indicates the relevant information of the component track where it is located. The association between the atlas slice information and the component track (i.e., the atlas slice and the component track) can be refined to the granularity of a single component track, so that the information of the component track can be retained as comprehensively as possible, and the information of the component track in the encapsulated target file is also more organized and refined. Furthermore, after the target file is sent to the user device, the user device can accurately select the required component tracks to achieve the purpose of partial decapsulation and partial decoding, thereby eliminating the need to fully decapsulate and decode the target file, saving time and resources consumed in data processing, and thereby improving the accuracy and efficiency of data processing.

[0144] Further, see Figure 6 , Figure 6 This is a flow chart of a data decapsulation method provided by an embodiment of the present application. Figure 6 As shown, the following steps are included:

[0145] Step S601: Obtain k component tracks in a target file, where the component tracks include component track data boxes.

[0146] Specifically, the component track includes a component track data box, which includes the number of shards and spatial region information; the number of shards refers to the number of atlas shards corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer.

[0147] Step S602: decapsulate the target file according to the component track data box to obtain fragmented display content.

[0148] Specifically, when responding to a display request for a target component track, the target component track data box included in the target component track is obtained from the target file, and the target number of shards corresponding to the target component track is obtained from the target component track data box. The position of the target component track data box in the component track is known, and the computer device can obtain the data box position and obtain the target component track data box at the data box position in the target component track. Optionally, the target component track data box can be a track group data box, a component information data box, or a combination of a track group data box and a component information data box. If the computer device determines the target component track data box included in the target track component and obtains the target data box position of the target component track data box, then the target component track data box is obtained at the target data box position in the target component track; if the computer device does not obtain the constituent elements of the target component track data box, or the target component track data box includes a track group data box and a component information data box, then the first data box position corresponding to the track group data box and the second data box position corresponding to the component information data box can be obtained, and a search is performed at the first data box position and the second data box position of the target component track, and subsequent processing is performed based on the found data boxes. In other words, the track group data box and the component information data box can be replaced with each other and are essentially used to represent the same information. The target component track can be any component track among the k component tracks.

[0149] Furthermore, the computer device may iteratively decapsulate the target file h times based on the target number of slices to obtain slice display content corresponding to the target component track, where h is the target number of slices. In other words, the target component track corresponds to h atlas slices, and the target component track in the target file is iteratively decapsulated h times to obtain atlas slice information corresponding to each of the h atlas slices.

[0150] When responding to a display request for an atlas fragment to be processed, obtain the atlas fragment identifier of the atlas fragment to be processed. Obtain the component track data boxes respectively included in the k component tracks from the target file, and traverse and search the component track data boxes respectively included in the k component tracks based on the number of fragments in the component track data box and the atlas fragment identifier to be processed. Specifically, the component track corresponds to h atlas fragments, that is, the component track data box included in the component track includes the atlas fragment identifiers of the h atlas fragments corresponding to the component track. The computer device can traverse and search the atlas fragment identifiers in the component track data boxes respectively included in the k component tracks based on the number of fragments h. If the atlas fragment identifier to be processed is not obtained after h searches, it means that the atlas fragment identifier to be processed does not exist in the component track currently being searched. Further, the component track to which the component track data box with the atlas fragment identifier to be processed belongs is determined as the component track to be processed; the component track to be processed is decapsulated to obtain the fragment display content corresponding to the atlas fragment to be processed. As Figure 5 As shown, if the atlas tile to be processed is tile1, the computer device traverses and searches the component track data boxes respectively included in the k component tracks to obtain the component tracks to be processed, including component track 1 and component track 3.

[0151] Further optionally, the computer device can decapsulate the target file according to the component track data box to obtain k component tracks and the atlas slice corresponding to each component track; based on the spatial area information in the component track data boxes respectively included in the k component tracks, determine the spatial area where the k component tracks are located in the volumetric video; and generate slice display content according to the spatial area where the k component tracks are located in the volumetric video and the atlas slice corresponding to each component track.

[0152] Optionally, when responding to a display request for a target component track, the computer device may obtain the target track component from the target file, decapsulate the target track component according to the target component track data box included in the target track component, and obtain a coded bit stream corresponding to the target track component; decode the coded bit stream to generate a decoding signal corresponding to the coded bit stream; obtain atlas slice data corresponding to the target component track, and determine point cloud information of the atlas slice corresponding to the target component track based on the atlas slice data; and determine the slice display content corresponding to the target component track based on the point cloud information.

[0153] The embodiment of the present application can obtain k component tracks and the atlas slices corresponding to each component track; the component track includes a component track data box, and the component track data box includes the number of slices and spatial area information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial area information refers to the spatial area of ​​the component track to which the component track data box belongs; k is a positive integer; the k component tracks and the atlas slices corresponding to each component track are encapsulated according to the component track data box to obtain a target file. Through the above process, a component track data box can be encapsulated in each component track, and the component track data box indicates the relevant information of the component track where it is located. The association between the atlas slice information and the component track (i.e., the atlas slice and the component track) can be refined to the granularity of a single component track, so that the information of the component track can be retained as comprehensively as possible, and the information of the component track in the encapsulated target file is also more organized and refined. Furthermore, after the target file is sent to the user device, the user device can accurately select the required component tracks to achieve the purpose of partial decapsulation and partial decoding, thereby eliminating the need to fully decapsulate and decode the target file, saving time and resources consumed in data processing, and thereby improving the accuracy and efficiency of data processing.

[0154] Further, see Figure 7 , Figure 7 This is a schematic diagram of a data processing scenario provided by an embodiment of the present application. Figure 7 As shown, the real world or synthetic visual scene (A) is captured by a group of cameras or virtual cameras, and a camera has multiple lenses and sensors. The real world or synthetic visual scene (A) can refer to volumetric video, etc. The acquisition result of the computer device is the source volumetric data (B). The source volumetric data is encoded to obtain an encoded V3C bit stream (Ev). The encoded V3C bit stream includes an atlas bit stream, at most one occupancy bit stream, a geometry bit stream, and zero or at least one attribute bit stream. Furthermore, the computer device can perform file / segment encapsulation on one or more encoded V3C bit streams according to a specific media container file format to obtain a media file (F) for local playback or an initialization segment and a sequence of media segments (Fs) for streaming. The media container file format can be the ISO basic media file format specified in ISO / IEC14496-12. The sequence of initialization segments and media segments (Fs) can be sent to the user device based on the delivery mechanism.

[0155] Furthermore, the media file (F) output by the file encapsulator is the same as the file (F') used as input by the file decapsulator. The file decapsulator can process the file (F') or the received segment (Fs'), that is, perform file / segment decapsulation on the processed file (F') or the received segment (Fs'), extract the encoded bit stream (Ev'), decode the encoded bit stream, and obtain a decoded signal (D'). Based on the current viewing direction or viewport, the decoded signal (D') is reconstructed to obtain reconstruction information (B'), the reconstruction information (B') is rendered to obtain segmented display content (A'), and the segmented display content (A') can be displayed in the user device, such as on a head-mounted display or a screen of any other display device. Optionally, the current viewing direction is determined by a head tracking or eye tracking function. Optionally, the computer device can pass the current viewing direction to a policy module, and determine the trajectory to be received based on the policy module.

[0156] Among them, when performing file / segment encapsulation on one or more encoded V3C bitstreams, the Figure 4 In the steps shown, one or more encoded V3C bitstreams are encapsulated to obtain a target file, including a media file (F) or a sequence of initialization segments and media segments (Fs). When decapsulating the processed file (F') or the received segment (Fs'), the following can be used: Figure 6 In each step, the processed file (F') or the received segment (Fs') is decapsulated to obtain the fragmented display content.

[0157] See also Figure 8 , Figure 8 Schematic diagram of a data processing device provided in an embodiment of the present application. The above-mentioned data processing device can be a computer program (including program code) running on a computer device, for example, the data processing device is an application software; the device can be used to execute the corresponding steps of the method provided in an embodiment of the present application. Figure 8 As shown, the data processing device 800 can be used for the above Figure 4 The computer device in the corresponding embodiment may specifically include: a data acquisition module 11 and a data packaging module 12 .

[0158] The data acquisition module 11 is used to acquire k component tracks and the atlas slices corresponding to each component track; the component track includes a component track data box, which includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer;

[0159] The data encapsulation module 12 is configured to encapsulate the k component tracks and the atlas slices corresponding to each component track according to the component track data box to obtain a target file.

[0160] The component track data box includes a track group data box; the track group data box includes an atlas identifier and an atlas slice identifier; the atlas identifier indicates the identifier of the atlas of the volumetric video to which the atlas slice belongs;

[0161] When the atlas slice identifier is associated with the atlas identifier, the atlas slice corresponding to the atlas slice identifier belongs to the atlas of the volumetric video corresponding to the atlas identifier.

[0162] The component track data box includes a track group data box; the track group data box includes an atlas identifier and an atlas slice identifier; the atlas identifier indicates the identifier of the atlas to which the atlas slice belongs; the atlas slice corresponding to each component track includes a first atlas slice and a second atlas slice;

[0163] When the atlas identifier associated with the first atlas fragment identifier of the first atlas fragment is the same as the atlas identifier associated with the second atlas fragment identifier of the second atlas fragment, the first atlas fragment and the second atlas fragment belong to the same fragment track group.

[0164] The component track further includes a component track type; the k component tracks include a first component track and a second component track;

[0165] When the first component track type of the first component track is the same as the second component track type of the second component track, the atlas fragments corresponding to the first component track and the atlas fragments corresponding to the second component track belong to the same fragment track group.

[0166] The component track data box includes a component information data box, the component information data box includes a data box type, and the data box type indicates information about the component track to which the component information data box belongs; the k component tracks include a target component track;

[0167] When the component information data box is encapsulated in the target component track, the component information data box indicates the atlas slice and spatial information corresponding to the target component track.

[0168] The data encapsulation module 12 includes:

[0169] The information acquisition unit 121 is configured to acquire track group division information, and based on the track group division information, acquire component track information of the k component tracks from the component track data boxes respectively included in the k component tracks; the track group division information indicates a method for grouping the k component tracks;

[0170] The track grouping unit 122 is configured to divide the k component tracks based on the track group division information and the component track information to obtain M fragment track groups; M is a positive integer;

[0171] The track encapsulation unit 123 is configured to encapsulate the M slice track groups and the atlas slices associated with each slice track group to obtain a target file.

[0172] The track group division information includes atlas division information, and the component track information of the k component tracks includes atlas identifiers associated with atlas slices corresponding to the k component tracks; the atlas identifier represents an identifier of the atlas of the volumetric video to which the atlas slice belongs;

[0173] The track grouping unit 122 includes:

[0174] The atlas grouping subunit 1221 is configured to group the component tracks corresponding to atlas slices associated with the same atlas identifier in the k component tracks based on the atlas partitioning information to obtain M slice track groups.

[0175] The track group division information includes atlas component division information, and the component track information of the k component tracks includes atlas identifiers associated with the atlas slices corresponding to the k component tracks, and component track types corresponding to the k component tracks; the atlas identifier indicates the identifier of the atlas of the volumetric video to which the atlas slice belongs;

[0176] The track grouping unit 122 includes:

[0177] The type grouping subunit 1222 is configured to group the component tracks having the same component track type and associated with the same atlas identifier among the k component tracks based on the atlas component partitioning information, to obtain M fragment track groups.

[0178] The track grouping unit 122 includes:

[0179] The type determination subunit 1223 is configured to determine, based on the component track information, the component track types corresponding to the k component tracks and the atlas identifiers to which the atlas slices corresponding to the k component tracks belong; the atlas identifier represents an identifier of the atlas of the volumetric video to which the atlas slices belong;

[0180] The track splitting subunit 1224 is configured to split the component track to be split into at least two component sub-tracks based on the atlas identifier to which the atlas fragment corresponding to the component track to be split belongs, if there is a component track to be split among the k component tracks; the component track to be split is a component track corresponding to at least two atlas fragments, and the atlas identifiers corresponding to the atlas fragments are different; the atlas fragments corresponding to the component sub-tracks are associated with the same atlas identifier;

[0181] The track determination subunit 1225 is configured to determine at least two component sub-tracks and a regular component track as candidate component tracks; the regular component track refers to the component tracks among the k component tracks excluding the component track to be divided;

[0182] The track grouping subunit 1226 is configured to divide candidate component tracks having the same component track type and associated with the same atlas identifier into M fragment track groups.

[0183] The track packaging unit 123 includes:

[0184] The index obtaining subunit 1231 is used to obtain the track index i of the component track type information corresponding to the i-th fragment track group among the M fragment track groups; i is a positive integer less than or equal to M;

[0185] The index association subunit 1232 is configured to add a track index i to the i-th fragment track group, and associate the i-th fragment track group with the atlas fragments corresponding to the component tracks included in the i-th fragment track group based on the track index i;

[0186] The file generation subunit 1233 is configured to encapsulate the M fragment track groups and the atlas fragments associated with each fragment track group according to the association result to obtain a target file.

[0187] An embodiment of the present application provides a data processing device, which can obtain k component tracks and the atlas slices corresponding to each component track; the component track includes a component track data box, and the component track data box includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer; the k component tracks and the atlas slices corresponding to each component track are encapsulated according to the component track data box to obtain a target file. Through the above process, a component track data box can be encapsulated in each component track, and the component track data box indicates the relevant information of the component track in which it is located. The association between the atlas slice information and the component track (i.e., the atlas slice and the component track) can be refined to the granularity of a single component track, so that the information of the component track can be retained as comprehensively as possible, and the information of the component track in the encapsulated target file is also more organized and refined. Furthermore, after the target file is sent to the user device, the user device can accurately select the required component tracks to achieve the purpose of partial decapsulation and partial decoding, thereby eliminating the need to fully decapsulate and decode the target file, saving time and resources consumed in data processing, and thereby improving the accuracy and efficiency of data processing.

[0188] See also Figure 9 , Figure 9 Schematic diagram of another data processing device provided in an embodiment of the present application. The above-mentioned data processing device can be a computer program (including program code) running on a computer device, for example, the data processing device is an application software; the device can be used to execute the corresponding steps of the method provided in an embodiment of the present application. Figure 9 As shown, the data processing device 900 can be used for the above Figure 6 The computer device in the corresponding embodiment may specifically include: a track acquisition module 21 and a data decapsulation module 22 .

[0189] Track acquisition module 21 is used to acquire k component tracks in the target file; the component track includes a component track data box, which includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer;

[0190] The data decapsulation module 22 is used to decapsulate the target file according to the component track data box to obtain the fragmented display content.

[0191] The data decapsulation module 22 includes:

[0192] The quantity obtaining unit 221 is configured to, when responding to a display request for a target component track, obtain a target component track data box included in the target component track from the target file, and obtain the target number of shards corresponding to the target component track from the target component track data box;

[0193] The decapsulation iteration unit 222 is configured to iteratively decapsulate the target file h times based on the target number of fragments to obtain fragment display content corresponding to the target component track; h is the target number of fragments.

[0194] The data decapsulation module 22 includes:

[0195] The identification obtaining unit 223 is configured to obtain the to-be-processed atlas fragment identification of the to-be-processed atlas fragment when responding to a display request for the to-be-processed atlas fragment;

[0196] The data box search unit 224 is configured to obtain component track data boxes respectively included in the k component tracks from the target file, and traverse and search the component track data boxes respectively included in the k component tracks based on the number of slices in the component track data box and the slice identifier of the atlas to be processed;

[0197] The track selection unit 225 is configured to determine the component track to which the component track data box with the to-be-processed atlas fragment identifier belongs as the to-be-processed component track;

[0198] The track decapsulation unit 226 is used to decapsulate the component track to be processed to obtain the fragment display content corresponding to the atlas fragment to be processed.

[0199] The data decapsulation module 22 includes:

[0200] A file decapsulation unit 227 is configured to decapsulate the target file according to the component track data box to obtain k component tracks and an atlas slice corresponding to each component track;

[0201] a space determination unit 228 for determining the spatial regions in which the k component tracks are located in the volumetric video based on the spatial region information in the component track data boxes respectively included in the k component tracks;

[0202] The content generation unit 229 is configured to generate slice display content according to the spatial regions where the k component tracks are located in the volumetric video and the atlas slice corresponding to each component track.

[0203] The embodiment of the present application provides a data processing device, which can obtain k component tracks in a target file; the component track includes a component track data box, the component track data box includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer; the target file is decapsulated according to the component track data box to obtain the slice display content. Through the above process, a component track data box can be encapsulated in each component track, and the component track data box indicates the relevant information of the component track where it is located. The association relationship between atlas slice information and component track (i.e., atlas slice and component track) can be refined to the granularity of a single component track. After receiving the target file, the user device can obtain the information of each component track based on the component track data box of each component track at the granularity of a single component track. The user device can decapsulate and decode the required part of the target file based on needs, which can save time and resources consumed by data processing, thereby improving the accuracy and efficiency of data processing.

[0204] See also Figure 10 , Figure 10 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 10As shown, the computer device in the embodiment of the present application may include: one or more processors 1001, memory 1002, and input / output interface 1003. The processor 1001, memory 1002, and input / output interface 1003 are connected via a bus 1004. The memory 1002 is used to store computer programs, which include program instructions. The input / output interface 1003 is used to receive and output data to implement data exchange between the computer device and the user device; the processor 1001 is used to execute the program instructions stored in the memory 1002.

[0205] When the processor 1001 is located in a computer device, it performs the following operations:

[0206] Get k component tracks and the atlas slices corresponding to each component track; the component track includes a component track data box, which includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer;

[0207] According to the component track data box, the k component tracks and the atlas slices corresponding to each component track are encapsulated to obtain the target file.

[0208] Alternatively, when the processor 1001 is located in the user equipment, it performs the following operations:

[0209] Get k component tracks in the target file; the component track includes a component track data box, which includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer;

[0210] Decapsulate the target file according to the component track data box to obtain the fragmented display content.

[0211] In some feasible implementations, the processor 1001 may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0212] The memory 1002 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1001 and the input / output interface 1003. A portion of the memory 1002 may also include a non-volatile random access memory. For example, the memory 1002 may also store device type information.

[0213] In a specific implementation, the above computer device can execute the above-mentioned functions through its built-in functional modules. Figure 4 or Figure 6 For details on the implementation methods provided in each step, please refer to the above Figure 4 or Figure 6 The implementation methods provided in each step are not repeated here.

[0214] The embodiment of the present application provides a computer device, including: a processor, an input and output interface, and a memory, wherein the processor obtains computer instructions in the memory and executes the above Figure 4 or Figure 6 The various steps of the method shown in are used to perform data processing operations. The embodiment of the present application can encapsulate a component track data box in each component track, and the component track data box indicates the relevant information of the component track where it is located, and can refine the association between the atlas slice information and the component track (i.e., the atlas slice and the component track) to the granularity of a single component track, so that the information of the component track can be retained as comprehensively as possible, and the information of the component track in the encapsulated target file is also more organized and refined. Furthermore, after the target file is sent to the user device, the user device can accurately select the required component track to achieve the purpose of partial decapsulation and partial decoding, thereby eliminating the need to fully decapsulate and decode the target file, saving time and resources consumed in data processing, and thereby improving the accuracy and efficiency of data processing.

[0215] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, which includes program instructions. When the program instructions are executed by the above-mentioned processor, Figure 4 or Figure 6 The data processing methods provided in each step are detailed in the above Figure 4 or Figure 6 The implementation methods provided by each step will not be described in detail here. In addition, the description of the beneficial effects of adopting the same method will not be described in detail. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the program instructions can be deployed to be executed on one computer device, or on multiple computer devices located in one place, or on multiple computer devices distributed in multiple places and interconnected by a communication network.

[0216] The computer-readable storage medium may be the data processing device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Furthermore, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0217] The term "comprising" and any variations thereof in the description, claims, and drawings of the embodiments of this application are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or modules is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other steps and modules inherent to the process, method, apparatus, product, or device.

[0218] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0219] The methods and related devices provided by the embodiments of the present application are described with reference to the method flow charts and / or structural diagrams provided by the embodiments of the present application. Specifically, each process and / or block in the method flow charts and / or structural diagrams, as well as the combination of processes and / or blocks in the flow charts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 The flow or flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0220] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Obtain k component tracks and atlas slices corresponding to each component track; the component track includes a component track data box, which includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer; the component track data box includes a track group data box; the track group data box includes an atlas identifier and an atlas slice identifier, and the atlas identifier indicates the identifier of the atlas of the volumetric video to which the atlas slice belongs; The k component tracks and the atlas slice corresponding to each component track are encapsulated according to the component track data box to obtain a target file.

2. The method according to claim 1, wherein When the atlas slice identifier is associated with the atlas identifier, the atlas slice corresponding to the atlas slice identifier belongs to the atlas of the volumetric video corresponding to the atlas identifier.

3. The method according to claim 1, wherein The atlas slices corresponding to each component track include a first atlas slice and a second atlas slice; When the atlas identifier associated with the first atlas fragment identifier of the first atlas fragment is the same as the atlas identifier associated with the second atlas fragment identifier of the second atlas fragment, the first atlas fragment and the second atlas fragment belong to the same fragment track group.

4. The method according to claim 1, wherein The component track further includes a component track type; the k component tracks include a first component track and a second component track; When the first component track type of the first component track is the same as the second component track type of the second component track, the atlas fragments corresponding to the first component track and the atlas fragments corresponding to the second component track belong to the same fragment track group.

5. The method according to claim 1, wherein The component track data box includes a component information data box, the component information data box includes a data box type, and the data box type indicates information indicating the component track to which the component information data box belongs; the k component tracks include a target component track; When the component information data box is encapsulated in the target component track, the component information data box indicates the atlas slice and spatial information corresponding to the target component track.

6. The method according to any one of claims 1 to 5, wherein The step of encapsulating the k component tracks and the atlas slices corresponding to each component track according to the component track data box to obtain a target file includes: Acquire track group division information, and acquire component track information of the k component tracks from component track data boxes respectively included in the k component tracks based on the track group division information; the track group division information indicates a method for grouping the k component tracks; Divide the k component tracks based on the track group division information and the component track information to obtain M shard track groups; M is a positive integer; The M fragment track groups and the atlas fragments associated with each fragment track group are encapsulated to obtain a target file.

7. The method according to claim 6, wherein The track group division information includes atlas division information, and the component track information of the k component tracks includes atlas identifiers associated with atlas slices corresponding to the k component tracks; the atlas identifier represents an identifier of an atlas of the volumetric video to which the atlas slice belongs; The k component tracks are divided based on the track group division information and the component track information to obtain M shard track groups, including: Among the k component tracks, based on the atlas partitioning information, the component tracks corresponding to atlas slices associated with the same atlas identifier are grouped to obtain M slice track groups.

8. The method according to claim 6, wherein The track group division information includes atlas component division information, and the component track information of the k component tracks includes atlas identifiers associated with the atlas slices corresponding to the k component tracks, and component track types corresponding to the k component tracks; the atlas identifier represents an identifier of the atlas of the volumetric video to which the atlas slice belongs; The k component tracks are divided based on the track group division information and the component track information to obtain M shard track groups, including: Among the k component tracks, component tracks having the same component track type and associated with the same atlas identifier are grouped based on the atlas component partitioning information to obtain M fragment track groups.

9. The method according to claim 6, wherein The k component tracks are divided based on the track group division information and the component track information to obtain M shard track groups, including: Determining, based on the component track information, component track types corresponding to the k component tracks, and atlas identifiers to which the atlas slices corresponding to the k component tracks belong; the atlas identifier represents an identifier of an atlas of the volumetric video to which the atlas slices belong; If there is a component track to be divided among the k component tracks, then based on the atlas identifier to which the atlas fragment corresponding to the component track to be divided belongs, the component track to be divided is split into at least two component sub-tracks; the component track to be divided is a component track corresponding to at least two atlas fragments, and the atlas identifiers corresponding to the at least two atlas fragments are different; the atlas fragments corresponding to the component sub-tracks are associated with the same atlas identifier; Determine the at least two component sub-tracks and the regular component track as candidate component tracks; the regular component track refers to the component tracks among the k component tracks except the component track to be divided; The candidate component tracks with the same component track type and associated with the same atlas identifier are divided to obtain M fragment track groups.

10. The method according to claim 6, wherein The encapsulating the M fragment track groups and the atlas fragments associated with each fragment track group to obtain a target file includes: Obtain the track index i of the component track type information corresponding to the i-th fragment track group in the M fragment track groups; i is a positive integer less than or equal to M; Add the track index i to the i-th shard track group, and associate the i-th shard track group with the atlas shards corresponding to the component tracks included in the i-th shard track group based on the track index i; The M fragment track groups and the atlas fragments associated with each fragment track group are encapsulated according to the association result to obtain a target file.

11. A data processing method, characterized in that: The method comprises: Obtain k component tracks in a target file; the component track includes a component track data box, which includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer; the component track data box includes a track group data box; the track group data box includes an atlas identifier and an atlas slice identifier, the atlas identifier indicating the identifier of the atlas of the volumetric video to which the atlas slice belongs; The target file is decapsulated according to the component track data box to obtain sliced ​​display content.

12. The method according to claim 11, wherein The step of decapsulating the target file according to the component track data box to obtain fragmented display content includes: When responding to a display request for a target component track, obtaining a target component track data box included in the target component track from a target file, and obtaining a target number of shards corresponding to the target component track from the target component track data box; The target file is iteratively decapsulated h times based on the target number of fragments to obtain fragment display content corresponding to the target component track; h is the target number of fragments.

13. The method according to claim 11, wherein The step of decapsulating the target file according to the component track data box to obtain fragmented display content includes: When responding to a display request for a to-be-processed atlas fragment, obtaining an to-be-processed atlas fragment identifier of the to-be-processed atlas fragment; Obtaining component track data boxes respectively included in the k component tracks from the target file, and traversing and searching the component track data boxes respectively included in the k component tracks based on the number of shards in the component track data boxes and the shard identifier of the atlas to be processed; Determine the component track to which the component track data box identifying the atlas slice to be processed belongs as the component track to be processed; The component track to be processed is decapsulated to obtain the fragment display content corresponding to the atlas fragment to be processed.

14. The method according to claim 11, wherein The step of decapsulating the target file according to the component track data box to obtain fragmented display content includes: Decapsulate the target file according to the component track data box to obtain the k component tracks and the atlas slice corresponding to each component track; Determining, based on spatial region information in component track data boxes respectively included in the k component tracks, spatial regions where the k component tracks are located in the volumetric video; Slice display content is generated according to the spatial regions where the k component tracks are located in the volumetric video and the atlas slice corresponding to each component track.

15. A data processing device, characterized in that: The device comprises: A data acquisition module is configured to acquire k component tracks and atlas slices corresponding to each component track; the component track includes a component track data box, which includes the number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer; the component track data box includes a track group data box; the track group data box includes an atlas identifier and an atlas slice identifier, the atlas identifier indicating the identifier of the atlas of the volumetric video to which the atlas slice belongs; A data encapsulation module is used to encapsulate the k component tracks and the atlas slices corresponding to each component track according to the component track data box to obtain a target file.

16. A data processing device, characterized in that: The device comprises: A track acquisition module is configured to acquire k component tracks from a target file; the component track includes a component track data box, which includes a number of slices and spatial region information; the number of slices refers to the number of atlas slices corresponding to the component track to which the component track data box belongs; the spatial region information refers to the spatial region of the component track to which the component track data box belongs; k is a positive integer; the component track data box includes a track group data box; the track group data box includes an atlas identifier and an atlas slice identifier, the atlas identifier indicating the identifier of the atlas of the volumetric video to which the atlas slice belongs; The data decapsulation module is used to decapsulate the target file according to the component track data box to obtain the fragmented display content.

17. A computer device, characterized in that: Includes processor, memory, input and output interfaces; The processor is connected to the memory and the input / output interface, respectively, wherein the input / output interface is used to receive and output data, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method described in any one of claims 1 to 10, or executes the method described in any one of claims 11 to 14.

18. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 10, or executes the method according to any one of claims 11 to 14.

Images