Volume media data processing method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a data processing method and device for volume media, equipment and a storage medium, and the method comprises the steps: obtaining replacement relation indication information and replacement attribute information between every two playing groups in at least two playing groups of the volume media; the replacement relationship indication information is used for indicating a media track pair with a replacement relationship in every two playing groups, and the replacement attribute information is used for reflecting an attribute difference between media tracks in the media track pair with the replacement relationship; and sending the at least two playing groups, the replacement relationship indication information and the replacement attribute information to a playing terminal. According to the invention, processing resources of the playing terminal can be saved.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to the field of point cloud technology, and particularly to a data processing method, apparatus, device, and storage medium for volumetric media. Background Art

[0002] With the development of three-dimensional scene capture and rendering technologies, it has promoted the innovation of three-dimensional volumetric media featuring high degrees of freedom and true three dimensions. For example, point cloud technology, as a typical representative of volumetric media content and services, has been widely applied. Among them, a point cloud is a set of discrete points that are irregularly distributed in space and represent the spatial structure and surface attributes of a three-dimensional object or scene. Each point in the point cloud has at least three-dimensional position information, and depending on the application scenario, it may also have color, material, or other information. In addition, as another application of volumetric media, multi-view video and free viewpoint use real or virtual cameras to capture 3D scene information, supporting the presentation of a three-dimensional 3D scene with six degrees of freedom (6DoF) within a limited viewing position and direction range.

[0003] Volumetric media usually includes multiple media tracks, such as video tracks, audio tracks, and text tracks, etc. In practice, it is found that when there is a certain association relationship between the media tracks of volumetric media, the playback terminal only needs to perform a decoding operation on some media tracks of the volumetric media to reconstruct and present the volumetric media. For example, when the volumetric media includes two video tracks with a replacement relationship, the playback terminal only needs to perform a decoding operation on one of the video tracks. However, since the playback terminal cannot recognize the association relationship between these media tracks, the playback terminal needs to perform a decoding operation on all media tracks of the volumetric media, resulting in waste of processing resources of the playback terminal. Summary of the Invention

[0004] The technical problem to be solved by the embodiments of this application is to provide a data processing method, apparatus, device, and storage medium for volumetric media, which can save the processing resources of the playback terminal.

[0005] On the one hand, an embodiment of this application provides a data processing method for volumetric media, including:

[0006] Obtain the replacement relationship indication information and replacement attribute information between every two of at least two playback groups of the volumetric media; send the at least two playback groups, the replacement relationship indication information, and the replacement attribute information to a playback terminal.

[0007] On the one hand, an embodiment of this application provides a data processing apparatus for volumetric media, including:

[0008] An obtaining module, configured to obtain the replacement relationship indication information and replacement attribute information between every two of at least two playback groups of the volumetric media;

[0009] A sending module, configured to send the at least two playback groups, the replacement relationship indication information, and the replacement attribute information to a playback terminal.

[0010] On the one hand, this application provides a computer device, including: a processor and a memory;

[0011] Wherein, the memory is used to store a computer program, the processor is used to call the computer program, and when the computer program is executed by the processor, the steps of the above method are implemented.

[0012] On the one hand, an embodiment of this application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the steps of the above method are implemented.

[0013] In this application, a service device can mark media track pairs with a replacement relationship in every two playback groups of volumetric media to obtain replacement relationship indication information between every two playback groups; and obtain attribute information of the media tracks in the media track pairs, and perform difference identification on the attribute information of the media tracks in the media track pairs to obtain replacement attribute information between every two playback groups. Send at least two playback groups, the replacement relationship indication information, and the replacement attribute information to the playback terminal. It can be seen that by sending the replacement relationship indication information and the replacement attribute information to the playback terminal, it is beneficial for the playback terminal to adaptively select a playback group for decoding processing according to its own processing ability or user requirements, so as to reconstruct and present volumetric media, and personalized presentation can be realized. In addition, it is not necessary to decode all media tracks, which can save the processing resources of the playback terminal. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of this application, and those of ordinary skill in the art can also obtain other drawings based on these drawings without creative efforts.

[0015] Figure 1 Shows an architecture diagram of a volumetric media system provided by this application;

[0016] Figure 2 Shows a schematic diagram of 6DoF provided by this application;

[0017] Figure 3 Shows a schematic diagram of 3DoF provided by an embodiment of this application;

[0018] Figure 4 Shows a schematic diagram of a 3DoF+ provided by this application;

[0019] Figure 5 Shows a schematic diagram of a media track of a volumetric medium provided by this application;

[0020] Figure 6 Is a flowchart of a data processing method for a volumetric medium provided by this application;

[0021] Figure 7 Shows a schematic diagram of a media track of a volumetric medium provided by this application;

[0022] Figure 8 Shows a schematic diagram of a media track of a volumetric medium provided by this application;

[0023] Figure 9 Is a flowchart of a data processing method for a volumetric medium provided by this application;

[0024] Figure 10 Is a flowchart of a data processing method for a volumetric medium provided by this application;

[0025] Figure 11 Is a schematic diagram of a scenario for obtaining replacement relationship indication information and replacement attribute information provided by this application;

[0026] Figure 12 Is a schematic diagram of a scenario for obtaining replacement relationship indication information and replacement attribute information provided by this application;

[0027] Figure 13 Is a schematic diagram of the structure of a data processing device for a volumetric medium provided by an embodiment of this application;

[0028] Figure 14 Is a schematic diagram of the structure of a data processing device for a volumetric medium provided by an embodiment of this application;

[0029] Figure 15 Is a schematic diagram of the structure of a data processing device for a volumetric medium provided by an embodiment of this application;

[0030] Figure 16 Is a schematic diagram of the structure of a computer device provided by an embodiment of this application. Detailed implementation

[0031] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application.

[0032] The embodiments of the present application relate to data processing technologies for volumetric media. The so-called volumetric media (visual volumetric video-based coding media) refers to immersive media that captures visual content from three-dimensional space and provides a 3DoF+ or 6DoF viewing experience, is encoded in traditional video format, and includes a volumetric video type track in the file encapsulation, including multi-view video, video-encoded point clouds, etc.

[0033] Figure 1 The architecture diagram of a volumetric media system provided by an exemplary embodiment of the present application is shown; as Figure 1 shown, the volumetric media system includes a service device and a playback terminal. The service device may refer to the computer device used by the provider of the volumetric media, and this computer device may be a terminal (such as a PC (Personal Computer), a smart mobile device (such as a smart phone), etc.) or a server. Among them, the server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. The playback terminal may refer to the computer device used by the user of the volumetric media, and this computer device may be a terminal (such as a PC (Personal Computer), a smart mobile device (such as a smart phone), a VR device (such as a VR helmet, VR glasses, etc.)). The data processing process of the volumetric media includes the data processing process on the service device side and the data processing process on the playback terminal side.

[0034] The data processing process at the service device end mainly includes: (1) the process of acquiring and producing the media content of volumetric media; (2) the process of encoding and file encapsulation of volumetric media. The data processing process at the playback terminal end mainly includes: (3) the process of file decapsulation and decoding of volumetric media; (4) the rendering process of volumetric media. In addition, there is a transmission process of volumetric media between the service device and the playback terminal, and this transmission process can be carried out based on various transmission protocols, and the transmission protocols here may include but are not limited to: DASH (Dynamic Adaptive Streaming over HTTP) protocol, MMT (Moving Picture Experts Group Media Transport), HLS (HTTP Live Streaming) protocol, SMTP (Smart Media Transport Protocaol), TCP (Transmission Control Protocol), etc.

[0035] The following will be combined with Figure 1 to introduce each process involved in the data processing process of volumetric media in detail.

[0036] I. The data processing process at the service device end:

[0037] (1) The process of acquiring and producing the media content of volumetric media.

[0038] 1) The process of acquiring the media content of volumetric media.

[0039] The media content of volumetric media is obtained by a capture device collecting the real-world audio-visual scene. In one implementation, the capture device may refer to a hardware component provided in the service device. For example, the capture device refers to the microphone, camera, sensor, etc. of the terminal. In another implementation, the capture device may also be a hardware device connected to the service device, such as a camera connected to the service device, which is used to provide the acquisition service of the media content of volumetric media for the service device. The capture device may include but is not limited to: audio devices, imaging devices, and sensing devices. Among them, the audio devices may include audio sensors, microphones, etc. The imaging devices may include ordinary cameras, stereo cameras, light field cameras, etc. The sensing devices may include laser devices, radar devices, etc. The number of capture devices may be multiple, such as a camera array, and these capture devices are deployed at some specific positions in the real space to simultaneously capture the audio-visual content at different angles in this space, and the audio content and video content in this audio-visual content are synchronized both in time and space.

[0040] 2) The production process of the media content of volumetric media.

[0041] The captured audio content itself is the content suitable for audio encoding of volumetric media. After a series of production processes are performed on the captured video content, it can become the content suitable for video encoding of volumetric media. The production processes include:

[0042] ① Stitching. Since the captured video content is shot by the capture device from different angles, stitching refers to stitching the video content shot from these various angles into a complete video that can reflect the 360-degree visual panorama of the real space, that is, the stitched video is a panoramic video (or spherical video) represented in three-dimensional space.

[0043] ② Projection. Projection refers to the process of mapping a three-dimensional video formed by stitching onto a two-dimensional (3-Dimension, 2D) image. The 2D image formed by projection is called a projection image; the projection methods can include but are not limited to: longitude and latitude map projection, regular hexahedron projection.

[0044] ③ Region encapsulation. The projection image can be directly encoded, or it can be encoded after region encapsulation. It is found in practice that in the data processing of volumetric media, encoding after region encapsulation of the two-dimensional projection image can greatly improve the video encoding efficiency of volumetric media. Therefore, the region encapsulation technology is widely applied to the video processing of volumetric media. The so-called region encapsulation refers to the process of performing transformation processing on the projection image by region. The region encapsulation process converts the projection image into an encapsulated image. The specific process of region encapsulation includes: dividing the projection image into multiple mapping regions, then performing transformation processing on the multiple mapping regions respectively to obtain multiple encapsulated regions, and mapping the multiple encapsulated regions into a 2D image to obtain the encapsulated image. Among them, the mapping region refers to the region obtained by division in the projection image before region encapsulation; the encapsulated region refers to the region located in the encapsulated image after region encapsulation. The transformation processing can include but are not limited to: mirroring, rotation, rearrangement, upsampling, downsampling, changing the resolution of the region, and moving, etc.

[0045] It should be noted that since the capture device can only capture panoramic videos, after such videos are processed by the service device and transmitted to the playback terminal for corresponding data processing, users on the playback terminal side can only watch 360-degree video information by performing some specific actions (such as head rotation), while performing non-specific actions (such as moving the head) cannot obtain corresponding video changes, resulting in a poor VR experience. Therefore, it is necessary to additionally provide depth information matching the panoramic video to enable users to obtain a better immersion and a better VR experience, which involves 6DoF (Six Degrees of Freedom) production technology. When users can move relatively freely in the simulated scene, it is called 6DoF. When using 6DoF production technology to produce video content of volumetric media, the capture device generally selects light field cameras, laser devices, radar devices, etc. to capture point cloud data or light field data in space, and some specific processes are also required during the execution of the above production processes ①-③, such as the processes of cutting and mapping point cloud data, the calculation process of depth information, etc. Figure 2 FIG. shows a schematic diagram of 6DoF provided by an exemplary embodiment of the present application; 6DoF is divided into window 6DoF, omnidirectional 6DoF, and 6DoF. Among them, window 6DoF means that the user's rotational movement along the X-axis and Y-axis is restricted, and the translational movement along the Z-axis is restricted; for example, the user cannot see the scene outside the window frame, and the user cannot pass through the window. Omnidirectional 6DoF means that the user's rotational movement along the X-axis, Y-axis, and Z-axis is restricted. For example, the user cannot freely pass through the three-dimensional 360-degree VR content in the restricted movement area. 6DoF means that the user can freely translate along the X-axis, Y-axis, and Z-axis. For example, the user can freely walk in the three-dimensional 360-degree VR content. Similar to 6DoF, there are also 3DoF and 3DoF+ production technologies. Figure 3 FIG. shows a schematic diagram of 3DoF provided by an exemplary embodiment of the present application; as Figure 3 shown, 3DoF means that the center point of the user in a three-dimensional space is fixed, and the user's head rotates along the X-axis, Y-axis, and Z-axis to view the picture provided by the media content. Figure 4 FIG. shows a schematic diagram of 3DoF+ provided by an exemplary embodiment of the present application, as Figure 4 shown, 3DoF+ means that when the virtual scene provided by the volumetric media has certain depth information, the user's head can move within a limited space based on 3DoF to view the picture provided by the media content.

[0046] (2) The process of encoding and file encapsulation of volumetric media.

[0047] The captured audio content can be directly subjected to audio encoding to form the audio track of the volumetric media. After the above production processes ①-② or ①-③, the video content is subjected to video encoding to obtain the video track of the volumetric media. For example, encoding the projection graphics results in a volumetric video track carrying the 2D and 3D mapping relationships; encoding the encapsulated images results in a volumetric video track carrying the encapsulated data. A video stream carrying attribute data and a video stream carrying position data are obtained from the panoramic video represented in three-dimensional space; the video stream carrying attribute data is encoded to obtain a volumetric video track carrying attribute data; the attribute data refers to attributes such as the color information and material of the shooting object. The video stream carrying position data is encoded to obtain a volumetric video track carrying position data; the position data refers to the three-dimensional position information of the shooting object.

[0048] It should be noted here that the above video encoding methods may include but are not limited to: video encoding includes AVC (Advanced Video Coding), HVC (High Video Coding), AVS (Audio Video Coding Standard, the national video coding standard of China), HEVC / H.265 (High Efficiency Video Coding, the international video coding standard), VVC / H.266 (Versatile Video Coding, the international video coding standard). To adapt to different application scenarios, the service device can use different video encoding methods to encode the video content, so as to obtain a pair of media tracks with a replacement relationship. The pair of media tracks with a replacement relationship is obtained by using different video encoding methods for the same video content, that is, it can be said that the track categories of the volumetric video tracks in the pair of media tracks with a replacement relationship are the same. Usually, track identifiers are used to distinguish different volumetric video tracks, and replacement identifiers are used to identify the volumetric video tracks with a replacement relationship.

[0049] For example, as Figure 5 shown, Figure 5Schematic diagram of the replacement relationship between volumetric video tracks in volumetric media V3C. The volumetric media includes 8 volumetric video tracks, and the track identifiers of each volumetric video track are 1-8 respectively. Volumetric video track 1 is the volumetric video track carrying the 2D and 3D mapping relationship. The track categories of volumetric video track 2 and volumetric video track 5 are both volumetric video tracks carrying attribute data, and the coding methods of volumetric video track 2 and volumetric video track 5 are avc1 (i.e., Advanced Video Coding) and hvc1 (i.e., High-Definition Video Coding) respectively. Therefore, there is a replacement relationship between volumetric video track 2 and volumetric video track 5, and the replacement identifier of volumetric video track 2 and volumetric video track 5 is 10, that is, volumetric video track 2 and volumetric video track 5 form media track pair 10. The track categories of volumetric video track 3 and volumetric video track 6 are both volumetric video tracks carrying position data, and the coding methods of volumetric video track 3 and volumetric video track 6 are avc1 and hvc1 respectively. Therefore, there is a replacement relationship between volumetric video track 3 and volumetric video track 6, and the replacement identifier of volumetric video track 3 and volumetric video track 6 is 11, that is, volumetric video track 3 and volumetric video track 6 can form media track pair 11. The track categories of volumetric video track 4, volumetric video track 7, and volumetric video track 8 are all volumetric video tracks carrying placeholder map data, and the coding methods of volumetric video track 4, volumetric video track 7, and volumetric video track 8 are avc1, hvc1, and vvc1 respectively. Therefore, there is a replacement relationship between any two of volumetric video track 4, volumetric video track 7, and volumetric video track 8, and the replacement identifier of volumetric video track 4, volumetric video track 7, and volumetric video track 8 is 12. Any two of volumetric video track 4, volumetric video track 7, and volumetric video track 8 form media track pair 12.

[0050] If the playback terminal decodes these volumetric video tracks to reconstruct and present the volumetric media, it is likely to cause waste of the processing resources of the playback terminal. Based on this, in order to enable the playback terminal to select one volumetric video track from the media pairs with replacement relationships for decoding, the volumetric video tracks can be grouped to obtain multiple playback groups. There is a joint playback relationship between the volumetric video tracks within a playback group, that is, the volumetric video tracks within a playback group can form a complete volumetric media (i.e., volumetric video), and only one volumetric video track from the media pairs with replacement relationships is included in the playback group. Figure 5 in which each volumetric track can form multiple playback groups, such as Figure 5 in, volumetric video tracks 1-4 can form playback group 1, volumetric video track 1, volumetric video tracks 5-7 can form playback group 2, and volumetric video track 1, volumetric video tracks 5-6, volumetric video track 8 can form playback group 3. Of course, other combinations of volumetric video tracks can also form playback groups, which will not be elaborated here.

[0051] It should be noted that audio coding includes but is not limited to: MP3 (Moving Picture Experts Group Audio Layer III), AAC (Advanced Audio Coding), and FLAC (Free Lossless Audio Codec). The service device can generate audio track pairs with replacement relationships using different audio coding methods, add an audio track (one of the audio track pairs with replacement relationships) to each playback group, and generate replacement relationship indication information and replacement attribute information for every two playback groups. The replacement relationship indication information is used to indicate the media track pairs with replacement relationships in every two playback groups, and the replacement attribute information is used to reflect the attribute differences between the media tracks (such as volumetric video tracks or audio tracks) in the media track pairs with replacement relationships. Here, the attribute differences can include the coding methods of the media tracks in the media track pairs, etc. Further, the playback groups, replacement relationship indication information, and replacement attribute information can be encapsulated in a file container according to the file format of the volumetric media (such as ISOBMFF (ISO Base Media File Format)) to form a media file resource of the volumetric media. This media file resource can be a media file or a media segment that forms a media file of the volumetric media. And according to the requirements of the file format of the volumetric media, Media Presentation Description (MPD) is used to record the metadata of the media file resource of the volumetric media. Here, the metadata is a general term for information related to the presentation of the volumetric media, and the metadata can include description information of the media content, description information of the viewport, and signaling information related to the presentation of the media content, etc. As Figure 1 shown, the service device will store the media presentation description information and media file resources formed after the data processing process.

[0052] II. Data processing process at the playback terminal:

[0053] (3) The process of file de-encapsulation and decoding of the volumetric media;

[0054] The playback terminal can obtain the media file resources of volumetric media and the corresponding media presentation description information from the service device through the recommendation of the service device or adaptively and dynamically according to the processing capabilities of the playback terminal. For example, the playback terminal can request and obtain the corresponding media file resources from the service device according to the decoding capabilities of the playback terminal. The media file resources and the media presentation description information are transmitted from the service device to the playback terminal through a transmission mechanism (such as DASH, SMT). The process of file de-encapsulation at the playback terminal is the reverse of the file encapsulation process at the service device. The playback terminal de-encapsulates the media file resources according to the file format requirements of volumetric media to obtain the audio track and the volumetric video track. The decoding process at the playback terminal is the reverse of the encoding process at the service device. The playback terminal decodes the audio track to restore the audio content. In addition, the playback terminal decodes the volumetric video track to obtain the video content.

[0055] (4) The rendering process of volumetric media.

[0056] The playback terminal renders the audio content obtained by audio decoding and the 3D image obtained by video decoding according to the metadata related to rendering and window in the media presentation description information. After the rendering is completed, the playback output of the 3D image is realized. In particular, if the production technologies of 3DoF and 3DoF+ are adopted, the playback terminal mainly renders the 3D image based on the current viewpoint, parallax, depth information, etc. If the production technology of 6DoF is adopted, the playback terminal mainly renders the 3D image within the window based on the current viewpoint. Among them, the viewpoint refers to the user's viewing position point, the parallax refers to the line-of-sight difference generated by the user's binoculars or the line-of-sight difference generated by movement, and the window refers to the viewing area.

[0057] The volumetric media system supports data boxes (Boxes). A data box refers to a data block or object obtained by encapsulating the playback groups of volumetric media, the indication information of the replacement relationship between every two playback groups, the replacement attribute information, and the metadata.

[0058] In summary, in this application, by sending the replacement relationship indication information and the replacement attribute information to the playback terminal, it is beneficial for the playback terminal to adaptively select a playback group for decoding processing according to its own processing capabilities or user needs to reconstruct and present volumetric media, and personalized presentation can be realized. In addition, it is not necessary to decode all media tracks, which can save the processing resources of the playback terminal.

[0059] Further, please refer to Figure 6 , which is a schematic flowchart of a data processing method for volumetric media provided by an embodiment of this application. As Figure 6 shown, this method can be executed by the application service device in Figure 1 , and this method can at least include the following S101 - S102:

[0060] S101. Obtain the replacement relationship indication information and replacement attribute information between every two of at least two playback groups of volumetric media; the replacement relationship indication information is used to indicate the media track pairs with a replacement relationship in every two of the playback groups, and the replacement attribute information is used to reflect the attribute differences between the media tracks in the media track pairs with a replacement relationship.

[0061] In this application, the media tracks in a media track pair may refer to volumetric video tracks, audio tracks, text tracks, etc.; between every two playback groups of volumetric media, there is included one media track pair or at least two media track pairs with a replacement relationship. The attribute differences between the media tracks in a media track pair can be used to reflect the quality differences between the playback groups, and the quality differences include playback quality differences. The attribute differences between the media tracks in a media track pair may include the attribute information of each media track in the media track pair, and the media track attribute information includes one or more of the encoding method, transmission rate, quality level, and encoding type of the encoding method. The encoding method may include a video encoding method and an audio encoding method, and the encoding type of the encoding method may include a lossy encoding method, a lossless encoding method, etc.

[0062] Among them, the attribute difference reflects the playback quality difference between two playback groups with a replacement relationship.

[0063] It should be noted that the attribute information of a media track can be used to reflect the playback quality of the media track, and further, it can be used to reflect the playback quality of the playback group to which the media track belongs. For example, the playback quality of a media track with a lossless encoding type of the encoding method is better than that of a media track with a lossy encoding type of the encoding method. At the same time, the transmission resources (such as network bandwidth) consumed by the channel to transmit the media track and the resources consumed by the playback terminal to process the media track are related to the playback quality of the media track. For example, the higher the quality level of the media track, the more transmission resources consumed by the channel to transmit the media track and the more resources consumed by the playback terminal to process the media track. On the contrary, the lower the quality level of the media track, the fewer transmission resources consumed by the channel to transmit the media track and the fewer resources consumed by the playback terminal to process the media track.

[0064] Optionally, the media track pair includes a first media track belonging to playback group P m and a second media track belonging to playback group P n The playback group P m and the playback group P nThe replacement attribute information between them includes the attribute information of the first media track and the attribute information of the second media track. Both m and n are positive integers less than Z, and Z is the number of groups of the at least two playback groups. The attribute information of the first media track includes the encoding method of the first media track, and the attribute information of the second media track includes the encoding method of the second media track. The encoding method of the first media track is different from the encoding method of the second media track. For example, both the first media track and the second media track are volumetric video tracks. The encoding method of the first media track is avc1, and the encoding method of the second media track is hvc1. By marking the encoding methods of the media tracks, it is beneficial to more precisely know the playback quality differences between each playback group. Furthermore, it is beneficial for the playback terminal to quickly identify the playback group that matches the processing capacity of the playback terminal, or to identify the playback group that matches the user's needs.

[0065] Optionally, the attribute information of the first media track further includes one or more of the transmission rate, structure organization, quality level, and encoding type corresponding to the encoding method of the first media track. That is, the attribute information of the first media track includes one or more of the encoding method, transmission rate, structure organization, quality level, and encoding type corresponding to the encoding method of the first media track. The attribute information of the second media track further includes one or more of the transmission rate, structure organization, quality level, and encoding type corresponding to the encoding method of the second media track. That is, the attribute information of the second media track includes one or more of the encoding method, transmission rate, structure organization, quality level, and encoding type corresponding to the encoding method of the second media track. The transmission rate includes one or both of the frame rate or bit rate of the media track. By marking the attribute information of the media track in multiple dimensions, more differential attributes between the playback groups are provided for the playback terminal, which is beneficial for the playback terminal to more accurately select the playback group that matches itself.

[0066] Optionally, the replacement relationship indication information further includes a replacement level field. When the playback group P m and the playback group P n the replacement level field in the replacement relationship indication information between them is the first value (such as 1), the replacement level field is used to reflect the replacement level between the playback group P m and the playback group P n is at the media content level. When the playback group P m and the playback group P n the replacement level field in the replacement relationship indication information is the second value (such as 0), the replacement level field is used to reflect the replacement level between the playback group P m and the playback group P n is at the track level.

[0067] Optionally, when the replacement level between this playback group P m and this playback group P n is at the media content level, the number of media track pairs with a replacement relationship in this playback group P m and this playback group P n is R, where R is the number of media tracks in this playback group P m , and the number of media tracks in this playback group P m is the same as the number of tracks in this playback group P n ; that is to say, when the replacement level between this playback group P m and this playback group P n is at the media content level, there is a one-to-one replacement relationship between the media tracks of this playback group P m and this playback group P n .

[0068] For example, as Figure 7 shown, volumetric media includes two playback groups, namely playback group 1 and playback group 2. Both playback group 1 and playback group 2 include media tracks carrying location data and media tracks carrying attribute data. The encoding type of the encoding method for each media track in playback group 1 is a lossless encoding type, and the encoding type of the encoding method for each media track in playback group 2 is a lossy encoding type. Therefore, there is a replacement relationship between the media tracks carrying location data in playback group 1 and the media tracks carrying location data in playback group 2, and they can form a media track pair. Similarly, there is a replacement relationship between the media tracks carrying attribute data in playback group 1 and the media tracks carrying attribute data in playback group 2, and they can form a media track pair. It can be seen that the number of tracks in both playback group 1 and playback group 2 is 2, and the number of media track pairs between playback group 1 and playback group 2 is 2. Therefore, it can be said that the replacement level between playback group 1 and playback group 2 is at the media content level.

[0069] Optionally, when the replacement level between this playback group P m and this playback group P n is at the track level, the number of media track pairs with a replacement relationship in this playback group P m and this playback group P n is a positive integer less than R. That is to say, when the replacement level between this playback group P m and this playback group P n is at the track level, there are media tracks in this playback group P m that do not have a replacement relationship with the media tracks in this playback group P n , that is, some media tracks in this playback group P m have a replacement relationship with the media tracks in this playback group P n .

[0070] For example, as Figure 8 shown, the volumetric media includes six volumetric video tracks, namely volumetric video tracks 1-6. Volumetric video track 1 is the volumetric video track carrying the 2D and 3D mapping relationship. The playback groups of the volumetric media can at least include playback group 1 and playback group 2. Playback group 1 can include volumetric video track 1, volumetric video track 2, and volumetric video track 3. Playback group 2 can include volumetric video track 1, volumetric video track 5, and volumetric video track 6. The track types of volumetric video track 2 and volumetric video track 5 are both volumetric video tracks carrying attribute data, and the coding methods of volumetric video track 2 and volumetric video track 5 are avc1 and hvc1 respectively. Therefore, there is a replacement relationship between volumetric video track 2 and volumetric video track 5, and the replacement identifier of volumetric video track 2 and volumetric video track 5 is 10, that is, volumetric video track 2 and volumetric video track 5 form media track pair 10. The track types of volumetric video track 3 and volumetric video track 6 are both volumetric video tracks carrying position data, and the coding methods of volumetric video track 3 and volumetric video track 6 are avc1 and hvc1 respectively. Therefore, there is a replacement relationship between volumetric video track 3 and volumetric video track 6, and the replacement identifier of volumetric video track 3 and volumetric video track 6 is 11, that is, volumetric video track 3 and volumetric video track 6 can form media track pair 11. It can be seen that the number of media track pairs between playback group 1 and playback group 2 is 2, and the number of tracks in both playback group 1 and playback group 2 is 3, that is, the number of media track pairs between playback group 1 and playback group 2 is less than the number of tracks in playback group 1. Therefore, the replacement level between playback group 1 and playback group 2 can be called the track level.

[0071] Optionally, when the replacement level between this playback group P m and this playback group P n is the media content level, the coding methods of the media tracks in this playback group P m are all the first coding method; the coding methods of the media tracks in this playback group P n are all the second coding method. Encoding the media tracks in the playback group using the same coding method is beneficial for the playback terminal to quickly decode the media tracks, quickly obtain the volumetric media, and improve the efficiency of reconstructing the volumetric media.

[0072] Optionally, there is an associated playback relationship between the media tracks within each of the at least two playback groups, and the track categories of the media tracks within each of the playback groups are different; the track categories of the media tracks in each pair of media track pairs with a replacement relationship in each two playback groups are the same. The track category can be used to reflect the content contained in the media track. If the track category includes an audio track, a video track, a text track, etc., further, the track category of the video track can be subdivided according to the data carried by the video track. The track category of the video track includes a volumetric video track carrying a mapping relationship between 2D and 3D, a volumetric video track carrying attribute data, a volumetric video track carrying position data, etc. The media tracks with an associated playback relationship may refer to volumetric media that can be reconstructed based on the media tracks with an associated playback relationship, that is, a complete volumetric media is formed.

[0073] Optionally, the replacement relationship indication information includes replacement identifiers for each pair of media track pairs with a replacement relationship in each two playback groups.

[0074] It should be noted that the service device can obtain the replacement relationship indication information and replacement attribute information between each two of the at least two playback groups of the volumetric media through the data box or transmission protocol of the volumetric media.

[0075] For example, the service device can obtain the data box of the volumetric media; the data box of the volumetric media includes the replacement relationship indication information and replacement attribute information between each two of the playback groups. The data box may also include at least two playback groups of the volumetric media, that is, the data box may refer to an encapsulation of at least two playback groups of the volumetric media, replacement relationship indication information, and replacement attribute information. The syntax of the data box based on the volumetric media can be seen in Table 1 below:

[0076] Table 1

[0077]

[0078]

[0079] Among them, in Table 1, is_content_alternative is the replacement level field between two playout groups. When the value of this field is 1, it indicates that the replacement level between the two playout groups is at the media content level. When the value of this field is 0, it indicates that the replacement level between the two playout groups is at the track level, that is, there is a replacement relationship between some tracks within the two playout groups. alternative_info_flag is the replacement identification field. When the value of this field is 1, the media track in the playout group carries differential information, that is, this media track is one of the media tracks in the media track pair. When the value of this field is 0, it indicates that this media track does not carry differential information, that is, this media track does not belong to one of the media tracks in the media track pair. PlayoutAlternativeInfoBox is the data box field, alternative_type: the differential attribute type of the replaceable track. According to the value of this differential type, this track can have one or more differential attributes. quality_ranking is the quality level field. The smaller the value of this field, the higher the quality of the corresponding track. lossless_flag: When the value of this field is 0, it indicates that the encoding type of the encoding method of the corresponding media track is a lossy encoding type; when the value of this field is 1, it indicates that the encoding type of the encoding method of the corresponding media track is a lossless encoding type. Bitrate is the bitrate information of the media track, indicating the bitrate of the corresponding media track. Framerate is the frame rate information of the media track, indicating the frame rate of the corresponding media track. codec_type is the encoding method of the media track, is_tile_based is the track structure organization field. When the value of this field is 1, it indicates that the media track is organized in a tile structure; when the value of this field is 0, it indicates that the media track is not divided into tiles. When this field exists, the value of is_content_alternative must be 1.

[0080] Optionally, the service device may obtain the transmission protocol of the volumetric media, and the transmission protocol includes the replacement relationship indication information and replacement attribute information between every two playback groups. Specifically, when the transmission protocol is the Dynamic Adaptive Streaming over HTTP (DASH) protocol, the replacement relationship indication information and replacement attribute information between every two playback groups are located in the track description field of the DASH protocol; when the transmission protocol is the Moving Picture Experts Group (MPEG) media transport protocol, the replacement relationship indication information and replacement attribute information between every two playback groups are located within the resource group of the MPEG media transport protocol. By adding the replacement relationship indication information and replacement attribute information to the transmission protocol, it is possible to encapsulate the indication information (i.e., the replacement relationship indication information and replacement attribute information) separately from at least two playback groups. In this way, the transmission information can be preferentially sent to the playback terminal, and the playback terminal can request the playback group that matches it according to the transmission protocol, avoiding the problem of wasting network bandwidth by sending all playback groups to the playback terminal and saving network bandwidth resources. For example, when the transmission protocol is the DASH protocol, the fields added to the DASH protocol regarding the replacement relationship indication information and replacement attribute information are shown in Table 2:

[0081] Table 2

[0082]

[0083]

[0084] Among them, in Table 2, the DASH protocol includes three extension fields, namely: alternative_Id, quality_ranking, lossless_flag. alternative_Id represents the replacement identifier of the media track pair, and the data type is unsigned integer. quality_ranking represents the quality level of the media track. The smaller the value of this field, the higher the quality of the media track, and the data type is unsigned byte. lossless_flag indicates whether the coding type of the encoding method of the media track is lossless encoding. When the value of this field is 0, it indicates that the coding type of the encoding method of the media track is lossy encoding type; when the value is 1, it indicates that the coding type of the encoding method of the media track is lossless encoding type, and the data type is boolean. The "O" corresponding to alternative_Id, quality_ranking, and lossless_flag indicates that these fields are all optional fields.

[0085] For example, when the transmission protocol is the DASH protocol, the fields added to the DASH protocol regarding the replacement relationship indication information and replacement attribute information are shown in Table 3:

[0086] Table 3

[0087]

[0088]

[0089]

[0090] Among them, in Table 3, the alternative_content_flag represents the replacement level field. When the value of this field is 1, it indicates that the corresponding media resource group (i.e., the playback group) and other media resource groups form an alternative relationship, and the replacement level between this media resource group and other resource groups is at the media content level. When the value of this field is 0, it indicates that the corresponding media resource group does not form an alternative relationship with any media resource group. The alternative_asset_flag is the replacement identifier. When the value of this field is 1, it indicates that there is an alternative relationship between the media tracks of the corresponding media resource group and the media tracks within other groups. When the value of this field is 0, it indicates that there are no replaceable media tracks within the corresponding media resource group. The alternative_type represents the difference attribute type of the replacement media resource (media track). According to the value of this difference type, this track can have one or more difference attributes. The quality_ranking represents the quality level information of the media resource. The smaller the value of this field, the higher the quality of the corresponding media resource. The lossless_flag represents the encoding type of the encoding method of the media resource. When the value of this field is 0, it indicates that the encoding type of the corresponding media resource is a lossy encoding type; when the value of this field is 1, it indicates that the encoding type of the corresponding media resource is a lossless encoding type. The Bitrate represents the bitrate information, indicating the bitrate of the corresponding media resource. The Framerate represents the frame rate information, indicating the frame rate of the corresponding media resource. The codec_type represents the encoding method, indicating the encoding method of the corresponding media resource. The is_tile_based represents the structural organization field of the media resource. When the value of this field is 1, it indicates that the media resources within this media resource group are organized in a tile structure; when the value of this field is 0, it indicates that there is no tile structure for the media resources within this media resource group.

[0091] S102. Send the at least two playback groups, the replacement relationship indication information, and the replacement attribute information to the playback terminal.

[0092] In this application, the service device can encapsulate the at least two playback groups, the replacement relationship indication information, and the replacement attribute information to obtain a media resource file, and send the media resource file to the playback terminal. The playback terminal can perform a de-encapsulation process on the media resource file to obtain the at least two playback groups, the replacement relationship indication information, and the replacement attribute information. In one embodiment, the playback terminal can select a target playback group that is adapted to the processing capability of the playback terminal from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information, and decode the target playback group to obtain a volumetric media. In one embodiment, the playback terminal can select a target playback group that matches the user's needs from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information, and decode the target playback group to obtain a volumetric media.

[0093] In this application, the service device can obtain the replacement relationship indication information and the replacement attribute information between each of the at least two playback groups of the volumetric media, and send the replacement relationship indication information, the replacement attribute information, and the at least two playback groups to the playback terminal. Since the replacement relationship indication information is used to indicate the media track pairs with a replacement relationship in every two playback groups, and the replacement attribute information is used to reflect the attribute differences between the media tracks in the media track pair; that is to say, the playback quality of each playback group can be known according to the replacement relationship indication information and the replacement attribute information. By sending the replacement relationship indication information and the replacement attribute information to the playback terminal, it is beneficial for the playback terminal to adaptively select a playback group for decoding processing according to its own processing capability or user needs, so as to reconstruct and present the volumetric media, and personalized presentation can be realized. In addition, it is not necessary to decode all media tracks, which can save the processing resources of the playback terminal.

[0094] Further, please refer to Figure 9 , which is a schematic flowchart of a data processing method for a volumetric media provided by an embodiment of this application. As Figure 9 shown, this method can be executed by the service device in Figure 1 , and the method can at least include the following S201-S203:

[0095] S201. Obtain at least two playback groups of the volumetric media.

[0096] Optionally, step S201 may include the following steps s11 to s12.

[0097] s11. Obtain at least two media tracks of the volumetric media, and the track categories of each of the at least two media tracks;

[0098] s12. Divide the at least two media tracks according to the track categories of the respective media tracks to obtain at least two playback groups; there is an associated playback relationship between the media tracks within each of the at least two playback groups, and the track categories between the media tracks within each playback group are different.

[0099] In steps s11 - s12, the service device can obtain at least two media tracks of the volumetric media and the track category of each media track, and divide the media tracks with different track categories into the same playback group according to the track category of each media track to obtain at least two playback groups.

[0100] S202. Mark every two of the at least two playback groups to obtain replacement relationship indication information and replacement attribute information between every two of the playback groups; the replacement relationship indication information is used to indicate media track pairs with a replacement relationship in every two of the playback groups, and the replacement attribute information is used to reflect the attribute differences between the media tracks in the media track pairs with a replacement relationship.

[0101] In this application, the service device can mark the media track pairs with a replacement relationship in every two of the at least two playback groups to obtain the replacement relationship indication information between every two of the playback groups; obtain the attribute differences between the attribute information of the media tracks in the media track pairs to obtain the replacement attribute information between every two of the playback groups. By marking the replacement relationship indication information and the replacement attribute information between every two of the playback groups, it is beneficial for the playback terminal to dynamically and adaptively select a playback group for decoding processing according to its own processing ability or user requirements to reconstruct and present the volumetric media, and personalized presentation can be realized.

[0102] Optionally, step S202 may include the following steps s21 - s26.

[0103] s21. Determine candidate media track pairs according to the track category of the media tracks in playback group P m and the track category of the media tracks in playback group P n ; the media tracks in the candidate media track pairs have the same track category, and both m and n are positive integers less than Z, where Z is the number of groups of the at least two playback groups.

[0104] s22. Obtain the attribute information of the media tracks in the candidate media track pairs.

[0105] s23. Perform difference identification on the attribute information of the media tracks in the candidate media track pairs to obtain a difference identification result.

[0106] s24. If the difference recognition result indicates that there are differences in the attribute information of the media tracks in the candidate media track pair, then use the candidate media track pair as the media track pair with a replacement relationship, and generate a replacement identifier for the media track pair.

[0107] s25. Use the replacement identifier as the replacement relationship indication information between the playback group P m and the playback group P n

[0108] s26. Use the attribute information of the media tracks in the candidate media track pair as the replacement attribute information between the playback group P m and the playback group P n

[0109] In steps s21 to s26, the service device can identify candidate media track pairs with the same track category from the playback group P m and the playback group P n , and obtain the attribute information of the media tracks in the candidate media track pair. Perform difference recognition on the attribute information between the media tracks in the candidate media track pair to obtain a difference recognition result, which is used to reflect whether there are differences in the attribute information between the media tracks in the candidate media track pair. If there are no differences in the attribute information between the media tracks in the candidate media track pair, it indicates that the attribute information of the media tracks in the candidate media track pair is the same, that is, the playback quality of the media tracks in the candidate media track pair is the same, then there is no replacement relationship between the media tracks in the candidate media track pair. If there are differences in the attribute information between the media tracks in the candidate media track pair, it indicates that the attribute information of the media tracks in the candidate media track pair is different, that is, the playback quality of the media tracks in the candidate media track pair is different, then there is a replacement relationship between the media tracks in the candidate media track pair. Further, the candidate media track pair can be used as the media track pair with a replacement relationship, and a replacement identifier for the media track pair can be generated. Further, the replacement identifier can be used as the replacement relationship indication information between the playback group P m and the playback group P n , and the attribute information of the media tracks in the candidate media track pair can be used as the replacement attribute information between the playback group P m and the playback group P n . By marking the replacement relationship indication information and replacement attribute information between every two playback groups, it is beneficial for the playback terminal to dynamically and adaptively select a playback group for decoding processing according to its own processing capabilities or user needs to reconstruct and present volumetric media, and personalized presentation can be achieved.

[0110] S203. Send the at least two playback groups, the replacement relationship indication information, and the replacement attribute information to the playback terminal. ​​

[0111] In this application, the service device can encapsulate the at least two playback groups, the replacement relationship indication information, and the replacement attribute information to obtain a media resource file, and send the media resource file to the playback terminal. The playback terminal can perform a decompression process on the media resource file to obtain the at least two playback groups, the replacement relationship indication information, and the replacement attribute information. Further, the playback terminal can select a target playback group that matches the playback terminal from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information, and perform a decoding process on the target playback group to obtain volumetric media. The playback terminal does not need to perform a decoding process on all media tracks, which can save the processing resources of the playback terminal.

[0112] Optionally, in step S203, the service device can send the replacement relationship indication information and the replacement attribute information to the playback terminal through a data box or a transmission protocol. Specifically, when the service device can send the replacement relationship indication information and the replacement attribute information to the playback terminal through a data box, step S203 may include steps s31 - s32.

[0113] s31. Encapsulate the at least two playback groups, the replacement relationship indication information between each two playback groups, and the replacement attribute information to obtain a data box of the volumetric media.

[0114] s32. Send the data box including the at least two playback groups, the replacement relationship indication information, and the replacement attribute information to the playback terminal.

[0115] In steps s31 - s32, the service device can encapsulate the at least two playback groups, the replacement relationship indication information between each two playback groups, and the replacement attribute information according to the file format of the volumetric media (such as ISOBMFF (ISO BaseMedia File Format)) to obtain a data box of the volumetric media, and send the data box including the at least two playback groups, the replacement relationship indication information, and the replacement attribute information to the playback terminal. That is, encapsulate the at least two playback groups, the replacement relationship indication information, and the replacement attribute information into the same resource file. The playback terminal can obtain the at least two playback groups, the replacement relationship indication information, and the replacement attribute information by downloading the resource file, which is beneficial to improving the convenience for the playback terminal to obtain these data.

[0116] Optionally, when the service device can send the replacement relationship indication information and the replacement attribute information to the playback terminal through a transmission protocol, step S203 may include steps s41 - s45.

[0117] s41. Obtain the transmission protocol of the volumetric media.

[0118] S42. Add the replacement relationship indication information and replacement attribute information between every two playback groups to the transport protocol.

[0119] S43. Send the transport protocol including the above-mentioned replacement relationship indication information and the above-mentioned replacement attribute information to the playback terminal.

[0120] S44. Receive the acquisition request for the target playback group sent by the above-mentioned playback terminal, where the target playback group is the playback group among the at least two playback groups that matches the above-mentioned playback terminal;

[0121] S45. Send the above-mentioned target playback group to the above-mentioned playback terminal.

[0122] In steps S41 - S45, the service device can obtain the transport protocol of the volumetric media, and the transport protocol includes the Dynamic Adaptive Streaming over HTTP (DASH) protocol or the Moving Picture Experts Group Media Transport Protocol (MPEG-MP); add the replacement relationship indication information and replacement attribute information between every two playback groups to the transport protocol. Then, send the transport protocol including the replacement relationship indication information and the replacement attribute information to the playback terminal. The playback terminal can determine the packet identifier corresponding to the target playback group that matches the playback terminal according to the replacement relationship indication information and the replacement attribute information in the transport protocol, and request the target playback group corresponding to the packet identifier from the service device. That is to say, encapsulate the at least two playback groups and the replacement relationship indication information into a single file, so that it is not necessary to send all the playback groups to the playback terminal, which can save network bandwidth resources.

[0123] Optionally, step S42 may include: when the transport protocol is the Dynamic Adaptive Streaming over HTTP (DASH) protocol, add the replacement relationship indication information and replacement attribute information between every two playback groups to the track description field of the Dynamic Adaptive Streaming over HTTP (DASH) protocol. When the transport protocol is the Moving Picture Experts Group Media Transport Protocol (MPEG-MP), add the replacement relationship indication information and replacement attribute information between every two playback groups to the resource group of the Moving Picture Experts Group Media Transport Protocol (MPEG-MP). By adding the replacement relationship indication information and the replacement attribute information to the Dynamic Adaptive Streaming over HTTP (DASH) protocol or the Moving Picture Experts Group Media Transport Protocol (MPEG-MP), it is beneficial for the playback terminal to predict the replacement relationship indication information and the replacement attribute information in advance, which can improve the decoding efficiency of the playback group to a certain extent, thus bringing a better presentation effect for the volumetric media.

[0124] In this application, the service device can mark media track pairs with a replacement relationship in every two playback groups of volumetric media to obtain replacement relationship indication information between every two playback groups; and obtain the attribute information of the media tracks in the media track pairs, and identify differences in the attribute information of the media tracks in the media track pairs to obtain replacement attribute information between every two playback groups. The at least two playback groups, the replacement relationship indication information, and the replacement attribute information are sent to the playback terminal. It can be seen that by sending the replacement relationship indication information and the replacement attribute information to the playback terminal, it is beneficial for the playback terminal to adaptively select a playback group for decoding processing according to its own processing capabilities or user requirements to reconstruct and present volumetric media, enabling personalized presentation. In addition, it is not necessary to decode all media tracks, which can save the processing resources of the playback terminal.

[0125] Further, please refer to Figure 10 , which is a schematic flowchart of a data processing method for volumetric media provided by an embodiment of this application. As Figure 10 shown, this method can be executed by the Figure 1 playback terminal in, and at least includes the following S301 - S303:

[0126] S301. Obtain the replacement relationship indication information and the replacement attribute information between every two of at least two playback groups of volumetric media; the replacement relationship indication information is used to indicate media track pairs with a replacement relationship in every two of these playback groups, and the replacement attribute information is used to reflect the attribute differences between the media tracks in the media track pairs with a replacement relationship.

[0127] The playback terminal can obtain the replacement relationship indication information and the replacement attribute information between every two of at least two playback groups of volumetric media from the service device.

[0128] S302. Identify a target playback group that matches the playback terminal from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information.

[0129] In one embodiment, the playback terminal can determine the processing resources required to process each playback group according to the replacement relationship indication information and the replacement attribute information, etc.; then, the playback terminal can select a target playback group that matches the processing capabilities of the playback terminal from the at least two playback groups according to the processing resources required to process each playback group. In one embodiment, the playback quality of each playback group can be determined according to the replacement relationship indication information and the replacement attribute information, and a target playback group that matches the user's playback quality requirements can be selected from the at least two playback groups according to the playback quality of each playback group.

[0130] S303. Decode the target playback group to obtain the volumetric media, and play the volumetric media.

[0131] The playback terminal can decode the target playback group to obtain the volumetric media and play the volumetric media, that is, it is not necessary for the playback terminal to decode all playback groups, which can save the processing resources of the playback terminal.

[0132] Optionally, as Figure 11 shown, when the service device transmits the replacement relationship attribute information and the replacement attribute information through a transmission protocol, step S302 may include: receiving the transmission protocol of the volumetric media sent by the service device, and parsing the transmission protocol to obtain the replacement relationship indication information and the replacement attribute information.

[0133] Optionally, the above parsing of the transmission protocol to obtain the replacement relationship indication information and the replacement attribute information includes: when the transmission protocol is the Dynamic Adaptive Streaming over HTTP (DASH) protocol, reading the replacement relationship indication information and the replacement attribute information from the track description field of the DASH protocol; when the transmission protocol is the Moving Picture Experts Group (MPEG) media transport protocol, reading the replacement relationship indication information and the replacement attribute information from within the resource group of the MPEG media transport protocol. The playback terminal can receive the transmission protocol of the volumetric media sent by the service device, and the transmission protocol may refer to the DASH protocol or the MPEG media transport protocol. When the transmission protocol is the DASH protocol, the playback terminal can obtain the replacement relationship indication information and the replacement attribute information from the track description field of the DASH protocol. When the transmission protocol is the MPEG media transport protocol, the playback terminal can read the replacement relationship indication information and the replacement attribute information from within the resource group of the MPEG media transport protocol.

[0134] When the service device transmits the replacement relationship attribute information and the replacement attribute information through a transmission protocol, step S303 may include steps s51 - s54.

[0135] s51. Determine the grouping identifier of the candidate playback group that matches the playback terminal from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information.

[0136] s52. Generate an acquisition request carrying the grouping identifier of the candidate playback group, and send the acquisition request to the service device.

[0137] s53. Receive the candidate playback group returned by the service device in response to the acquisition request.

[0138] s54. Use the candidate playback group as the target playback group that matches the playback terminal.

[0139] In steps S51 - S54, as Figure 11 shown, the playback terminal can, according to the replacement relationship indication information and the replacement attribute information, determine the grouping identifier of a candidate playback group that matches the processing capability of the playback terminal from the at least two playback groups; alternatively, the playback terminal can, according to the replacement relationship indication information and the replacement attribute information, determine the grouping identifier of a candidate playback group that matches the user requirements corresponding to the playback terminal from the at least two playback groups. Then, a retrieval request carrying the grouping identifier of the candidate playback group can be generated, the retrieval request can be sent to the service device, the candidate playback group returned by the service device in response to the retrieval request can be received, and the candidate playback group can be used as the target playback group that matches the playback terminal. By pre - retrieving from the service device the transmission protocol carrying the replacement relationship indication information and the replacement attribute information, it is beneficial for the playback terminal to determine the target playback group that matches the playback terminal according to the replacement relationship indication information and the replacement attribute information. Only the target playback group needs to be retrieved from the service device, without the need to download all the playback groups, which can save network bandwidth resources.

[0140] Optionally, step S51 may include steps S61 - S63.

[0141] S61. Obtain the attribute information of the playback terminal, and determine the processing performance level of the playback terminal for processing volumetric media according to the attribute information of the playback terminal.

[0142] S62. According to the above - mentioned replacement relationship indication information and the above - mentioned replacement attribute information, determine the processing difficulty level of each of the at least two playback groups; the processing difficulty level is determined according to the amount of processing resources consumed for processing each of the playback groups.

[0143] S63. Use the playback groups among the at least two playback groups whose processing difficulty levels match the processing performance level of the playback terminal as candidate playback groups, and obtain the grouping identifiers of the candidate playback groups.

[0144] In steps S61 - S63, the playback terminal can obtain the attribute information of the playback terminal, which includes the storage space size, data processing speed, etc. According to the attribute information of the playback terminal, determine the processing performance level (such as the decoding ability level) of the playback terminal for processing volumetric media. Further, the amount of processing resources required to process each playback group can be determined according to the replacement relationship indication information and replacement attribute information. According to the amount of processing resources required to process each playback group, generate the processing difficulty level for processing each playback group in the at least two playback groups. That is, there is a positive correlation between the amount of processing resources required to process each playback group and the processing difficulty level of processing each playback group. That is, the processing difficulty level of processing each playback group increases as the amount of processing resources required to process each playback group increases, and decreases as the amount of processing resources required to process each playback group decreases. Further, the playback group in which the processing difficulty level in the at least two playback groups matches the processing performance level of the playback terminal can be used as the candidate playback group, and the grouping identifier of the candidate playback group can be obtained. By using the processing difficulty level of the playback group and the processing performance level of the playback terminal to select the target playback group, personalized presentation can be achieved. For example, if the processing performance level of the playback terminal is relatively high, the target playback group with a high processing difficulty level can be obtained from the service device. At this time, the playback quality of the target playback group is higher, which can improve the playback effect of the volumetric media. On the contrary, if the processing performance level of the playback terminal is relatively low, the target playback group with a low processing difficulty level can be obtained from the service device, which is beneficial to ensuring the smoothness of playing the volumetric media.

[0145] Optionally, as Figure 12 shown, when the service device transmits the replacement relationship attribute information and replacement attribute information through the data cartridge, step S301 may include: obtaining the data cartridge of the volumetric media from the service device, and performing decompression processing on the data cartridge of the volumetric media to obtain the at least two playback groups, the replacement relationship indication information and replacement attribute information between each two playback groups in the at least two playback groups.

[0146] Optionally, step S51 may include: obtaining the first quality level of the volumetric media played by the playback terminal in history; determining the second quality level of each playback group in the at least two playback groups according to the replacement relationship indication information and the replacement attribute information; using the playback group in which the second quality level in the at least two playback groups matches the first quality level as the candidate playback group, and obtaining the grouping identifier of the candidate playback group.

[0147] The playback terminal can select a matching target playback group according to historical playback behavior data. Specifically, it can obtain the first quality level of the volumetric media played by the playback terminal in the past (such as in the recent week or recent month), and determine the second quality level of each playback group in the at least two playback groups according to the replacement relationship indication information and the replacement attribute information. For example, determine the second quality level of each playback group according to the coding type of the coding method of the media track pair, that is, the quality level of the playback group corresponding to the media track with the lossless coding type is higher than the quality level of the playback group corresponding to the media track with the lossy coding type. The playback group with the same second quality level as the first quality level in the at least two playback groups is used as the candidate playback group, and the grouping identifier of the candidate playback group is obtained. By selecting a matching target playback group according to historical playback behavior data, it is beneficial to present satisfactory volumetric media to users.

[0148] Optionally, the playback terminal can determine the second quality level of each playback group in the at least two playback groups according to the replacement relationship indication information and the replacement attribute information, display the second quality level of each playback group, and in response to the selection operation of the playback group from the at least two playback groups according to the above second quality level, use the selected playback group as the candidate playback group and obtain the grouping identifier of the candidate playback group. By displaying the second quality level of each playback group, it is beneficial for users to select playback groups according to their own preferences and is conducive to realizing personalized presentation of volumetric media. When the service device transmits the replacement relationship attribute information and the replacement attribute information through the data box, the playback terminal can obtain the replacement relationship attribute information, the replacement attribute information, and the at least two playback groups through the data box. Specifically, the playback terminal can obtain the data box of the volumetric media from the service device, perform decompression processing on the data box of the volumetric media, and obtain the at least two playback groups, the replacement relationship indication information between every two playback groups in the at least two playback groups, and the replacement attribute information. By performing decompression processing on the data box of the volumetric media, the at least two playback groups, the replacement relationship indication information, and the replacement attribute information can be obtained, which can improve the convenience of obtaining these information.

[0149] Optionally, the above step S303 includes: performing decoding processing on the target playback group to obtain the media content of the volumetric media; the media content of the volumetric media includes audio content, video content, and text content; performing time synchronization on the audio content, the video content, and the text content to obtain the volumetric media, and playing the volumetric media.

[0150] The playback terminal can determine the encoding methods of the respective media tracks in the target playback group from the description information of the target playback group, and determine the decoding methods of the respective media tracks according to the encoding methods of the respective media tracks in the target playback group. Decode the respective media tracks using the decoding methods of the respective media tracks to obtain the media content of the volumetric media; perform time synchronization on the audio content, the video content, and the text content to obtain the volumetric media, and play the volumetric media. Only by the playback terminal performing a decoding operation on one of at least two playback groups can the volumetric media be obtained, without the need to perform decoding operations on all playback groups, which can save the decoding resources of the playback terminal. In this application, the playback terminal dynamically and adaptively selects a target playback group that matches it from at least two playback groups according to the replacement relationship indication information and the replacement attribute information. Only by performing a decoding operation on the target playback group can the volumetric media be obtained, without the need to perform decoding operations on all playback groups, which can save the processing resources of the playback terminal.

[0151] Please refer to Figure 13 , which is a schematic structural diagram of a data processing device 1 for volumetric media provided by an embodiment of this application. The above-mentioned data processing device 1 for volumetric media can be a computer program (including program code) running in a computer device. For example, the data processing device 1 for volumetric media is an application software; this device can be used to execute the corresponding steps in the method provided by the embodiment of this application. As Figure 13 shown, the data processing device 1 for volumetric media can include: an acquisition module 131 and a sending module 132.

[0152] The acquisition module is used to acquire the replacement relationship indication information and the replacement attribute information between every two of at least two playback groups of the volumetric media;

[0153] The sending module is used to send the above-mentioned at least two playback groups, the above-mentioned replacement relationship indication information, and the above-mentioned replacement attribute information to the playback terminal.

[0154] Optionally, the above-mentioned replacement relationship indication information is used to indicate the media track pairs with a replacement relationship in every two of the above-mentioned playback groups, and the above-mentioned replacement attribute information is used to reflect the attribute differences between the media tracks in the media track pairs with a replacement relationship.

[0155] Optionally, the above-mentioned attribute difference reflects the playback quality difference between two playback groups with a replacement relationship.

[0156] Optionally, the above-mentioned acquisition module acquires the replacement relationship indication information and the replacement attribute information between every two of at least two playback groups of the volumetric media, including:

[0157] Obtain a data box for the volumetric media; the data box for the volumetric media includes replacement relationship indication information and replacement attribute information between every two playback groups.

[0158] Optionally, the obtaining module obtains replacement relationship indication information and replacement attribute information between every two of at least two playback groups of the volumetric media, including:

[0159] Obtain the transmission protocol of the volumetric media, and the transmission protocol includes replacement relationship indication information and replacement attribute information between every two of the playback groups.

[0160] Optionally, when the transmission protocol is a Dynamic Adaptive Streaming over HTTP (DASH) protocol, the replacement relationship indication information and the replacement attribute information between every two of the playback groups are located in the track description field of the DASH protocol;

[0161] When the transmission protocol is an MPEG media transport protocol, the replacement relationship indication information and the replacement attribute information between every two of the playback groups are located within the resource group of the MPEG media transport protocol.

[0162] Optionally, the media track pair includes a first media track belonging to playback group P m and a second media track belonging to playback group P n , and the replacement attribute information between the playback group P m and the playback group P n includes the attribute information of the first media track and the attribute information of the second media track. Both m and n are positive integers less than Z, and Z is the number of groups of the at least two playback groups;

[0163] Optionally, the attribute information of the first media track includes the encoding method of the first media track, the attribute information of the second media track includes the encoding method of the second media track, and the encoding method of the first media track is different from the encoding method of the second media track.

[0164] Optionally, the attribute information of the first media track includes at least one of the transmission rate, structure organization, quality level, and encoding type corresponding to the encoding method of the first media track;

[0165] The attribute information of the second media track includes at least one of the transmission rate, structure organization, quality level, and encoding type corresponding to the encoding method of the second media track.

[0166] Optionally, the replacement relationship indication information further includes a replacement level field; when the playback group P m and the playback group P nWhen the replacement level field in the replacement relationship indication information between is the first numerical value, the above replacement level field is used to reflect the above playback group P m and the above playback group P n The replacement level between is at the media content level;

[0167] When the above playback group P m and the above playback group P n When the replacement level field in the replacement relationship indication information of is the second numerical value, the above replacement level field is used to reflect the above playback group P m and the above playback group P n The replacement level between is at the track level.

[0168] Optionally, when the replacement level between the above playback group P m and the above playback group P n is at the media content level, the above playback group P m and the above playback group P n The number of media track pairs with a replacement relationship in is R, and R is the number of media tracks in the above playback group P m The number of media tracks in the above playback group P m The number of media tracks in is the same as the number of tracks in the above playback group P n ;

[0169] When the above playback group P m and the above playback group P n When the replacement level between is at the track level, the above playback group P m and the above playback group P n The number of media track pairs with a replacement relationship in is a positive integer less than R.

[0170] Optionally, when the replacement level between the above playback group P m and the above playback group P n is at the media content level, the media tracks in the above playback group P m are all encoded in the first encoding method; the media tracks in the above playback group P n are all encoded in the second encoding method.

[0171] Optionally, there is a joint playback relationship between the media tracks in each of the above at least two playback groups, and the track categories between the media tracks in each of the above playback groups are different;

[0172] The track categories between the media tracks in the media track pairs with a replacement relationship in each of the above two playback groups are the same.

[0173] Optionally, the above replacement relationship indication information includes the replacement identifiers of the media track pairs with a replacement relationship in each of the above two playback groups.

[0174] According to an embodiment of the present application, Figure 6 the steps involved in the data processing method of the volumetric medium shown can be performed by Figure 13 each module in the data processing device of the volumetric medium shown. For example, Figure 6 the step S101 shown in Figure 13 can be performed by the acquisition module 131 in Figure 6 the step S102 shown in Figure 13 can be performed by the sending module 132 in

[0175] According to an embodiment of the present application, Figure 13 each module in the data processing device of the volumetric medium shown can be separately or all combined into one or several units to form, or a certain one (or some) of the units can be further split into multiple smaller sub-units in terms of function, and the same operations can be achieved without affecting the realization of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions. In practical applications, the function of one module can also be realized by multiple units, or the functions of multiple modules can be realized by one unit. In other embodiments of the present application, the data processing device of the volumetric medium may also include other units. In practical applications, these functions can also be assisted by other units and can be realized by the cooperation of multiple units.

[0176] According to an embodiment of the present application, it can be achieved by running a computer program (including program code) capable of executing the respective steps involved in the corresponding method shown in Figure 6 on a general-purpose computer device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM), to construct the data processing device of the volumetric medium shown in Figure 13 and to implement the data processing method of the volumetric medium of the embodiments of the present application. The above computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above computing device through the computer-readable recording medium and run therein.

[0177] In this application, the service device can mark media track pairs with a replacement relationship in every two playback groups of the volumetric media to obtain replacement relationship indication information between every two playback groups; and obtain the attribute information of the media tracks in the media track pairs, and identify the differences in the attribute information of the media tracks in the media track pairs to obtain replacement attribute information between every two playback groups. The at least two playback groups, the replacement relationship indication information, and the replacement attribute information are sent to the playback terminal. It can be seen that by sending the replacement relationship indication information and the replacement attribute information to the playback terminal, it is beneficial for the playback terminal to adaptively select a playback group for decoding processing according to its own processing capabilities or user requirements to reconstruct and present the volumetric media, enabling personalized presentation. In addition, it is not necessary to decode all media tracks, which can save the processing resources of the playback terminal.

[0178] Please refer to Figure 14 , which is a schematic structural diagram of a data processing device 2 for volumetric media provided by an embodiment of this application. The above-mentioned data processing device 2 for volumetric media can be a computer program (including program code) running in a computer device. For example, the data processing device 2 for volumetric media is an application software; this device can be used to execute the corresponding steps in the method provided by an embodiment of this application. As Figure 14 shown, the data processing device 2 for volumetric media may include: an acquisition module 141, a marking module 142, and a sending module 143.

[0179] The acquisition module is used to acquire at least two playback groups of the volumetric media;

[0180] The marking module is used to mark every two of the above-mentioned at least two playback groups to obtain the replacement relationship indication information and the replacement attribute information between every two of the above-mentioned playback groups;

[0181] The sending module is used to send the above-mentioned at least two playback groups, the above-mentioned replacement relationship indication information, and the above-mentioned replacement attribute information to the playback terminal.

[0182] Optionally, the above-mentioned replacement relationship indication information is used to indicate the media track pairs with a replacement relationship in every two of the above-mentioned playback groups, and the above-mentioned replacement attribute information is used to reflect the attribute differences between the media tracks in the media track pairs with a replacement relationship.

[0183] Optionally, the above-mentioned acquisition module acquires at least two playback groups of the volumetric media, including:

[0184] Acquire at least two media tracks of the volumetric media, and the track categories of each of the above-mentioned at least two media tracks;

[0185] According to the track categories of the above-mentioned respective media tracks, the above-mentioned at least two media tracks are divided to obtain at least two play groups; the media tracks within each play group among the above-mentioned at least two play groups have a combined play relationship, and the track categories among the media tracks within each of the above-mentioned play groups are different.

[0186] Optionally, the above-mentioned marking module marks every two play groups among the above-mentioned at least two play groups to obtain replacement relationship indication information and replacement attribute information between every two play groups, including:

[0187] According to the track category of the media tracks of play group P m and the track category of the media tracks in play group P n to determine candidate media track pairs; the media tracks in the above-mentioned candidate media track pairs have the same track category, where m and n are both positive integers less than Z, and Z is the number of groups of the above-mentioned at least two play groups;

[0188] Obtain the attribute information of the media tracks in the above-mentioned candidate media track pairs;

[0189] Perform difference identification on the attribute information of the media tracks in the above-mentioned candidate media track pairs to obtain a difference identification result;

[0190] If the above-mentioned difference identification result indicates that there are differences in the attribute information between the media tracks in the above-mentioned candidate media track pairs, then use the above-mentioned candidate media track pairs as media track pairs with a replacement relationship, and generate a replacement identifier for the above-mentioned media track pairs;

[0191] Use the above-mentioned replacement identifier as the replacement relationship indication information between the above-mentioned play group P m and the above-mentioned play group P n ;

[0192] Use the attribute information of the media tracks in the above-mentioned candidate media track pairs as the replacement attribute information between the above-mentioned play group P m and the above-mentioned play group P n .

[0193] Optionally, the above-mentioned sending module sends the above-mentioned at least two play groups, the above-mentioned replacement relationship indication information, and the above-mentioned replacement attribute information to a playback terminal, including:

[0194] Package the above-mentioned at least two play groups, the replacement relationship indication information and the replacement attribute information between every two play groups to obtain a data box of the above-mentioned volumetric media;

[0195] Send the data box including the above-mentioned at least two play groups, the above-mentioned replacement relationship indication information, and the above-mentioned replacement attribute information to the above-mentioned playback terminal.

[0196] Optionally, the sending module sends the at least two playback groups, the replacement relationship indication information, and the replacement attribute information to the playback terminal, including:

[0197] Obtain the transmission protocol of the volumetric media;

[0198] Add the replacement relationship indication information and the replacement attribute information between every two playback groups to the transmission protocol;

[0199] Send the transmission protocol including the replacement relationship indication information and the replacement attribute information to the playback terminal;

[0200] Receive a request for obtaining a target playback group sent by the playback terminal, where the target playback group is a playback group among the at least two playback groups that matches the playback terminal;

[0201] Send the target playback group to the playback terminal.

[0202] Optionally, the sending module adds the replacement relationship indication information and the replacement attribute information between every two playback groups to the transmission protocol, including:

[0203] When the transmission protocol is a Dynamic Adaptive Streaming over HTTP (DASH) protocol, add the replacement relationship indication information and the replacement attribute information between every two playback groups to the track description field of the DASH protocol;

[0204] When the transmission protocol is a Moving Picture Experts Group (MPEG) media transport protocol, add the replacement relationship indication information and the replacement attribute information between every two playback groups to the resource group of the MPEG media transport protocol.

[0205] According to an embodiment of the present application, Figure 9 The steps involved in the data processing method of the volumetric media shown can be Figure 14 executed by each module in the data processing device of the volumetric media shown. For example, Figure 9 The step S201 shown in Figure 14 can be executed by the obtaining module 141 in Figure 9 The step S202 shown in Figure 14 can be executed by the marking module 142 in Figure 9 The step S203 shown in Figure 14 can be executed by the sending module 143 in

[0206] According to an embodiment of the present application, Figure 14Each module of the data processing device 2 for volumetric media shown can be separately or all combined into one or several units to form, or a certain one (or some) of the units can be further split into multiple smaller sub-units in terms of function, and the same operations can be achieved without affecting the realization of the technical effects of the embodiments of this application. The above modules are divided based on logical functions. In practical applications, the function of one module can also be realized by multiple units, or the functions of multiple modules can be realized by one unit. In other embodiments of this application, the data processing device for volumetric media can also include other units. In practical applications, these functions can also be assisted by other units and can be realized through the cooperation of multiple units.

[0207] According to an embodiment of this application, it can be achieved by running a computer program (including program code) capable of executing the respective steps involved in the corresponding method shown in Figure 9 on a general-purpose computer device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM), to construct the data processing device for volumetric media shown in Figure 14 and to implement the data processing method for volumetric media of the embodiments of this application. The above computer program can be recorded on, for example, a computer-readable recording medium, loaded into the above computing device through the computer-readable recording medium, and run therein.

[0208] In this application, the service device can mark the media track pairs with a replacement relationship in every two playback groups of the volumetric media to obtain the replacement relationship indication information between every two playback groups; and obtain the attribute information of the media tracks in the media track pair, and perform difference recognition on the attribute information of the media tracks in the media track pair to obtain the replacement attribute information between every two playback groups. Send at least two playback groups, the replacement relationship indication information, and the replacement attribute information to the playback terminal. It can be seen that by sending the replacement relationship indication information and the replacement attribute information to the playback terminal, it is beneficial for the playback terminal to adaptively select a playback group for decoding processing according to its own processing ability or user requirements to reconstruct and present the volumetric media, and personalized presentation can be achieved. Additionally, it is not necessary to decode all media tracks, which can save the processing resources of the playback terminal.

[0209] Please refer to Figure 15 , which is a schematic structural diagram of a data processing device 3 for volumetric media provided by an embodiment of this application. The above data processing device 3 for volumetric media can be a computer program (including program code) running in a computer device. For example, the data processing device 3 for volumetric media is an application software; this device can be used to execute the corresponding steps in the method provided by the embodiments of this application. As Figure 15As shown in the figure, the data processing device 3 for volumetric media may include: an acquisition module 151, an identification module 152, and a decoding module 153.

[0210] An acquisition module, configured to acquire replacement relationship indication information and replacement attribute information between every two of at least two playback groups of the volumetric media;

[0211] An identification module, configured to identify a target playback group matching the playback terminal from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information;

[0212] A decoding module, configured to perform decoding processing on the target playback group to obtain the volumetric media and play the volumetric media.

[0213] Optionally, the replacement relationship indication information is used to indicate media track pairs having a replacement relationship in every two of the playback groups, and the replacement attribute information is used to reflect the attribute differences between the media tracks in the media track pairs having a replacement relationship.

[0214] Optionally, the acquisition module acquires the replacement relationship indication information and the replacement attribute information between every two of at least two playback groups of the volumetric media, including:

[0215] Receiving the transmission protocol of the volumetric media sent by the service device;

[0216] Parsing the transmission protocol to obtain the replacement relationship indication information and the replacement attribute information.

[0217] Optionally, the identification module identifies a target playback group matching the playback terminal from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information; including:

[0218] Determining the grouping identifier of a candidate playback group matching the playback terminal from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information;

[0219] Generating an acquisition request carrying the grouping identifier of the candidate playback group and sending the acquisition request to the service device;

[0220] Receiving the candidate playback group returned by the service device in response to the acquisition request;

[0221] Taking the candidate playback group as the target playback group matching the playback terminal.

[0222] Optionally, the identification module determines the grouping identifier of a candidate playback group matching the playback terminal from the at least two playback groups according to the replacement relationship indication information and the replacement attribute information, including:

[0223] Obtain the attribute information of the above playback terminal, and determine the processing performance level of the above playback terminal for processing volumetric media according to the attribute information of the above playback terminal;

[0224] According to the above replacement relationship indication information and the above replacement attribute information, determine the processing difficulty level of each of the at least two playback groups; the above processing difficulty level is determined according to the amount of processing resources consumed for processing each of the above playback groups;

[0225] Use the playback groups among the above at least two playback groups whose processing difficulty levels match the processing performance level of the above playback terminal as candidate playback groups, and obtain the grouping identifiers of the above candidate playback groups.

[0226] Optionally, the above recognition module determines the grouping identifiers of the candidate playback groups that match the playback terminal from the above at least two playback groups according to the above replacement relationship indication information and the above replacement attribute information, including:

[0227] Obtain the first quality level of the volumetric media played by the above playback terminal historically;

[0228] According to the above replacement relationship indication information and the above replacement attribute information, determine the second quality level of each of the above at least two playback groups;

[0229] Use the playback groups among the above at least two playback groups whose second quality levels match the above first quality level as candidate playback groups, and obtain the grouping identifiers of the above candidate playback groups. Optionally, the above obtaining module parses the above transmission protocol to obtain the above replacement relationship indication information and the above replacement attribute information, including:

[0230] When the above transmission protocol is the Dynamic Adaptive Streaming over HTTP (DASH) protocol, read the above replacement relationship indication information and the above replacement attribute information from the track description field of the above DASH protocol;

[0231] When the above transmission protocol is the Moving Picture Experts Group Media Transport Protocol (MPEG-DASH), read the above replacement relationship indication information and the above replacement attribute information from within the resource group of the above MPEG-DASH protocol.

[0232] Optionally, the above obtaining module obtains the replacement relationship indication information and the replacement attribute information between every two of the at least two playback groups of the volumetric media, including:

[0233] Obtain the data box of the above volumetric media from the service device;

[0234] The data box of the above volumetric media is unpacked to obtain the above at least two playback groups, the indication information of the replacement relationship between every two of the above at least two playback groups, and the replacement attribute information.

[0235] Optionally, the decoding module decodes the target playback group to obtain the above volumetric media and plays the volumetric media, including:

[0236] Decoding the target playback group to obtain the media content of the above volumetric media; the media content of the above volumetric media includes audio content, video content, and text content;

[0237] Performing time synchronization on the above audio content, the above video content, and the above text content to obtain the above volumetric media and playing the volumetric media.

[0238] According to an embodiment of the present application, Figure 10 the steps involved in the data processing method of the volumetric media shown can be performed by Figure 15 each module in the data processing device of the volumetric media shown. For example, Figure 10 the step S301 shown in Figure 15 can be performed by the acquisition module 151 in Figure 10 the step S302 shown in Figure 15 can be performed by the recognition module 152 in Figure 10 the step S303 shown in Figure 15 can be performed by the decoding module 153 in

[0239] According to an embodiment of the present application, Figure 15 each module of the data processing device 3 of the volumetric media shown can be separately or all combined into one or several units to form, or a certain one (or some) of the units can be further split into multiple smaller sub-units in terms of function, and the same operations can be achieved without affecting the realization of the technical effects of the embodiments of the present application. The above modules are divided based on logical functions. In actual applications, the function of one module can also be realized by multiple units, or the functions of multiple modules can be realized by one unit. In other embodiments of the present application, the data processing device of the volumetric media may also include other units. In actual applications, these functions can also be assisted by other units and can be realized by the cooperation of multiple units.

[0240] According to an embodiment of the present application, it can be achieved by running, on a general computer device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM), a program capable of performing as Figure 10A computer program (including program code) for each step involved in the corresponding method shown in the following can be used to construct a data processing device for volumetric media as shown in Figure 15 and to implement the data processing method for volumetric media according to the embodiments of the present application. The above computer program can be recorded on, for example, a computer-readable recording medium, loaded into the above computing device through the computer-readable recording medium, and run therein.

[0241] In the present application, the playback terminal dynamically and adaptively selects a target playback group that matches it from at least two playback groups according to the replacement relationship indication information and the replacement attribute information. Only by performing a decoding operation on the target playback group can the volumetric media be obtained, and it is not necessary to perform a decoding operation on all the playback groups, which can save the processing resources of the playback terminal.

[0242] Please refer to Figure 16 , which is a schematic structural diagram of a computer device provided by an embodiment of the present application. As Figure 16 shown, the above computer device 1000 may include: a processor 1001, a network interface 1004, and a memory 1005. In addition, the above computer device 1000 may further include: a user interface 1003 and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display) and a keyboard (Keyboard). Optionally, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. Optionally, the memory 1005 may further be at least one storage device located far from the aforementioned processor 1001. As Figure 16 shown, the memory 1005, as a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

[0243] In Figure 16 the computer device 1000 shown, the network interface 1004 can provide network communication functions; while the user interface 1003 is mainly used to provide an input interface for users; and the processor 1001 can be used to call the device control application program stored in the memory 1005 to implement the steps of the method described above.

[0244] In this application, the service device can mark the media track pairs with a replacement relationship in every two playback groups of the volumetric media to obtain replacement relationship indication information between every two playback groups; and obtain the attribute information of the media tracks in the media track pairs, and identify the differences in the attribute information of the media tracks in the media track pairs to obtain replacement attribute information between every two playback groups. The at least two playback groups, the replacement relationship indication information, and the replacement attribute information are sent to the playback terminal. It can be seen that by sending the replacement relationship indication information and the replacement attribute information to the playback terminal, it is beneficial for the playback terminal to adaptively select a playback group for decoding processing according to its own processing capabilities or user needs to reconstruct and present the volumetric media, and personalized presentation can be achieved. In addition, it is not necessary to decode all media tracks, which can save the processing resources of the playback terminal.

[0245] It should be understood that the computer device 1000 described in the embodiments of this application can execute the description of the data processing method of the above-mentioned volumetric media in the corresponding embodiments of the foregoing Figure 6 , Figure 9 and the foregoing Figure 10 The description of the corresponding embodiments will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. Figures 13 - 15 The description of the data processing device of the above-mentioned volumetric media in the corresponding embodiments will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated.

[0246] In addition, it should be pointed out here that: The embodiments of this application also provide a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the data processing device of the above-mentioned volumetric media mentioned above, and the computer program includes program instructions. When the above-mentioned processor executes the above-mentioned program instructions, it can execute the description of the data processing method of the above-mentioned volumetric media in the corresponding embodiments of the foregoing Figure 6 , Figure 9 and the foregoing Figure 10 The corresponding embodiments will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For the technical details not disclosed in the embodiments of the computer-readable storage medium involved in this application, please refer to the description of the method embodiments of this application.

[0247] As an example, the above-mentioned program instructions can be deployed to be executed on a computer device, or deployed to be executed on multiple computer devices located at one location, or, executed on multiple computer devices distributed at multiple locations and interconnected through a communication network. The multiple computer devices distributed at multiple locations and interconnected through a communication network can form a blockchain network.

[0248] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The above program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the above storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0249] The above-disclosed are only the preferred embodiments of the present application. Of course, the scope of rights of the present application cannot be limited thereby. Therefore, equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. A data processing method for volumetric media, characterized in that, Including: Obtaining indication information on replacement relationships and replacement attribute information between every two of at least two playback groups of volumetric media; Sending the at least two playback groups, the indication information on replacement relationships, and the replacement attribute information to a playback terminal.

2. The method according to claim 1, wherein The indication information on replacement relationships is used to indicate media track pairs with replacement relationships in every two of the playback groups, and the replacement attribute information is used to reflect the attribute differences between the media tracks in the media track pairs with replacement relationships.

3. The method according to claim 2, characterized in that, The attribute differences reflect the playback quality differences between two playback groups with replacement relationships.

4. The method according to any one of claims 1 to 3, characterized in that, The obtaining of the indication information on replacement relationships and the replacement attribute information between every two of at least two playback groups of volumetric media includes: Obtaining a data box of the volumetric media; the data box of the volumetric media includes the indication information on replacement relationships and the replacement attribute information between every two of the playback groups.

5. The method according to any one of claims 1 to 3, characterized in that, The obtaining of the indication information on replacement relationships and the replacement attribute information between every two of at least two playback groups of volumetric media includes: Obtaining a transmission protocol of the volumetric media, where the transmission protocol includes the indication information on replacement relationships and the replacement attribute information between every two of the playback groups.

6. The method according to claim 5, characterized in that, When the transmission protocol is a Dynamic Adaptive Streaming over HTTP (DASH) protocol, the indication information on replacement relationships and the replacement attribute information between every two of the playback groups are located in the track description field of the DASH protocol. When the transmission protocol is a Moving Picture Experts Group (MPEG) media transport protocol, the indication information on replacement relationships and the replacement attribute information between every two of the playback groups are located within the resource group of the MPEG media transport protocol.

7. The method according to any one of claims 1 to 3, characterized in that The media track pair includes a first media track belonging to playback group P m and a second media track belonging to playback group P n . The replacement attribute information between the playback group P m and the playback group P n includes the attribute information of the first media track and the attribute information of the second media track. Both m and n are positive integers less than Z, and Z is the number of groups of the at least two playback groups.

8. The method according to claim 7, wherein The attribute information of the first media track includes the encoding method of the first media track, the attribute information of the second media track includes the encoding method of the second media track, and the encoding method of the first media track is different from the encoding method of the second media track.

9. The method according to claim 7, wherein The attribute information of the first media track includes at least one of the transmission rate, structure organization, quality level, and encoding type corresponding to the encoding method of the first media track; The attribute information of the second media track includes at least one of the transmission rate, structure organization, quality level, and encoding type corresponding to the encoding method of the second media track.

10. The method according to claim 7, characterized in that, The replacement relationship indication information includes a replacement level field; when the replacement level field in the replacement relationship indication information between the playback group P m and the playback group P n is a first value, the replacement level field is used to reflect that the replacement level between the playback group P m and the playback group P n is at the media content level; When the playback group P m and the playback group P n When the replacement level field in the replacement relationship indication information is the second value, the replacement level field is used to reflect the playback group P m and the playback group P n The replacement level between them is at the track level.

11. The method according to claim 10, wherein When the replacement level between the playback group P m and the playback group P n is at the media content level, the number of media track pairs with a replacement relationship in the playback group P m and the playback group P n is R, where R is the number of media tracks in the playback group P m , and the number of media tracks in the playback group P m is the same as the number of tracks in the playback group P n ; When the replacement level between the playback group P m and the playback group P n is at the track level, the number of media track pairs with a replacement relationship in the playback group P m and the playback group P n is a positive integer less than R.

12. The method according to claim 11, wherein When the replacement level between the playback group P m and the playback group P n is at the media content level, the encoding methods of the media tracks in the playback group P m are all the first encoding method; the encoding methods of the media tracks in the playback group P n are all the second encoding method.

13. The method according to any one of claims 1 to 3, characterized in that, There is a joint playback relationship between the media tracks within each of the at least two playback groups, and the track categories between the media tracks within each of the playback groups are different; The track categories between the media tracks in the media track pairs with replacement relationships in every two of the playback groups are the same.

14. The method according to any one of claims 1 to 3, characterized in that, The indication information on replacement relationships includes replacement identifiers of the media track pairs with replacement relationships in every two of the playback groups.

15. A data processing device for volumetric media, characterized in that, Including: An obtaining module, configured to obtain indication information on replacement relationships and replacement attribute information between every two of at least two playback groups of volumetric media; A sending module, configured to send the at least two playback groups, the indication information on replacement relationships, and the replacement attribute information to a playback terminal.

16. A computer device, characterized in that, Including: A processor and a memory; The processor is connected to the memory, wherein the memory is used for storing program code, and the processor is used for calling the program code to execute the method according to any one of claims 1-14.

17. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, and the computer program is adapted to be loaded and executed by a processor to execute the method according to any one of claims 1-14.

Images