Data Processing Method, Apparatus, Device, and Readable Storage Medium for Point Cloud Media

Abstract

An embodiment of the present application provides a method, apparatus, device, and readable storage medium for processing point cloud media data. The method includes: performing point cloud encoding on a target point cloud media according to at least two point cloud density levels to obtain at least two point cloud encoded files; the point cloud density levels are used to characterize the media quality of the target point cloud media; generating a file encapsulation data box for the target point cloud media according to the at least two point cloud density levels and the at least two point cloud encoded files; performing file encapsulation processing on the at least two point cloud encoded files respectively according to the file encapsulation data box to obtain media file resources corresponding to the at least two point cloud density levels respectively. By using the present application, it is possible to dynamically adapt to user requirements, save the transmission bandwidth of point cloud media, and reduce the data transmission time.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a data processing method, apparatus, device, and readable storage medium for point cloud media. Background Art

[0002] A point cloud is a set of discrete points in space that are irregularly distributed and represent the spatial structure and surface attributes of a three-dimensional object or scene. Point clouds have a wide range of applications, for example, in virtual reality (VR) games, computer-aided design (CAD), three-dimensional point cloud remote presentation, three-dimensional reconstruction of biological tissues and organs, and other scenarios.

[0003] Currently, for the transmission of point cloud media, the main process is that the server encodes the point cloud media, encapsulates the encoded data stream to obtain a point cloud file, and transmits the point cloud file to the user terminal. Subsequently, the user terminal needs to first unpack the point cloud file and decode the unpacked data stream to obtain the display data for playback. In the prior art, since the server encodes the complete point cloud data corresponding to the point cloud media (or the point cloud media segment of the point cloud media), and the data volume corresponding to the complete point cloud data is large, when transmitting the point cloud file, if the network quality is poor, it takes a lot of time to transmit the point cloud file with a large data volume, and there will also be latency when it reaches the user terminal. Summary of the Invention

[0004] Embodiments of this application provide a data processing method, apparatus, device, and readable storage medium for point cloud media, which can dynamically adapt to user requirements, save the transmission bandwidth of point cloud media, and improve the decoding efficiency.

[0005] One aspect of the embodiments of this application provides a data processing method for point cloud media, including:

[0006] Performing point cloud encoding on the target point cloud media according to at least two point cloud density levels to obtain at least two point cloud encoding files; the point cloud density level is used to characterize the media quality of the target point cloud media;

[0007] Generating a file encapsulation data box for the target point cloud media according to at least two point cloud density levels and at least two point cloud encoding files; the file encapsulation data box includes at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that the i-th point cloud encoding file of the target point cloud media has a point cloud encoding file with a data dependency relationship, the file identification information includes the file identification of the point cloud encoding file with a data dependency relationship with the i-th point cloud encoding file, where i is a positive integer less than or equal to the number of at least two point cloud encoding files;

[0008] Perform file encapsulation processing on at least two point cloud encoded files according to the file encapsulation data box, to obtain media file resources corresponding to at least two point cloud density levels respectively.

[0009] One aspect of an embodiment of the present application provides a data processing method for point cloud media, including:

[0010] Based on at least two point cloud density levels corresponding to the target point cloud media, obtain target media file resources associated with the target point cloud density level; the target point cloud density level is used to characterize the media quality of the target point cloud media; the media file resources corresponding to at least two point cloud density levels respectively include the target media file resources, and each media file resource is obtained by performing file encapsulation processing on the point cloud encoded file corresponding to the point cloud density level to which it belongs;

[0011] Obtain a file encapsulation data box; the file encapsulation data box includes at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that the i-th point cloud encoded file of the target point cloud media has a point cloud encoded file with a data dependency relationship, the file identification information indicates the file identification of the point cloud encoded file having a data dependency relationship with the i-th point cloud encoded file, where i is a positive integer and 1 ≤ i ≤ N; N is the number of point cloud encoded files included in the target point cloud media;

[0012] Decode the target media file resources according to the file encapsulation data box, to obtain display data content with the media quality characterized by the target point cloud density level, and output the display data content.

[0013] One aspect of an embodiment of the present application provides a data processing device for point cloud media, including:

[0014] A media encoding module, configured to perform point cloud encoding on the target point cloud media according to at least two point cloud density levels, to obtain at least two point cloud encoded files; the point cloud density level is used to characterize the media quality of the target point cloud media;

[0015] A data box generation module, configured to generate a file encapsulation data box for the target point cloud media according to at least two point cloud density levels and at least two point cloud encoded files; the file encapsulation data box includes at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that the i-th point cloud encoded file of the target point cloud media has a point cloud encoded file with a data dependency relationship, the file identification information includes the file identification of the point cloud encoded file having a data dependency relationship with the i-th point cloud encoded file, where i is a positive integer less than or equal to the number of at least two point cloud encoded files;

[0016] A file encapsulation module, which is used to perform file encapsulation processing on at least two point cloud encoding files according to a file encapsulation data box, so as to obtain media file resources corresponding to at least two point cloud density levels respectively.

[0017] Among them, at least two point cloud density levels are represented as K i ; i is an integer greater than or equal to 1;

[0018] The media encoding module includes:

[0019] An associated density acquisition unit, which is used to acquire the associated point cloud density level corresponding to the point cloud density level K; the media quality represented by the associated point cloud density level is lower than the media quality represented by the point cloud density level K i ; i

[0020] An associated data acquisition unit, which is used to acquire the point cloud data S corresponding to the point cloud density level K, and the associated point cloud data corresponding to the associated point cloud density level; the point cloud data S i refers to the point cloud data in the target point cloud media that matches the point cloud density level K i ; the associated point cloud data refers to the point cloud data in the target point cloud media that matches the associated point cloud density level; the data volume of the associated point cloud data is lower than the data volume of the point cloud data S i ; i i

[0021] An incremental data encoding unit, which is used to determine the incremental point cloud data between the associated point cloud data and the point cloud data S, encode the incremental point cloud data, and obtain the point cloud encoding file T corresponding to the point cloud density level K i ; i i

[0022] Among them, the data box generation module includes:

[0023] An identifier acquisition unit, which is used to acquire a first dependency identifier; the first dependency identifier is used to represent that there is a point cloud encoding file with a data dependency relationship for the point cloud encoding file T i ;

[0024] An identifier addition unit, which is used to add the first dependency identifier to the file dependency information in the file encapsulation data box;

[0025] The identifier addition unit is also used to acquire the encoding file identifier of the associated point cloud encoding file, and add the encoding file identifier of the associated point cloud encoding file to the file identifier information in the file encapsulation data box; the associated point cloud encoding file refers to the point cloud encoding file obtained by performing point cloud encoding on the associated point cloud data.

[0026] ​​​​​Among them, at least two point cloud density levels are represented as K i ; i is an integer greater than or equal to 1;

[0027] The media encoding module includes:

[0028] A point cloud data encoding unit, configured to obtain the point cloud data S corresponding to the point cloud density level K i , perform point cloud encoding on the point cloud data S i to obtain a point cloud encoding file T corresponding to the point cloud density level K i ; the point cloud data S i refers to the point cloud data in the target point cloud media that matches the point cloud density level K i ; i i i i

[0029] Among them, the data box generation module includes:

[0030] A dependency information acquisition unit, configured to acquire a second dependency identifier, where the second dependency identifier is used to represent that there is no point cloud encoding file with a data dependency relationship for the point cloud encoding file T i ;

[0031] An information addition unit, configured to add the second dependency identifier to the file dependency information in the file encapsulation data box;

[0032] The information addition unit is further configured to set the encoding file identifier associated with the point cloud encoding file T in the file identifier information of the file encapsulation data box to an invalid value. i ;

[0033] Among them, the device further includes:

[0034] A media file identifier determination module, configured to acquire the encoding file identifier of the point cloud encoding file T corresponding to the point cloud density level K i as the media file identifier of the media file resource M i ; the media file resource M i refers to the media file resource obtained after performing file encapsulation processing on the point cloud encoding file T i ; i i

[0035] A mapping relationship creation module, configured to create a media mapping relationship among the target media identifier, media file identifier, and point cloud density level K of the target point cloud media i ;

[0036] A signaling file generation module, configured to generate a media signaling file according to the media mapping relationship; the media signaling file is used to instruct the user terminal to request the media file resource M included in the target point cloud media based on the point cloud density level K i ;i 。

[0037] Among them, the device further includes:

[0038] A request receiving module, configured to receive a media file resource acquisition request sent by a user terminal; the media file resource acquisition request is a request generated by the user terminal in response to a point cloud density level selection operation for a target point cloud media; the media file resource acquisition request carries a target point cloud density level, and the target point cloud density level is the point cloud density level selected by the point cloud density level selection operation;

[0039] A file resource acquisition module, configured to acquire a target media file resource corresponding to the target point cloud density level according to the media file resource acquisition request;

[0040] A resource return module, configured to return the target media file resource to the user terminal, so that the user terminal decodes the target media file resource to obtain display data content with the media quality characterized by the target point cloud density level, and outputs the display data content.

[0041] One aspect of the embodiments of the present application provides an encoding device, including:

[0042] A processor, adapted to implement one or more instructions; and,

[0043] A memory, storing one or more first instructions (or second instructions), and the one or more first instructions (or second instructions) are adapted to be loaded and executed by the processor to perform the above-mentioned data processing method for point cloud media.

[0044] In the embodiments of the present application, by dividing the point cloud density levels of the point cloud media at the server side (encoding side) of the point cloud media, one point cloud density level corresponds to one media quality; the server side creates media file resources of the point cloud media according to different point cloud density levels, and one media file resource corresponds to one point cloud density level. Thus, it can dynamically meet the media quality requirements of the user terminal (decoding side) for the point cloud media, support the user to select different point cloud density levels based on the current network state. Therefore, when transmitting data from the server side to the user terminal, only the corresponding data corresponding to the point cloud density level selected by the user needs to be transmitted, which can save bandwidth in transmission and avoid too long data transmission time. In summary, the present application can dynamically adapt to user needs, save the transmission bandwidth of point cloud media, and reduce data transmission time.

[0045] One aspect of the embodiments of the present application provides another data processing device for point cloud media, including:

[0046] A resource acquisition module, configured to acquire a target media file resource associated with a target point cloud density level based on at least two point cloud density levels corresponding to a target point cloud media; the target point cloud density level is used to characterize the media quality of the target point cloud media; the media file resources respectively corresponding to the at least two point cloud density levels include the target media file resource, and each media file resource is obtained by performing file encapsulation processing on a point cloud encoding file corresponding to the point cloud density level to which it belongs;

[0047] A data box acquisition module, configured to acquire a file encapsulation data box; the file encapsulation data box includes at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that the i-th point cloud encoding file of the target point cloud media has a point cloud encoding file with a data dependency relationship, the file identification information indicates the file identification of the point cloud encoding file having a data dependency relationship with the i-th point cloud encoding file, where i is a positive integer and 1 ≤ i ≤ N; N is the number of point cloud encoding files included in the target point cloud media;

[0048] A resource decoding module, configured to decode the target media file resource according to the file encapsulation data box to obtain display data content with the media quality characterized by the target point cloud density level, and output the display data content.

[0049] Among them, the resource acquisition module includes:

[0050] A signaling file acquisition unit, configured to acquire a media signaling file in response to a point cloud density level selection operation for the target point cloud media; the media signaling file includes the media identifier of the point cloud media, at least two point cloud density levels corresponding to the media identifier, and the media file identifier of the media file resources corresponding to the at least two point cloud density levels;

[0051] A request generation unit, configured to generate a media file resource acquisition request according to the media signaling file; the media file resource acquisition request carries the target point cloud density level; the target point cloud density level is the point cloud density level selected from the at least two point cloud density levels in the point cloud density level selection operation;

[0052] A request sending unit, configured to send the media file resource acquisition request to the server;

[0053] A resource receiving unit, configured to receive the target media file resource returned by the server according to the media file resource acquisition request.

[0054] Among them, the resource decoding module includes:

[0055] An associated file acquisition unit, configured to, if the file dependency information in the file encapsulation data box contains a first dependency identifier, determine an associated media file resource according to the associated media file identifier included in the file identifier information in the file encapsulation data box; the associated media file identifier is the media file identifier of an associated media file resource having a data dependency relationship with the target media file resource; the associated media file resource is a file obtained after the server performs encoding and encapsulation processing on the associated point cloud data; the associated point cloud data refers to the point cloud data in the target point cloud media that matches the associated point cloud density level; the media quality characterized by the associated point cloud density level is lower than the media quality characterized by the target point cloud density level.

[0056] A joint decoding unit, configured to perform joint decoding on the associated media file resource and the target media file resource to obtain display data content having the media quality characterized by the target point cloud density level.

[0057] Wherein, the joint decoding unit includes:

[0058] A decompackaging subunit, configured to perform decompackaging processing on the associated media file resource and the target media file resource respectively according to the file encapsulation data box to obtain an associated point cloud encoding file corresponding to the associated media file resource and a target point cloud encoding file corresponding to the target media file resource;

[0059] A decoding subunit, configured to perform decoding on the associated point cloud encoding file and the target point cloud encoding file respectively to obtain first display data content corresponding to the associated point cloud encoding file and second display data content corresponding to the target point cloud encoding file;

[0060] A data fusion subunit, configured to fuse the first display data content and the second display data content to obtain display data content having the media quality characterized by the target point cloud density level.

[0061] Wherein, the resource decoding module includes:

[0062] A decompackaging unit, configured to, if the file dependency information in the file encapsulation data box contains a second dependency identifier, perform decompackaging processing on the target media file resource according to the file encapsulation data box to obtain a target point cloud encoding file corresponding to the target media file resource;

[0063] A data decoding unit, configured to perform decoding on the target point cloud encoding file to obtain display data content having the media quality characterized by the target point cloud density level.

[0064] On the one hand, an embodiment of the present application provides a decoding device, including:

[0065] A processor, adapted to implement one or more instructions; and,

[0066] A memory stores one or more first instructions (or second instructions), and the one or more first instructions (or second instructions) are adapted to be loaded and executed by a processor to perform the above-mentioned data processing method for point cloud media.

[0067] In the embodiments of the present application, by dividing the point cloud density levels of the point cloud media on the server side (encoding side) of the point cloud media, one point cloud density level corresponds to one media quality; the server side creates media file resources for the point cloud media according to different point cloud density levels, and one media file resource corresponds to one point cloud density level. Thus, it can dynamically meet the media quality requirements of the user terminal (decoding side) for the point cloud media, support the user to select different point cloud density levels based on the current network state. Therefore, when transmitting data from the server side to the user terminal, only the corresponding data of the point cloud density level selected by the user needs to be transmitted, which can save bandwidth during transmission and avoid excessive data transmission time. In summary, the present application can dynamically adapt to user needs, save the transmission bandwidth of the point cloud media, and reduce the data transmission time. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0069] Figure 1 is an architecture diagram of a point cloud media system provided by an embodiment of the present application;

[0070] Figure 2 is a schematic diagram of encoding a point cloud media by an encoding side according to point cloud density levels provided by an embodiment of the present application;

[0071] Figure 3 is a schematic diagram of a scenario where a decoding side decodes media file resources provided by an embodiment of the present application;

[0072] Figure 4 is a schematic flowchart of a data processing method for point cloud media provided by an embodiment of the present application;

[0073] Figure 5 is a schematic flowchart of a data processing method for point cloud media provided by an embodiment of the present application;

[0074] Figure 6 is a schematic structural diagram of a data processing device for point cloud media provided by an embodiment of the present application;

[0075] Figure 7It is a schematic structural diagram of another data processing device for point cloud media provided by an embodiment of the present application;

[0076] Figure 8 It is a schematic structural diagram of an encoding device provided by an embodiment of the present application;

[0077] Figure 9 It is a schematic structural diagram of a decoding device provided by an embodiment of the present application. Detailed implementation manners

[0078] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0079] The embodiments of the present application relate to data processing technologies for point cloud media. Among them, point cloud media is a set of discrete points that are irregularly distributed in space and express the spatial structure and surface attributes of three-dimensional objects or scenes. Each point in the point cloud has at least three-dimensional position information, and may also have color, material, or other information according to different application scenarios. Usually, each point in the point cloud has the same number of additional attributes.

[0080] Point clouds can flexibly and conveniently express the spatial structure and surface attributes of three-dimensional objects or scenes, and thus are widely used, including virtual reality (VR) games, computer-aided design (CAD), geographic information system (GIS), autonomous navigation system (ANS), digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive telepresence, three-dimensional reconstruction of biological tissues and organs, etc.

[0081] Please refer to Figure 1 , Figure 1 It is an architecture diagram of a point cloud media system provided by an embodiment of the present application; as Figure 1As shown in the figure, the point cloud media system includes an encoding device and a decoding device. The encoding device may refer to the computer device used by the provider of the point cloud media, and this computer device may be a terminal (such as a PC (Personal Computer), a smart mobile device (such as a smart phone), etc.) or a server. The decoding device may refer to the computer device used by the user of the point cloud media, and this computer device may be a terminal (such as a PC (Personal Computer), a smart mobile device (such as a smart phone), a VR device (such as a VR headset, VR glasses, etc.)). The data processing process of the point cloud media includes the data processing process on the encoding device side and the data processing process on the decoding device side.

[0082] The data processing process on the encoding device side mainly includes: (1) the process of obtaining the media content of the point cloud media; (2) the process of encoding and file encapsulation of the point cloud media. The data processing process on the decoding device side mainly includes: (1) the process of file decapsulation and decoding of the point cloud media; (2) the rendering process of the point cloud media. In addition, there is a transmission process of the point cloud media between the encoding device and the decoding device, and this transmission process can be carried out based on various transmission protocols. The transmission protocols here may include but are not limited to: DASH (Dynamic Adaptive Streaming over HTTP) protocol, HLS (HTTP Live Streaming) protocol, SMTP (Smart Media Transport Protocaol), TCP (Transmission Control Protocol), etc.

[0083] The following will be combined with Figure 1 to introduce each process involved in the data processing process of the point cloud media in detail.

[0084] I. The data processing process on the encoding device side:

[0085] (1) The process of obtaining and producing the media content of the point cloud media.

[0086] 1) The process of obtaining the media content of the point cloud media.

[0087] The media content of point cloud media is obtained by a capture device collecting a real-world audio-visual scene. In one implementation, the capture device may refer to a hardware component provided in an encoding device. For example, the capture device refers to a microphone, a camera, a sensor, etc. of a terminal. In another implementation, the capture device may also be a hardware device connected to the encoding device. For example, a camera connected to a server, which provides an acquisition service for the encoding device to obtain the media content of point cloud media. The capture device may include, but is not limited to, an audio device, a camera device, and a sensing device. Among them, the audio device may include an audio sensor, a microphone, etc. The camera device may include an ordinary camera, a stereo camera, a light field camera, etc. The sensing device may include a laser device, a radar device, etc. The number of capture devices may be multiple, and these capture devices are deployed at some specific positions in the real space to simultaneously capture the audio content and video content at different angles within the space. The captured audio content and video content are synchronized both in time and space. The media content collected by the capture device is called the original point cloud data of point cloud media.

[0088] 2) The production process of the media content of point cloud media.

[0089] The captured original point cloud data (including audio content or video content) is content suitable for performing the encoding of point cloud media.

[0090] It should be noted that since only panoramic videos can be captured by the capture device, after such videos are processed by the encoding device and transmitted to the decoding device for corresponding data processing, users on the decoding device side can only view 360-degree video information by performing some specific actions (such as head rotation), and performing non-specific actions (such as moving the head) cannot obtain corresponding video changes, resulting in a poor VR experience. Therefore, it is necessary to additionally provide depth information matching the panoramic video to enable users to obtain a better immersion and a better VR experience, which involves 6DoF (Six Degrees of Freedom) production technology. When a user can move relatively freely in a simulated scene, it is called 6DoF. When using 6DoF production technology to produce the video content of point cloud media, the capture device generally selects a light field camera, a laser device, a radar device, etc. to capture the point cloud data or light field data in the space.

[0091] (2) The encoding and file encapsulation process of point cloud media.

[0092] The captured audio content can be directly audio-encoded to form the audio bitstream of the point cloud media. The captured video content can be video-encoded to obtain the video bitstream of the point cloud media. It should be noted here that if the 6DoF production technology is adopted, a specific encoding method (such as point cloud encoding) needs to be used during the video encoding process. The audio bitstream and the video bitstream are encapsulated in a file container according to the file format of the point cloud media (such as ISOBMFF (ISO Base Media File Format, ISO media file format)) to form the media file resource of the point cloud media. This media file resource can be a media file or a media segment to form the media file of the point cloud media; and the metadata of the media file resource of the point cloud media is recorded according to the requirements of the file format of the point cloud media using the Media Presentation Description (MPD). The metadata here is the general term for information related to the presentation of the point cloud media. This metadata can include the description information of the media content, the description information of the viewport, and the signaling information related to the presentation of the media content, etc. As Figure 1 shown, the encoding device will store the media presentation description information and the media file resource formed after the data processing process.

[0093] II. Data processing process at the decoding device side:

[0094] (3) The process of file de-encapsulation and decoding of the point cloud media;

[0095] The decoding device can obtain the media file resource and the corresponding media presentation description information of the point cloud media from the encoding device adaptively and dynamically through the recommendation of the encoding device or according to the user's needs at the decoding device side. For example, the decoding device can determine the orientation and position of the user based on the tracking information of the user's head / eyes / body, and then dynamically request the corresponding media file resource from the encoding device based on the determined orientation and position. The media file resource and the media presentation description information are transmitted from the encoding device to the decoding device through a transmission mechanism (such as DASH, SMT). The process of file de-encapsulation at the decoding device side is the reverse of the file encapsulation process at the encoding device side. The decoding device de-encapsulates the media file resource according to the requirements of the file format of the point cloud media (for example, the ISO media file format) to obtain the audio bitstream and the video bitstream. The decoding process at the decoding device side is the reverse of the encoding process at the encoding device side. The decoding device performs audio decoding on the audio bitstream to restore the audio content; the decoding device performs video decoding on the video bitstream to restore the video content.

[0096] (4) The rendering process of the point cloud media.

[0097] The decoding device renders the audio content obtained by audio decoding and the video content obtained by video decoding according to the rendering-related metadata in the media presentation description information corresponding to the media file resource. Once the rendering is completed, the playback output of the image is realized.

[0098] The point cloud media system supports data boxes (Boxes). A data box refers to a data block or object that includes metadata, that is, the data box contains the metadata of the corresponding media content. The point cloud media can include multiple data boxes. For example, it includes a file encapsulation data box (ISO Base Media File Format Box), which contains metadata for describing the corresponding information during file encapsulation. Among them, the ISO file encapsulation data box can include a sphere region zooming data box (Sphere Region Zooming Box), which contains metadata for describing sphere region zooming information; and so on.

[0099] From the above relevant descriptions of the point cloud media system, it can be seen that during the data processing of the point cloud media, for the encoding and file encapsulation processes of the point cloud media, the encoding device encodes the captured complete point cloud data (for example, audio content or video content) (for example, point cloud encoding) to obtain the audio bitstream or video bitstream of the point cloud media; while the decoding device decodes the media file containing the complete point cloud data transmitted by the encoding device to restore the complete data for rendering and output display. To dynamically adapt to the user's requirements for the media quality of different point cloud media at the decoding end, the present application adopts the method of dividing the point cloud density levels of the point cloud media during the encoding data processing of the point cloud media. One point cloud density level corresponds to a certain amount of point cloud data in the point cloud media. By encoding the corresponding point cloud data corresponding to different point cloud density levels, media file resources corresponding to different point cloud density levels are obtained; in this way, users at the decoding end can select the point cloud density level of the point cloud media according to their own needs, and determine the corresponding media file for transmission according to the selected point cloud density level by the user, so as to dynamically adapt to the user's requirements for media quality and support the user to select different point cloud density levels based on the current network status. Therefore, when the server transmits data to the user terminal, only the corresponding point cloud data corresponding to the point cloud density level selected by the user needs to be transmitted, and there is no need to transmit the remaining point cloud data in the point cloud media except this part of the point cloud data. This can reduce unnecessary transmission bandwidth and avoid too long data transmission time; at the same time, at the decoding end, only the media file resource containing this part of the point cloud data needs to be decoded, and there is no need to decode the complete point cloud data of the point cloud media, which can improve the decoding efficiency.

[0100] According to the coding standard for point cloud media (such as AVS (Audio Video coding Standard, source coding standard)), this application extends the ISO file encapsulation data box of the target point cloud media, and extends the point cloud density level field and related fields of the point cloud media into the ISO file encapsulation data box. Based on the coding standard, the extended syntax of the ISO file encapsulation data box for point cloud media in this application can be seen in Table 1 below:

[0101] Table 1

[0102]

[0103] Among them, point_cloud_dense_level is used to indicate the point cloud density level of the media file resources of the point cloud media contained in the PointCloudDenseLevelBox. The lower the value of this field, the lower the point cloud density level; the PointCloudDenseLevelBox field contains the media file identifier of the media file resources of the point cloud media;

[0104] point_cloud_dense_dependency_flag is used to indicate whether the point cloud media file resources at the current point cloud density level depend on the media file resources at a lower point cloud density level during decoding. If the value of this field is 1, the ID (file identifier) of the media file resources at the lower point cloud density level on which it depends is given by the TrackReferenceTypeBox data box. The TrackReferenceTypeBox data box is used to indicate the ID (file identifier) of the media file resources containing the lower point cloud density level, and at this time, the reference_type in the TrackReferenceTypeBox should take the value of 'pcdd'. When the reference_type takes the value of 'pcdd', this TrackReferenceTypeBox field is a valid field, and the ID of the media file resources contained in this TrackReferenceTypeBox is a valid ID.

[0105] It should be understood that in the encoding process of the point cloud data of the point cloud media in this application, a new concept of grading the point cloud density of the point cloud media is added. The encoding device (for example, a server) can perform point cloud encoding on some point cloud data of the point cloud media for different point cloud density levels, and then encapsulate the encoded data after point cloud encoding according to this extended ISO file encapsulation data box to obtain media file resources corresponding to different point cloud density levels.

[0106] It should be understood that after the encoding and encapsulation process of different point cloud data of the point cloud media is performed according to the point cloud density level at the encoding end (e.g., the server) to obtain the media file resource, a media signaling file (e.g., MPD signaling file) can be generated. The media signaling file contains the media identifier of the point cloud media and the media file identifier of the media file resource included in the point cloud media. It should be noted that since the media file resource is obtained by performing the encoding and encapsulation process on different point cloud data according to different point cloud density levels, the point cloud density level will also be extended in the media signaling file. For ease of understanding, please refer to Table 2. The extended fields of the MPD signaling file for the point cloud media in this application can be seen in Table 2 below:

[0107] Table 2

[0108]

[0109] Among them, @pccDenseLevel is used to indicate the point cloud density level corresponding to the current media file resource. The lower the field value of this field, the lower the point cloud density level, and the lower the media quality of the media file resource.

[0110] It should be understood that the server side can send the media signaling file (e.g., MPD signaling file) related to the point cloud media to the user terminal. In this MPD signaling file, the point cloud media includes one or more media file resources, and the corresponding point cloud density levels of each media file resource respectively. Then the user terminal can obtain the point cloud density level of the point cloud media through this MPD signaling file, and request the media file resource of the corresponding point cloud density level according to the media quality requirement.

[0111] For ease of understanding, please refer to Figure 2 , Figure 2 which is a schematic diagram of the encoding of the point cloud media by the encoding end according to the point cloud density level provided by the embodiment of the present application. As Figure 2 shown, the point cloud media A is a face image, and this face image is composed of multiple point cloud data. The point cloud media A can be divided into 3 point cloud density levels, namely point cloud density level 0, point cloud density level 1, and point cloud density level 2; among them, point cloud density level 0 is lower than point cloud density level 1, and point cloud density level 1 is lower than point cloud density level 2.

[0112] As Figure 2As shown, in the point cloud media A, the encoding end can obtain the point cloud data corresponding to the point cloud density level 0, as shown by the point cloud data 100a, the point cloud data corresponding to the point cloud density level 1, as shown by the point cloud data 100b, and the point cloud data corresponding to the point cloud density level 2, as shown by the point cloud data 100c. It should be understood that since the point cloud density level 0 is lower than the point cloud density level 1, and the point cloud density level 1 is lower than the point cloud density level 2, the amount of point cloud data contained in the point cloud data 100a corresponding to the point cloud density level 0 is lower than the amount of point cloud data contained in the point cloud data 100b corresponding to the point cloud density level 1; and the amount of point cloud data contained in the point cloud data 100b is lower than the amount of point cloud data contained in the point cloud data 100c corresponding to the point cloud density level 2. It can be seen that since the amount of point cloud data contained in the point cloud data 100a is lower than the amount of point cloud data contained in the point cloud data 100b, the media quality of the point cloud data 100a is lower than the media quality of the point cloud data 100b; similarly, the media quality of the point cloud data 100b is also lower than the media quality of the point cloud data 100c.

[0113] Furthermore, the point cloud data 100a can be point cloud encoded to obtain a point cloud encoded file 1, and the point cloud encoded file 1 can be encapsulated according to the ISO media file format to obtain a media file resource 1. It should be understood that the extended ISO file encapsulation data box contains a point_cloud_dense_level field and a point_cloud_dense_dependency_flag field. Therefore, the point cloud density level 0 corresponding to the media file resource 1 can be recorded in the file encapsulation data box of the media file resource 1. At the same time, since the media file resource is obtained by encoding the point cloud data 100a (complete point cloud data) corresponding to the point cloud density level 0, when decoding at the decoding end (such as a user terminal), there is no need to rely on the media file resources corresponding to lower point cloud density levels. Therefore, the point_cloud_dense_dependency_flag field corresponding to the media file resource 1 can be set to 0. Then, the point_cloud_dense_dependency_flag field of the media file resource 2 can be recorded in the file encapsulation data box of the media file resource 1 as point_cloud_dense_dependency_flag = 0. Then, the description information in the file encapsulation data box corresponding to the media file resource 1 can include: the file identifier of the media file resource 1, the point cloud density level corresponding to the media file resource 1 is the point cloud density level 0, the point_cloud_dense_dependency_flag of the media file resource 1 = 0, and the point cloud media corresponding to the media file resource 1 is the point cloud media A.

[0114] Similarly, the point cloud data 100b can be point cloud encoded to obtain a point cloud encoded file 2, and the point cloud encoded file 2 can be encapsulated according to the ISO media file format to obtain a media file resource 2. The point cloud density level 1 corresponding to the media file resource 2 can be recorded in the file encapsulation data box of the media file resource 2. At the same time, since the media file resource is obtained by encoding the point cloud data 100b (complete point cloud data) corresponding to the point cloud density level 1, when decoding at the decoding end (such as a user terminal), it does not need to rely on the media file resources corresponding to lower point cloud density levels. Therefore, the point_cloud_dense_dependency_flag field corresponding to the media file resource 2 can be set to 0, and the point_cloud_dense_dependency_flag field recorded in the file encapsulation data box of the media file resource 2 is point_cloud_dense_dependency_flag = 0. The description information of the media file resource 2 recorded in the file encapsulation data box of the media file resource 2 may include: the file identifier of the media file resource 2, the point cloud density level corresponding to the media file resource 2 is the point cloud density level 1, the point_cloud_dense_dependency_flag of the media file resource 2 = 0, and the point cloud media corresponding to the media file resource 2 is the point cloud media A.

[0115] As Figure 2 shown, the incremental point cloud data between the point cloud data 100b and the point cloud data 100c can be obtained. As Figure 2, the incremental point cloud data, as shown by the incremental point cloud data 100d, can be encoded for the incremental point cloud data 100d, and the obtained point cloud encoded file 3 can be used as the point cloud encoded file corresponding to the point cloud density level 2. According to the ISO media file format, the point cloud encoded file 3 can be encapsulated to obtain the media file resource 3. The point cloud density level 2 corresponding to the media file resource 3 can be recorded in the file encapsulation data box of the media file resource 3. At the same time, it should be understood that since the media file resource is obtained by encoding the incremental point cloud data 100d corresponding to the point cloud density level 2 (not the complete point cloud data 100c, but the incremental point cloud data), when decoding at the decoding end (such as a user terminal), it is necessary to rely on the media file resource (media file resource 2) corresponding to a lower point cloud density level (point cloud density level 1). That is to say, it is necessary to jointly decode the media file resource 2 containing the point cloud data 100b and the media file resource 3 containing the incremental point cloud data 100d, and the jointly decoded media file resource can have the media quality characterized by the point cloud density level 2 (i.e., the media quality composed of the point cloud data 100c). Therefore, the point_cloud_dense_dependency_flag field corresponding to the media file resource 3 can be set to 1, and the description information of the media file resource 3 recorded in the file encapsulation data box of the media file resource 3 can include: the file identifier of the media file resource 3, the point cloud density level corresponding to the media file resource 3 is the point cloud density level 2, the point_cloud_dense_dependency_flag of the media file resource 3 = 1, and the point cloud media corresponding to the media file resource 3 is the point cloud media A.

[0116] It can be understood that since the media file resource 3 needs to rely on the media file resource 2 for joint decoding, in the ISO file encapsulation data box, the media file identifier of the media file resource 2 can be added to the TrackReferenceTypeBox field in the file encapsulation data box of the media file resource 3. Then, through point_cloud_dense_dependency_flag = 1, it can be determined that the media file resource 3 needs to jointly decode other media file resources. Subsequently, through the media file identifier of the media file resource 2 in the TrackReferenceTypeBox field, it can be determined that the media file resource to be jointly decoded when decoding the media file resource 3 is the media file resource 2.

[0117] Furthermore, relevant information of point cloud media A (for example, the media identifier of point cloud media A, the media file identifier of the media file resource of point cloud media A, and the point cloud density level corresponding to each media file resource) can be recorded in a media signaling file. It should be understood that the media signaling file related to the point cloud media A can be sent to the user terminal (decoding end), and the user can obtain through the user terminal that the current point cloud media A has three point cloud density levels, namely point cloud density level 0, point cloud density level 1 and point cloud density level 2, and each point cloud density level corresponds to a media quality. The user can select a point cloud density level according to the requirement for media quality to obtain the corresponding media file resources for decoding and display.

[0118] For easier understanding, please also refer to Figure 3 , Figure 3 is a schematic diagram of a scenario in which a decoding end decodes a media file resource provided by an embodiment of the present application. Figure 3 The user terminal E shown can be a decoding end, and as Figure 3 The server shown may be an encoding end.

[0119] like Figure 3 As shown above, Figure 2 In the encoding and packaging method according to the point cloud density level described in the corresponding embodiment, the server divides the point cloud media A into three point cloud density levels, namely point cloud density level 0, point cloud density level 1 and point cloud density level 2. Then, in the user terminal E, the user E can select the point cloud density level of the point cloud media A. If the user E's demand for the point cloud media A is a higher sense of immersion, that is, the media quality of the point cloud media A is higher, the user E can click on the point cloud density level 2 through the user terminal E to request a media file resource with higher media quality.

[0120] User terminal E can respond to the trigger operation of user E and obtain the media signaling file related to the point cloud media A. Through the media signaling file, the media file resource corresponding to the point cloud density level 2 is obtained as media file resource 3. Then user terminal E can generate a resource acquisition request for media file resource 3.

[0121] Further, the user terminal E may send the acquisition request for the media file resource 3 to the server. In response to the acquisition request for the media file resource 3, the server can obtain the media file resource 3. And through the point_cloud_dense_dependency_flag = 1 field in the file encapsulation data box of the media file resource 3, the server can determine that the media file resource 3 needs to be jointly decoded with other media file resources. Subsequently, the server can obtain the media file identifier of the media file resource 2 in the TrackReferenceTypeBox field in the file encapsulation data box of the media file resource 3. Then the server can obtain the media file resource 2 and return the media file resource 2 and the media file resource 3 to the user terminal E together. After receiving the media file resource 2 and the media file resource 3, the user terminal E can determine from the point_cloud_dense_dependency_flag = 1 field and the TrackReferenceTypeBox field in the file encapsulation data box of the media file resource 3 that when decoding the media file resource 3, the media file resource 2 needs to be jointly decoded. Then the user terminal E can unpack the media file resource 2 to obtain the point cloud encoded file 2, and then decode the point cloud encoded file 2 to restore the point cloud data 100b. Similarly, the user terminal E can unpack the media file resource 3 to obtain the point cloud encoded file 3, and then decode the point cloud encoded file 3 to restore the incremental point cloud data 100d. Further, the user terminal E can fuse the point cloud data 100b and the incremental point cloud data 100d, so as to obtain the point cloud data 100c corresponding to the point cloud density level 2.

[0122] Further, the point cloud data 100c can be rendered and output. As Figure 3 shown, the user E can view the point cloud media A with the media quality characterized by the point cloud density level 2 through the user terminal E.

[0123] It should be understood that after the server receives the acquisition request for media file resource 3 sent by user terminal E and obtains media file resource 3, and determines that joint decoding with media file resource 2 is required, the server can first query whether there is a request record for media file resource 2 by the user terminal. If there is a request record for media file resource 2 by the user terminal, the server can determine that the user terminal has requested media file resource 2 before requesting media file resource 3, that is, media file resource 2 already exists locally in the user terminal, and the server can only send media file resource 3 to the user terminal; if there is no request record for media file resource 2 by the user terminal, the server can obtain media file resource 2 and send media file resource 2 and media file resource 3 to the user terminal together.

[0124] For further information, see Figure 4 , Figure 4 1 is a flow chart of a method for processing point cloud media data provided by an embodiment of the present application. The method may be executed by an encoding device in a point cloud media system, the encoding device may be a server, and the method may include the following steps:

[0125] Step S101, performing point cloud encoding on a target point cloud media according to at least two point cloud density levels to obtain at least two point cloud encoding files; the point cloud density levels are used to characterize the media quality of the target point cloud media.

[0126] In this application, the point cloud density level can be understood to be determined according to the amount of point cloud data in the point cloud media, for example, Figure 2 Taking point cloud media A in the corresponding embodiment as an example, the complete point cloud data corresponding to point cloud media A is 100c. Since point cloud data 100c contains a lot of point cloud data and a large amount of data, the corresponding point cloud density level is also high. Since the more point cloud data, the better the effect of point cloud media A presented, the higher the point cloud density level, the higher the media quality of the corresponding point cloud media. Figure 2 As shown, because the amount of point cloud data contained in point cloud data 100a is lower than that in point cloud data 100b, and the amount of point cloud data contained in point cloud data 100b is lower than that in point cloud data 100c, the point cloud density level 0 corresponding to point cloud data 100a is also lower than the point cloud density level 1 corresponding to point cloud data 100b, and the point cloud density level 1 is also lower than the point cloud density level 2 corresponding to point cloud data 100c; the media quality of point cloud media A represented by point cloud density level 0 is also lower than the media quality of point cloud media A represented by point cloud density level 1, and the media quality of point cloud media A represented by point cloud density level 1 is also lower than the media quality of point cloud media A represented by point cloud density level 2.

[0127] In this application, since one point cloud density level corresponds to point cloud data of a certain data volume, and at least two point cloud density levels are represented as K i (where i is an integer greater than or equal to 1) as an example, the specific method for point cloud encoding of the target point cloud media can be, in the point cloud data of point cloud media A, obtain the point cloud data S i matching the point cloud density level K i , and perform point cloud encoding on this point cloud data S i to obtain the point cloud encoding file T i corresponding to this point cloud density level K i .

[0128] For example, taking the point cloud encoding file corresponding to the point cloud density level 0 obtained in the above Figure 2 corresponding embodiment as an example, in the point cloud data of point cloud media A, the point cloud data 100a matching the point cloud density level 0 can be obtained, and point cloud encoding is performed on this point cloud data 100a to obtain the point cloud encoding file 1.

[0129] Optionally, it can be understood that the specific method for point cloud encoding of the target point cloud media can also be to obtain the associated point cloud density level corresponding to this point cloud density level K i ; wherein, the media quality represented by this associated point cloud density level is lower than the media quality represented by this point cloud density level K i ; subsequently, the point cloud data S i corresponding to this point cloud density level K i , and the associated point cloud data corresponding to this associated point cloud density level can be obtained; wherein, this associated point cloud data refers to the point cloud data in the target point cloud media matching the associated point cloud density level; the data volume of this associated point cloud data is lower than the data volume of this point cloud data S i ; subsequently, the incremental point cloud data between this associated point cloud data and this point cloud data S i can be determined, and encoding is performed on this incremental point cloud data to obtain an incremental encoding file, and this incremental encoding file can be used as the point cloud encoding file T i corresponding to this point cloud density level K i .

[0130] For example, the above Figure 2Taking the point cloud encoding file corresponding to point cloud density level 2 obtained in the corresponding embodiment as an example, in the point cloud data of point cloud media A, the point cloud data corresponding to point cloud density level 2 that can be obtained is point cloud data 100c. The server may not encode the point cloud data 100c, but obtain the point cloud data 100b corresponding to point cloud density level 1, and this point cloud data 100b can be used as associated point cloud data; the incremental point cloud data 100d between the point cloud data 100b and the point cloud data 100c can be determined, and the incremental point cloud data 100d is encoded to obtain an incremental encoding file, and this incremental encoding file can be used as the point cloud encoding file corresponding to this point cloud density level 2.

[0131] Step S102: Generate a file encapsulation data box of the target point cloud media according to the at least two point cloud density levels and the at least two point cloud encoding files; the file encapsulation data box includes the at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that the i-th point cloud encoding file of the target point cloud media has a point cloud encoding file with a data dependency relationship, the file identification information includes the file identification of the point cloud encoding file having the data dependency relationship with the i-th point cloud encoding file, where i is a positive integer less than or equal to the number of the at least two point cloud encoding files.

[0132] In this application, the file dependency information refers to information used to indicate whether there is a point cloud encoding file with a data dependency relationship for the point cloud encoding file T i The syntax of the file encapsulation data box (ISO Base Media File Format Box) can be referred to in Table 1 above. Step S102 is a process in which an encoding device (such as a server) assigns values to the corresponding syntax fields in the file encapsulation data box according to actual needs.

[0133] In one implementation manner, step S102 may specifically be that if the point cloud encoding file T i is obtained by the server encoding incremental point cloud data, a first dependency identifier can be obtained, where the first dependency identifier can be an identifier used to represent that the point cloud encoding file T i has a point cloud encoding file with a data dependency relationship. For example, the first dependency identifier can be 1. The first dependency identifier can be added to the file dependency information in the file encapsulation data box, that is, the file dependency information in the ISO file encapsulation data box is assigned a value of 1 (for example, the point_cloud_dense_dependency_flag field is set to 1); subsequently, the point cloud encoding file T iAn associated point cloud encoding file with data dependency relationships (a point cloud encoding file obtained after encoding the associated point cloud data), obtain the encoding file identifier of the associated point cloud encoding file, and add the encoding file identifier of the associated point cloud encoding file to the file identifier information in the file encapsulation data box. That is to say, (add the encoding file identifier of the associated point cloud encoding file in the TrackReferenceTypeBox field of the ISO file encapsulation data box).

[0134] It can be understood that when the server encodes the point cloud data S i it encodes the incremental point cloud data between the point cloud data S i and the associated point cloud data. Then, the point cloud encoding file T i obtained after encoding contains incremental point cloud data. At the decoding end, if only the point cloud encoding file T i is decoded, the restored point cloud data is incremental point cloud data, rather than the point cloud data S i . For example, taking the point cloud data 100c corresponding to the point cloud density level 2 in the above Figure 2 corresponding embodiment as an example, the server encodes the incremental point cloud data 100d between the point cloud data 100c and the point cloud data 100b (associated point cloud data) to obtain the point cloud encoding file 3. If the point cloud encoding file 3 is decoded, the restored point cloud data is the incremental point cloud data 100d, and the incremental point cloud data 100d cannot represent the media quality that the point cloud density level 2 should represent. Therefore, at the decoding end, it is necessary to jointly decode the point cloud encoding file 3 and the point cloud encoding file 2 corresponding to the point cloud data 100b. Thus, the decoding end can restore the point cloud data 100b and the incremental point cloud data 100d, and by fusing the point cloud data 100b and the incremental point cloud data 100d, the point cloud data 100c corresponding to the point cloud density level 2 can be obtained.

[0135] That is to say, when decoding the point cloud encoding file 3 at the decoding end, it is necessary to simultaneously decode the point cloud encoding file 2 (associated point cloud encoding file). Then, the file dependency information (point_cloud_dense_dependency_flag field) of the point cloud encoding file T i (for example, the point cloud encoding file 3) in the ISO file encapsulation data box can be set to the first dependency identifier (for example, set to 1, that is, point_cloud_dense_dependency_flag = 1) to indicate that the point cloud encoding file T i has an associated point cloud encoding file with data dependency relationships (for example, the point cloud encoding file 2);

[0136] Meanwhile, after setting the file dependency information to the first dependency identifier, the encoded file identifier of the associated point cloud encoded file (e.g., point cloud encoded file 2) can be added to the file identifier information (TrackReferenceTypeBox field) in the file dependency information of the ISO file encapsulated data box. Thus, through the file identifier information containing the encoded file identifier of the associated point cloud encoded file, it is possible to determine that the associated point cloud encoded file (e.g., point cloud encoded file 2), rather than other point cloud encoded files (e.g., point cloud encoded file 1), has a data dependency relationship with the point cloud encoded file T i (e.g., point cloud encoded file 3) has a data dependency relationship with the associated point cloud encoded file (e.g., point cloud encoded file 2), rather than other point cloud encoded files (e.g., point cloud encoded file 1).

[0137] In one implementation, step S102 can specifically be that when the server encodes the point cloud data S i if the point cloud data S i is the complete point cloud data (non-incremental point cloud data) corresponding to the point cloud density level K i , then the point cloud data S i after encoding processing to obtain the point cloud encoded file T i contains the point cloud data that is the complete point cloud data corresponding to the point cloud density level K i . At the decoding end, only decoding the point cloud encoded file T i can restore the complete point cloud data S i corresponding to the point cloud density level K i . Rendering the point cloud data S i can obtain the point cloud media with the media quality characterized by the point cloud density level K i .

[0138] For example, taking the encoding process of the point cloud data 100a in the above-mentioned Figure 2 corresponding embodiment as an example, the point cloud data 100a is the complete point cloud data corresponding to the point cloud density level 0. At the decoding end, directly decoding the point cloud encoded file 1 can restore the point cloud data 100a. Then, a second dependency identifier can be obtained, where the second dependency identifier can be an identifier used to indicate that there is no point cloud encoded file with a data dependency relationship for the point cloud encoded file T i . For example, the second dependency identifier can be 0; in the ISO file encapsulated data box, set the file dependency information of the point cloud encoded file T i corresponding to the point cloud density level K i (e.g., the point cloud encoded file 1 corresponding to the point cloud density level 0) to the second dependency identifier (e.g., set to 0, i.e., point_cloud_dense_dependency_flag = 0), to indicate that the point cloud encoded file T i corresponding to the point cloud density level Ki There is no associated media file resource with a data dependency; and set the encoding file identifier associated with the point cloud encoding file T in the file identifier information to an invalid value (e.g., set to the empty value "null"). i Set the encoding file identifier associated with the point cloud encoding file T to an invalid value (e.g., set to the empty value "null").

[0139] Step S103, perform file encapsulation processing on the at least two point cloud encoding files respectively according to the file encapsulation data box to obtain the media file resources corresponding to the at least two point cloud density levels respectively.

[0140] In this application, the point cloud encoding file T can be encapsulated in a file container according to the ISO file encapsulation format of the ISO file encapsulation data box to form a media file resource M of the point cloud media. i That is, the point cloud encoding file T is encapsulated in a file container to form a media file resource M of the point cloud media. i It can be understood that the media file resource M contains the point cloud encoding file T i and the file encapsulation data box of the point cloud encoding file T i and the file encapsulation data box of the point cloud encoding file T i The file encapsulation data box records relevant information such as the point cloud density level of the point cloud encoding file T i and whether there is a data dependency.

[0141] It can be understood that after the encoding and encapsulation of the target point cloud media is completed to obtain the media file resource, the relevant information of the target point cloud media can be recorded in a media signaling file (e.g., MPD signaling file). The specific method can be to obtain the encoding file identifier of the point cloud encoding file T corresponding to the point cloud density level K i corresponding to the point cloud density level K i and use it as the media file identifier of the media file resource M i where the media file resource M i refers to the media file resource obtained after performing file encapsulation processing on the point cloud encoding file T i ; subsequently, a media mapping relationship between the target media identifier of the target point cloud media, the media file identifier, and the point cloud density level K i can be created; a media signaling file can be generated according to the media mapping relationship; the media signaling file is used to instruct the user terminal to request the media file resource M included in the target point cloud media based on the point cloud density level K i ; i .

[0142] It can be understood that by dividing the point cloud density levels of the target point cloud media, the decoding end (user terminal) can select the point cloud density level, and the server can receive the media file resource acquisition request carrying the target point cloud density level sent by the user terminal. According to this media file resource acquisition request, the server can obtain the target media file resource corresponding to the target point cloud density level and return the target media file resource to the user terminal, so that the user terminal can decode the target media file resource to obtain the display data content with the media quality characterized by the target point cloud density level and output the display data content.

[0143] In the embodiment of the present application, by dividing the point cloud density levels of the point cloud media on the server side (encoding end) of the point cloud media, one point cloud density level corresponds to point cloud data of a data volume, and thus one point cloud density level corresponds to one media quality; the server side performs encoding and encapsulation processing on the corresponding point cloud data according to different point cloud density levels, so as to obtain the media file resources corresponding to different point cloud density levels, and one media file resource corresponds to one media quality. Therefore, the user of the user terminal (decoding end) can select the point cloud density level of the target point cloud media according to his own needs, so as to dynamically meet the media quality requirements of the user terminal (decoding end) for the point cloud media, support the user to select different point cloud density levels based on the current network state, and can select a lower point cloud density level when the network is poor. Thus, when the server side transmits data to the user terminal, only the corresponding data (the part of the point cloud data included in the media file resource) corresponding to the lower point cloud density level needs to be transmitted, which can save the bandwidth in the transmission and avoid too long data transmission time; at the same time, because the user terminal only needs to decode the media file resource that receives the corresponding data satisfying the media quality requirements and does not need to decode the complete point cloud data of the point cloud media, the decoding efficiency can be improved. In summary, the present application can dynamically adapt to user needs, save the point cloud media transmission bandwidth, reduce the data transmission time, and improve the decoding efficiency.

[0144] Further, please refer to Figure 5 , Figure 5 which is a schematic flowchart of a data processing method for point cloud media provided by an embodiment of the present application. This method can be executed by a decoding device in the point cloud media system, and the decoding device can be a user terminal. This method can include the following steps:

[0145] Step S201: Based on at least two point cloud density levels corresponding to the target point cloud media, obtain the target media file resource associated with the target point cloud density level; the target point cloud density level is used to characterize the media quality of the target point cloud media; the media file resources corresponding to at least two point cloud density levels respectively include the target media file resource, and each media file resource is obtained by performing file encapsulation processing on the point cloud encoding file corresponding to its own point cloud density level.

[0146] In this application, the user can select the point cloud density level of the target point cloud media through the user terminal. The user terminal can respond to the user's point cloud density level selection operation (selecting the target point cloud density level) and obtain the media signaling file; wherein, the media signaling file can refer to the one sent by the encoding end (server) in the corresponding embodiment above. Figure 4 The media signaling file records the target media identifier of the target point cloud media, the point cloud density level of the target point cloud media, and the media file resources corresponding to each point cloud density level respectively. The user terminal can determine which media resource corresponds to the target point cloud density level selected by the user according to the media signaling file, so as to generate a media file resource acquisition request carrying the target point cloud density level; subsequently, the user terminal can send the media file resource acquisition request to the server, and the server can obtain the corresponding target media file resource according to the target point cloud density level and return it to the user terminal. Among them, the target media file resource is the file obtained by the server after encoding and encapsulating the target point cloud data, and the target point cloud data refers to the point cloud data in the target point cloud media that matches the target point cloud density level. Regarding the specific implementation method for the server to obtain the target media file resource, reference can be made to the Figure 4 specific description of the server obtaining the media file resource M corresponding to the point cloud density level K i in the corresponding embodiment above. i

[0147] Step S202: Obtain the file encapsulation data box; the file encapsulation data box includes the at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that there is a point cloud encoding file with a data dependency relationship for the i-th point cloud encoding file of the target point cloud media, the file identification information indicates the file identification of the point cloud encoding file having the data dependency relationship with the i-th point cloud encoding file, where i is a positive integer and 1 ≤ i ≤ N.

[0148] In this application, after the user terminal obtains the target media file resource, it can also obtain the file encapsulation data box corresponding to the target media file resource. The file encapsulation data box contains the file identification information and file dependency information corresponding to the target media file resource. Through the file dependency information and the file identification, the user terminal can decode the target media file resource.

[0149] Step S203: Decode the target media file resource according to the file encapsulation data box to obtain the display data content with the media quality characterized by the target point cloud density level, and output the display data content.

[0150] In this application, if the file dependency information in the file encapsulation data box contains a first dependency identifier, the file identification information in the file encapsulation data box can be obtained, and the associated media file resource can be determined according to the associated media file identifier included in the file identification information. The associated media file resource is the file obtained after the server encodes and encapsulates the associated point cloud data. The associated point cloud data refers to the point cloud data in the target point cloud media that matches the associated point cloud density level. The media quality characterized by the associated point cloud density level is lower than the media quality characterized by the target point cloud density level. Subsequently, according to the file encapsulation data box, the associated media file resource and the target media file resource can be respectively unpacked to obtain the associated point cloud encoding file corresponding to the associated media file resource and the target point cloud encoding file corresponding to the target media file resource. Subsequently, the associated point cloud encoding file and the target point cloud encoding file can be respectively decoded to obtain the first display data content corresponding to the associated point cloud encoding file and the second display data content corresponding to the target point cloud encoding file. By fusing the first display data content and the second display data content, the display data content with the media quality characterized by the target point cloud density level can be obtained.

[0151] It should be understood that the user terminal can determine the media file resource corresponding to the target point cloud density level through the media file identifier corresponding to the target media file resource at the target point cloud density level in the media signaling file, so as to generate a resource acquisition request. After receiving the target media file resource sent by the server, the user terminal can obtain the file encapsulation data box corresponding to the target media file resource, and determine whether the target media file resource needs to be jointly decoded with other media file resources according to the file dependency information of the target media file resource in the file encapsulation data box. If so, the media file identifier of the media file resource having a data dependency relationship with the target media file resource can be obtained from the file identifier information in the file encapsulation data box, so that the media file resource corresponding to the media file identifier can be determined as the associated media file resource. Thus, the user terminal can jointly decode the target media file resource and the associated media file resource.

[0152] For example, taking the above Figure 2 and Figure 3 corresponding embodiment as an example, since the server encodes and encapsulates the incremental point cloud data 100d to obtain the media file resource 3, when the user clicks on the point cloud density level 2, the user terminal obtains that the media file resource corresponding to the point cloud density level 2 in the media signaling file is the media file resource 3. Then the user terminal can generate an acquisition request for the media file resource 3 and send the acquisition request for the media file resource 3 to the server. After receiving the acquisition request, the server can obtain the media file resource 3 and determine from the file encapsulation data box of the media file resource 3 that the file dependency information corresponding to the media file resource 3 contains a first dependency identifier (point_cloud_dense_dependency_flag = 1, and the first dependency identifier is 1). Then the server can determine that there is a media file resource having a data dependency relationship with it for the media file resource 3. Then the server can obtain the media file identifier (the file identifier of the media file resource 2) of the media file resource having a data dependency relationship with the media file resource 3 in the file identifier information (TrackReferenceTypeBox field) in the file encapsulation data box. Through this media file identifier, it can be determined that the media file resource having a data dependency relationship with the media file resource 3 is the media file resource 2. Subsequently, the server can query whether there is an acquisition request for the media file resource 2 from this user terminal in the request record (determine whether there is a media file resource 2 in the user terminal). If there is no acquisition request for the media file resource 2 from this user terminal in the request record, the server can obtain the media file resource 2 and return the media file resource 2 and the media file resource 3 to the user terminal together.

[0153] Subsequently, after the user terminal receives the media file resource 3 and the media file resource 2, it can, according to the file dependency information and file identification information in the file encapsulation data box of the media file resource 3, separately perform decompression processing on the media file resource 2 and the media file resource 3 returned by the server to obtain the point cloud encoding file 2 and the point cloud encoding file 3. By separately decoding the point cloud encoding file 2 and the point cloud encoding file 3, the point cloud data 100b and the incremental point cloud data 100d can be restored. By fusing the point cloud data 100b and the incremental point cloud data 100d, the point cloud data 100c corresponding to the point cloud density level 2 can be obtained.

[0154] In this application, if the file dependency information in the file encapsulation data box contains a second dependency identifier, the decompression processing can be directly performed on the target media file resource according to the file encapsulation data box to obtain the target point cloud encoding file corresponding to the target media file resource; subsequently, the target point cloud encoding file can be decoded to obtain the display data content with the media quality characterized by the target point cloud density level.

[0155] It should be understood that the user terminal can determine whether the target media file resource needs to be jointly decoded with other media file resources through the file dependency information of the target media file resource corresponding to the target point cloud density level in the file encapsulation data box. If the file dependency information contains a second dependency identifier, it can be determined that the target media file resource has no data dependency relationship with other media file resources, that is, it does not need to be jointly decoded with other media file resources. Then the user terminal can directly decode the target file resource to obtain the display data content.

[0156] For example, as described above Figure 2Taking the corresponding embodiment as an example, since the server encodes and encapsulates the point cloud data 100a to obtain the media file resource 1, when the user clicks on the point cloud density level 0, the user terminal obtains the media signaling file, and the media file resource corresponding to the point cloud density level 0 is the media file resource 1. Then the user terminal can generate a request for obtaining the media file resource 1 and send the request for obtaining the media file resource 1 to the server. After receiving the request, the server can obtain the media file resource 1 and determine from the file encapsulation data box of the media file resource 1 that the file dependency information corresponding to the media file resource 1 contains a second dependency identifier (point_cloud_dense_dependency_flag = 0, and the second dependency identifier is 0). Then the server can determine that there is no media file resource that has a data dependency relationship with the media file resource 1, and the server can return the media file resource 1 to the user terminal. The user terminal can perform decompression processing on the media file resource 1 returned by the server to obtain the point cloud encoding file 1, and decode the point cloud encoding file 1 to restore the point cloud data 100a.

[0157] In the embodiment of the present application, by dividing the point cloud density level of the point cloud media on the server side (encoding side) of the point cloud media, one point cloud density level corresponds to the point cloud data of a data volume, and one point cloud density level corresponds to a media quality; the server side encodes and encapsulates the corresponding point cloud data according to different point cloud density levels, so as to obtain the media file resources corresponding to different point cloud density levels, and one media file resource corresponds to a media quality. Thus, the user of the user terminal (decoding side) can select the point cloud density level of the target point cloud media according to their own needs, so as to dynamically meet the media quality requirements of the user terminal (decoding side) for the point cloud media, support the user to select different point cloud density levels based on the current network status, and can select a lower point cloud density level when the network is poor. Thus, when the server side transmits data to the user terminal, only the corresponding data (the part of the point cloud data included in the media file resource) corresponding to the lower point cloud density level needs to be transmitted, which can save the bandwidth in the transmission and avoid the data transmission time from being too long; at the same time, since the user terminal only needs to decode the media file resource that receives the corresponding data that meets the media quality requirements and does not need to decode the complete point cloud data of the point cloud media, the decoding efficiency can be improved. In summary, the present application can dynamically adapt to user needs, save the transmission bandwidth of the point cloud media, reduce the data transmission time, and improve the decoding efficiency.

[0158] Optionally, it can be understood that on the server (encoding side) for the point cloud data S i (which can be the point cloud data S ior the incremental point cloud data) to perform encoding and encapsulation processing to obtain the media file resource M i After that (for example, after performing encoding and encapsulation processing on the incremental point cloud data 100d to obtain the media file resource 3, or after performing encoding and encapsulation processing on the point cloud data 100a to obtain the media file resource 1), the file dependency information of the media file resource 3 in the ISO file encapsulation data box can be set to the first dependency identifier, and the media file identifier of the media file resource 2 can be added to the file identifier information; similarly, the file dependency information of the media file resource 1 can be set to the second dependency identifier, and the media file identifier can be set to an invalid value (for example, null value) in the file identifier information; these file dependency information and file identifier information can be written into the media signaling file, so that the user terminal (decoding end) can know through the media signaling file which point cloud density levels the point cloud media (such as point cloud media A) has, and then determine whether the media file resource needs to be jointly decoded with other media file resources through the file dependency information (point_cloud_dense_dependency_flag field). If necessary (the file dependency information contains the first dependency identifier), the media file resource that has a data dependency relationship with it can be determined through the file identifier information (TrackReferenceTypeBox field) in the media signaling file.

[0159] Thus, the user terminal can generate a resource acquisition request for the media file resource according to the media signaling file. For example, if the point cloud density level clicked by the user terminal is the point cloud density level 2, and its corresponding media file resource is the media file resource 3, the user terminal can then determine through the file dependency information and file identifier information in the media signaling file that the media file resource 3 needs to be jointly decoded with the media file resource 2. Then the user terminal can query whether the media file resource 2 exists locally. If not, it can generate an acquisition request for the media file resource 2 and the media file resource 3; if it exists, it can generate an acquisition request only for the media file resource 3. After receiving the acquisition request, the server can return the corresponding media file resource.

[0160] Further, please refer to Figure 6 , Figure 6 is a schematic structural diagram of a data processing device for point cloud media provided by an embodiment of the present application. The data processing device for point cloud media can be a computer program (including program code) running in an encoding device. For example, the data processing device for point cloud media can be an application software in the encoding device. The data processing device for point cloud media can be used to execute Figure 4 the steps of the data processing method for point cloud media in the corresponding embodiment, please refer to Figure 6, the data processing device 1 of the point cloud media may include: a media encoding module 11, a data box generation module 12, and a file encapsulation module 13.

[0161] The media encoding module 11 is configured to perform point cloud encoding on the target point cloud media according to at least two point cloud density levels to obtain at least two point cloud encoding files; the point cloud density levels are used to characterize the media quality of the target point cloud media;

[0162] The data box generation module 12 is configured to generate a file encapsulation data box of the target point cloud media according to at least two point cloud density levels and at least two point cloud encoding files; the file encapsulation data box includes at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that the i-th point cloud encoding file of the target point cloud media has a point cloud encoding file with a data dependency relationship, the file identification information includes the file identification of the point cloud encoding file having a data dependency relationship with the i-th point cloud encoding file, where i is a positive integer less than or equal to the number of at least two point cloud encoding files;

[0163] The file encapsulation module 13 is configured to perform file encapsulation processing on at least two point cloud encoding files respectively according to the file encapsulation data box to obtain media file resources corresponding to at least two point cloud density levels respectively.

[0164] Among them, for the specific implementation manners of the media encoding module 11, the data box generation module 12, and the file encapsulation module 13, reference may be made to the descriptions in steps S101 - S103 in the corresponding embodiments above, and details will not be elaborated here. Figure 4 The description in steps S101 - S103 of the corresponding embodiments above, and details will not be elaborated here.

[0165] Among them, at least two point cloud density levels are represented as Ki; i is an integer greater than or equal to 1;

[0166] Please refer to Figure 6 , the media encoding module 11 may include: an associated density acquisition unit 111, an associated data acquisition unit 112, and an incremental data encoding unit 113.

[0167] The associated density acquisition unit 111 is configured to acquire the associated point cloud density level corresponding to the point cloud density level K i ; the media quality characterized by the associated point cloud density level is lower than the media quality characterized by the point cloud density level K i ;

[0168] The associated data acquisition unit 112 is configured to acquire the point cloud data S i corresponding to the point cloud density level K i , and the associated point cloud data corresponding to the associated point cloud density level; the point cloud data S i refers to the point cloud data in the target point cloud media corresponding to the point cloud density level Ki Matched point cloud data; the associated point cloud data refers to the point cloud data in the target point cloud media that matches the associated point cloud density level; the data volume of the associated point cloud data is lower than the data volume of the point cloud data S i of the data;

[0169] The incremental data encoding unit 113 is used to determine the incremental point cloud data between the associated point cloud data and the point cloud data S i and encode the incremental point cloud data to obtain the point cloud encoding file T corresponding to the point cloud density level K i i .

[0170] Among them, for the specific implementation manners of the associated density acquisition unit 111, the associated data acquisition unit 112, and the incremental data encoding unit 113, reference may be made to the description in step S101 of the corresponding embodiment above, and details will not be elaborated here. Figure 4

[0171] Please refer to Figure 6 , the data box generation module 12 may include: an identifier acquisition unit 121 and an identifier addition unit 122.

[0172] The identifier acquisition unit 121 is used to acquire a first dependency identifier; the first dependency identifier is used to represent that there is a point cloud encoding file with a data dependency relationship for the point cloud encoding file T i

[0173] The identifier addition unit 122 is used to add the first dependency identifier to the file dependency information in the file encapsulation data box;

[0174] The identifier addition unit 122 is further used to acquire the encoding file identifier of the associated point cloud encoding file and add the encoding file identifier of the associated point cloud encoding file to the file identifier information in the file encapsulation data box; the associated point cloud encoding file refers to the point cloud encoding file obtained by performing point cloud encoding on the associated point cloud data.

[0175] Among them, for the specific implementation manners of the identifier acquisition unit 121 and the identifier addition unit 122, reference may be made to the description in step S102 of the corresponding embodiment above, and details will not be elaborated here. Figure 4

[0176] Among them, at least two point cloud density levels are represented as Ki; i is an integer greater than or equal to 1;

[0177] Please refer to Figure 6 , the media encoding module 11 may include: a point cloud data encoding unit 114.

[0178] The point cloud data encoding unit 114 is used to acquire the point cloud density level K i ​​​​The corresponding point cloud data S i , for the point cloud data S i perform point cloud encoding to obtain the point cloud density level K i The corresponding point cloud encoding file T i ; The point cloud data S i refers to the point cloud data in the target point cloud media that matches the point cloud density level K i .

[0179] Among them, for the specific implementation manner of the point cloud data encoding unit 114, reference can be made to the description in step S101 of the corresponding embodiment above, and details will not be elaborated here. Figure 4

[0180] Please refer to Figure 6 , the data box generation module 12 may include: a dependency information acquisition unit 123 and an information addition unit 124.

[0181] The dependency information acquisition unit 123 is used to acquire a second dependency identifier, and the second dependency identifier is used to characterize that there is no point cloud encoding file with a data dependency relationship for the point cloud encoding file T i ;

[0182] The information addition unit 124 is used to add the second dependency identifier to the file dependency information in the file encapsulation data box;

[0183] The information addition unit 124 is further used to set the encoding file identifier associated with the point cloud encoding file T i in the file identifier information of the file encapsulation data box to an invalid value.

[0184] Among them, for the specific implementation manner of the dependency information acquisition unit 123 and the information addition unit 124, reference can be made to the description in step S102 of the corresponding embodiment above, and details will not be elaborated here. Figure 4

[0185] Please refer to Figure 6 , the apparatus 1 may further include: a media file identifier determination module 14, a mapping relationship creation module 15, and a signaling file generation module 16.

[0186] The media file identifier determination module 14 is used to acquire the encoding file identifier of the point cloud encoding file T i corresponding to i as the media file identifier of the media file resource M i ; The media file resource M i refers to the media file resource obtained after performing file encapsulation processing on the point cloud encoding file T i ;

[0187] ​​A mapping relationship creation module 15 for creating a media mapping relationship between the target media identifier, media file identifier, and point cloud density level K of the target point cloud media; i between;

[0188] A signaling file generation module 16 for generating a media signaling file according to the media mapping relationship; the media signaling file is used to instruct the user terminal to request the media file resource M included in the target point cloud media based on the point cloud density level K i Request the media file resource M included in the target point cloud media i .

[0189] Among them, for the specific implementation manners of the media file identifier determination module 14, the mapping relationship creation module 15, and the signaling file generation module 16, reference can be made to the description in step S103 of the corresponding embodiment above, which will not be elaborated here. Figure 4 Please refer to

[0190] Please refer to Figure 6 , the apparatus 1 may further include: a request receiving module 17, a file resource obtaining module 18, and a resource returning module 19.

[0191] The request receiving module 17 is configured to receive a media file resource acquisition request sent by the user terminal; the media file resource acquisition request is a request generated by the user terminal in response to a point cloud density level selection operation for the target point cloud media; the media file resource acquisition request carries the target point cloud density level, and the target point cloud density level is the point cloud density level selected by the point cloud density level selection operation;

[0192] The file resource obtaining module 18 is configured to obtain the target media file resource corresponding to the target point cloud density level according to the media file resource acquisition request;

[0193] The resource returning module 19 is configured to return the target media file resource to the user terminal, so that the user terminal decodes the target media file resource to obtain display data content with the media quality characterized by the target point cloud density level, and outputs the display data content.

[0194] Among them, for the specific implementation manners of the request receiving module 17, the file resource obtaining module 18, and the resource returning module 19, reference can be made to the description in step S103 of the corresponding embodiment above, which will not be elaborated here. Figure 4 Please refer to

[0195] According to an embodiment of the present invention, Figure 6Each module or unit in the data processing device for point cloud media shown can be separately or all combined into one or several other units to form, or a certain one (or some) of the units can be further split into multiple smaller units with more specific functions to form, which can achieve the same operations without affecting the realization of the technical effects of the embodiments of the present invention. The above units are divided based on logical functions. In practical applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present application, the data processing device for point cloud media can also include other units. In practical applications, these functions can also be assisted by other units and can be realized through the cooperation of multiple units. According to another embodiment of the present application, it can be achieved by running a computer program (including program code) capable of executing the respective steps involved in the corresponding method shown in Figure 4 on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM), to construct a data processing device for point cloud media as shown in Figure 4 and to implement the data processing method for point cloud media of the embodiments of the present application. The computer program can be recorded on, for example, a computer-readable recording medium, loaded into the above computing device through the computer-readable recording medium, and run therein.

[0196] In the embodiments of the present application, by dividing the point cloud density levels of the point cloud media on the server side (encoding side) of the point cloud media, one point cloud density level corresponds to point cloud data of a certain data volume, and thus one point cloud density level corresponds to one media quality; the server side performs encoding and encapsulation processing on the corresponding point cloud data according to different point cloud density levels, so that media file resources corresponding to different point cloud density levels can be obtained, and one media file resource corresponds to one media quality. Thus, the user of the user terminal (decoding side) can select the point cloud density level of the target point cloud media according to their own needs, thereby dynamically meeting the media quality requirements of the user terminal (decoding side) for the point cloud media, supporting the user to select different point cloud density levels based on the current network status, and can select a lower point cloud density level when the network is poor. Thus, when the server side transmits data to the user terminal, only the corresponding data (the part of the point cloud data included in the media file resource) corresponding to the lower point cloud density level needs to be transmitted, which can save bandwidth in transmission and avoid excessive data transmission time; at the same time, since the user terminal only needs to decode the media file resource containing the corresponding data that meets the media quality requirements and does not need to decode the complete point cloud data of the point cloud media, the decoding efficiency can be improved. In summary, it can be seen that the present application can dynamically adapt to user needs, save the transmission bandwidth of point cloud media, reduce data transmission time, and improve decoding efficiency.

[0197] Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of another data processing device for point cloud media provided by an embodiment of the present application. The data processing device for point cloud media may be a computer program (including program code) running in a decoding device. For example, the data processing device for point cloud media may be an application software in the decoding device. The data processing device for point cloud media may be used to execute Figure 5 the steps of the data processing method for point cloud media in the corresponding embodiment. Please refer to Figure 7 , the data processing device 2 for point cloud media may include: a resource acquisition module 2001, a data box acquisition module 2002, and a resource decoding module 2003.

[0198] The resource acquisition module 2001 is configured to acquire target media file resources associated with a target point cloud density level based on at least two point cloud density levels corresponding to the target point cloud media; the target point cloud density level is used to characterize the media quality of the target point cloud media; the media file resources corresponding to at least two point cloud density levels respectively include the target media file resources, and each media file resource is obtained by performing file encapsulation processing on the point cloud encoding file corresponding to the point cloud density level to which it belongs;

[0199] The data box acquisition module 2002 is configured to acquire a file encapsulation data box; the file encapsulation data box includes at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that the i-th point cloud encoding file of the target point cloud media has a point cloud encoding file with a data dependency relationship, the file identification information indicates the file identification of the point cloud encoding file having a data dependency relationship with the i-th point cloud encoding file, where i is a positive integer and 1 ≤ i ≤ N; N is the number of point cloud encoding files included in the target point cloud media;

[0200] The resource decoding module 2003 is configured to decode the target media file resources according to the file encapsulation data box to obtain display data content with the media quality characterized by the target point cloud density level, and output the display data content.

[0201] Among them, for the specific implementation manners of the resource acquisition module 2001, the data box acquisition module 2002, and the resource decoding module 2003, reference may be made to the descriptions of steps S201 - S203 in the corresponding embodiment above, which will not be elaborated here. Figure 5

[0202] Please refer to Figure 7 , the resource acquisition module 2001 may include: a signaling file acquisition unit 20011, a request generation unit 20012, a request sending unit 20013, and a resource receiving unit 20014. ​

[0203] A signaling file acquisition unit 20011, configured to acquire a media signaling file in response to a point cloud density level selection operation for a target point cloud media; the media signaling file includes a media identifier of the point cloud media, at least two point cloud density levels corresponding to the media identifier, and media file identifiers of media file resources corresponding to the at least two point cloud density levels;

[0204] A request generation unit 20012, configured to generate a media file resource acquisition request according to the media signaling file; the media file resource acquisition request carries a target point cloud density level; the target point cloud density level is the point cloud density level selected from the at least two point cloud density levels in the point cloud density level selection operation;

[0205] A request sending unit 20013, configured to send the media file resource acquisition request to a server;

[0206] A resource receiving unit 20014, configured to receive a target media file resource returned by the server according to the media file resource acquisition request.

[0207] Among them, for the specific implementation manners of the signaling file acquisition unit 20011, the request generation unit 20012, the request sending unit 20013, and the resource receiving unit 20014, reference may be made to the description in step S201 in the corresponding embodiment above, which will not be elaborated here. Figure 5 For details, please refer to

[0208] Please refer to Figure 7 , the resource decoding module 2003 may include: an associated file acquisition unit 20031 and a joint decoding unit 20032.

[0209] An associated file acquisition unit 20031, configured to, if the file dependency information in the file encapsulation data box contains a first dependency identifier, determine an associated media file resource according to the associated media file identifier included in the file identifier information in the file encapsulation data box; the associated media file identifier is the media file identifier of an associated media file resource having a data dependency relationship with the target media file resource; the associated media file resource is a file obtained by the server after encoding and encapsulating the associated point cloud data; the associated point cloud data refers to the point cloud data in the target point cloud media that matches the associated point cloud density level; the media quality characterized by the associated point cloud density level is lower than the media quality characterized by the target point cloud density level;

[0210] A joint decoding unit 20032, configured to perform joint decoding on the associated media file resource and the target media file resource to obtain a display data content having the media quality characterized by the target point cloud density level.

[0211] Among them, for the specific implementation manners of the associated file acquisition unit 20031 and the joint decoding unit 20032, reference may be made to the description in step S203 in the corresponding embodiment above, which will not be elaborated here. Figure 5 The description in step S203 in the corresponding embodiment above will not be elaborated here.

[0212] Please refer to Figure 7 , the joint decoding unit 20032 may include: a decompression sub-unit 200321, a decoding sub-unit 200322, and a data fusion sub-unit 200323.

[0213] The decompression sub-unit 200321 is configured to perform decompression processing on the associated media file resource and the target media file resource respectively according to the file encapsulation data box, so as to obtain the associated point cloud encoding file corresponding to the associated media file resource and the target point cloud encoding file corresponding to the target media file resource;

[0214] The decoding sub-unit 200322 is configured to perform decoding on the associated point cloud encoding file and the target point cloud encoding file respectively, so as to obtain the first display data content corresponding to the associated point cloud encoding file and the second display data content corresponding to the target point cloud encoding file;

[0215] The data fusion sub-unit 200323 is configured to fuse the first display data content and the second display data content to obtain the display data content with the media quality characterized by the target point cloud density level.

[0216] Among them, for the specific implementation manners of the decompression sub-unit 200321, the decoding sub-unit 200322, and the data fusion sub-unit 200323, reference may be made to the description in step S203 in the corresponding embodiment above, which will not be elaborated here. Figure 5 The description in step S203 in the corresponding embodiment above will not be elaborated here.

[0217] Please refer to Figure 7 , the resource decoding module 2003 may include: a decompression unit 20033 and a data decoding unit 20034.

[0218] The decompression unit 20033 is configured to, if the file dependency information in the file encapsulation data box includes a second dependency identifier, perform decompression processing on the target media file resource according to the file encapsulation data box, so as to obtain the target point cloud encoding file corresponding to the target media file resource;

[0219] The data decoding unit 20034 is configured to perform decoding on the target point cloud encoding file to obtain the display data content with the media quality characterized by the target point cloud density level.

[0220] Among them, for the specific implementation manners of the decompression unit 20033 and the data decoding unit 20034, reference may be made to the description in the corresponding embodiment above, Figure 5The description in step S203 of the corresponding embodiment will not be repeated here.

[0221] According to an embodiment of the present invention, Figure 7 Each unit in the data processing device of the point cloud media shown can be respectively or wholly combined into one or several other units to form, or a certain (some) unit can also be further split into multiple smaller units in terms of function to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above units are divided based on logical functions. In practical applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present application, the data processing device of the point cloud media may also include other units. In practical applications, these functions can also be assisted by other units and can be realized through the cooperation of multiple units. According to another embodiment of the present application, it is possible to run a computer program (including program code) capable of executing the respective steps involved in the corresponding method shown Figure 5 on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM), to construct the data processing device of the point cloud media shown Figure 5 and to implement the data processing method of the point cloud media in the embodiments of the present application. The computer program can be recorded on, for example, a computer-readable recording medium, loaded into the above computing device through the computer-readable recording medium, and run therein.

[0222] In the embodiments of the present application, by dividing the point cloud density levels of point cloud media on the server side (encoding side) of the point cloud media, one point cloud density level corresponds to point cloud data with a certain amount of data, and thus one point cloud density level corresponds to one media quality; the server side performs encoding and encapsulation processing on the corresponding point cloud data according to different point cloud density levels, so as to obtain media file resources corresponding to different point cloud density levels, and one media file resource corresponds to one media quality. Therefore, the user of the user terminal (decoding side) can select the point cloud density level of the target point cloud media according to their own needs, so as to dynamically meet the media quality requirements of the user terminal (decoding side) for the point cloud media, support the user to select different point cloud density levels based on the current network status, and can select a lower point cloud density level when the network is poor. Thus, when the server side transmits data to the user terminal, only the corresponding data (the part of the point cloud data included in the media file resource) corresponding to the lower point cloud density level needs to be transmitted, which can save bandwidth during transmission and avoid excessive data transmission time; at the same time, because the user terminal only needs to decode the media file resource containing the corresponding data that meets the media quality requirements, and does not need to decode the complete point cloud data of the point cloud media, the decoding efficiency can be improved. In summary, the present application can dynamically adapt to user needs, save the transmission bandwidth of point cloud media, reduce data transmission time, and improve decoding efficiency.

[0223] Please refer to Figure 8 , Figure 8 is a schematic structural diagram of an encoding device provided by an embodiment of the present application. The encoding device may refer to the computer device used by the provider of the point cloud media, and the computer device may be a terminal (such as a PC, a smart mobile device (such as a smart phone), etc.) or a server. As Figure 8 shown, the encoding device includes a capture device 801, a processor 802, a memory 803, and a transmitter 804. Among them:

[0224] The capture device 801 is used to collect the sound-visual scene of the real world to obtain the original data of the point cloud media (including audio content and video content that are synchronized in time and space). The capture device 801 may include, but is not limited to: audio devices, imaging devices, and sensing devices. Among them, the audio devices may include audio sensors, microphones, etc. The imaging devices may include ordinary cameras, stereo cameras, light field cameras, etc. The sensing devices may include laser devices, radar devices, etc.

[0225] The processor 802 (or CPU (Central Processing Unit, central processor)) is the processing core of the encoding device. The processor 802 is adapted to implement one or more program instructions, and is specifically adapted to load and execute one or more program instructions to implement Figure 4The process of the data processing method for the point cloud media shown.

[0226] The memory 803 is a memory device in the encoding device, used to store programs and media resources. It can be understood that the memory 803 here can include both the built-in storage medium in the encoding device and, of course, the extended storage medium supported by the encoding device. It should be noted that the memory can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one memory located far from the aforementioned processor. The memory provides storage space, which is used to store the operating system of the encoding device. And, in this storage space, it is also used to store a computer program, which includes program instructions, and these program instructions are suitable for being called and executed by the processor to execute the various steps of the data processing method for the point cloud media. In addition, the memory 803 can also be used to store the point cloud media file formed after being processed by the processor, and this point cloud media file includes media file resources and media presentation description information.

[0227] The transmitter 804 is used to realize the transmission interaction between the encoding device and other devices, specifically used to realize the transmission of the point cloud media between the encoding device and the decoding device. That is, the encoding device transmits the relevant media resources of the point cloud media to the decoding device through the transmitter 804.

[0228] Please refer to again Figure 8 , the processor 802 may include an encoder 821 and a packager 822; where:

[0229] The encoder 821 is used to perform audio encoding on the captured audio content to form an audio bitstream of the point cloud media. It is also used to perform video encoding on the captured video content to obtain a video bitstream.

[0230] The packager 822 is used to package the audio bitstream and the video bitstream in a file container according to the file packaging format of the point cloud media (such as the ISO file packaging format) to form the media file resources of the point cloud media. This media file resource can be a media file or a media segment to form the media file of the point cloud media; and record the metadata of the media file resources of the point cloud media according to the requirements of the file format of the point cloud media using the media presentation description information. The packaged file of the point cloud media obtained by the packager will be saved in the memory and provided to the decoding device for presenting the point cloud media as needed.

[0231] In one embodiment, the processor 802 (specifically, each component included in the processor) executes by calling one or more instructions in the memory Figure 4Steps of the data processing method for the point cloud media shown. Specifically, the memory 803 stores one or more first instructions, and the one or more first instructions are adapted to be loaded and executed by the processor 802 to perform the following steps:

[0232] Perform point cloud encoding on the target point cloud media according to at least two point cloud density levels to obtain at least two point cloud encoded files; the point cloud density level is used to characterize the media quality of the target point cloud media;

[0233] Generate a file encapsulation data box for the target point cloud media according to at least two point cloud density levels and at least two point cloud encoded files; the file encapsulation data box includes at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that the i-th point cloud encoded file of the target point cloud media has a point cloud encoded file with a data dependency relationship, the file identification information includes the file identification of the point cloud encoded file with a data dependency relationship with the i-th point cloud encoded file, where i is a positive integer less than or equal to the number of at least two point cloud encoded files;

[0234] Perform file encapsulation processing on at least two point cloud encoded files respectively according to the file encapsulation data box to obtain media file resources corresponding to at least two point cloud density levels respectively.

[0235] In the embodiments of the present application, by dividing the point cloud density levels of the point cloud media at the server side (encoding side) of the point cloud media, one point cloud density level corresponds to point cloud data of a data volume, and thus one point cloud density level corresponds to one media quality; the server side performs encoding and encapsulation processing on the corresponding point cloud data according to different point cloud density levels, so that media file resources corresponding to different point cloud density levels can be obtained, and one media file resource corresponds to one media quality. Therefore, the user of the user terminal (decoding side) can select the point cloud density level of the target point cloud media according to their own needs, so as to dynamically meet the media quality requirements of the user terminal (decoding side) for the point cloud media, support the user to select different point cloud density levels based on the current network state, and can select a lower point cloud density level when the network is poor. Thus, when the server side transmits data to the user terminal, only the corresponding data (the part of the point cloud data included in the media file resource) corresponding to the lower point cloud density level needs to be transmitted, which can save bandwidth in transmission and avoid too long data transmission time; at the same time, because the user terminal only needs to decode the received media file resource containing the corresponding data that meets the media quality requirements, without decoding the complete point cloud data of the point cloud media, the decoding efficiency can be improved. In summary, the present application can dynamically adapt to user needs, save the transmission bandwidth of the point cloud media, reduce the data transmission time, and improve the decoding efficiency.

[0236] Please refer to Figure 9 ,Figure 9 This is a schematic structural diagram of a decoding device provided by an embodiment of the present application. The decoding device can be a computer device used by a user of point cloud media, and the computer device can be a terminal (such as a PC, a smart mobile device (such as a smart phone), a VR device (such as a VR helmet, VR glasses, etc.)). As Figure 9 shown, the decoding device includes a receiver 901, a processor 902, a memory 903, and a display / playback device 904. Among them:

[0237] The receiver 901 is used to implement the transmission interaction between decoding and other devices, specifically for implementing the transmission of point cloud media between the encoding device and the decoding device. That is, the decoding device receives the relevant media file resources of the point cloud media transmitted by the encoding device through the receiver 901.

[0238] The processor 902 (or CPU (Central Processing Unit)) is the processing core of the encoding device. The processor 902 is adapted to implement one or more program instructions, specifically adapted to load and execute one or more program instructions to implement Figure 5 the process of the data processing method of the point cloud media shown.

[0239] The memory 903 is a memory device in the decoding device, used to store programs and media resources. It can be understood that the memory 903 here can include both the built-in storage medium in the decoding device, and of course can also include the extended storage medium supported by the decoding device. It should be noted that the memory 903 can be a high-speed RAM memory, or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one memory located far from the aforementioned processor. The memory 903 provides a storage space, which is used to store the operating system of the decoding device. And, in this storage space, it is also used to store a computer program, the computer program includes program instructions, and the program instructions are adapted to be called and executed by the processor to execute the steps of the data processing method of the point cloud media. In addition, the memory 903 can also be used to store the three-dimensional image of the point cloud media formed after being processed by the processor, the audio content corresponding to the three-dimensional image, and the information required for rendering the three-dimensional image and audio content, etc.

[0240] The display / playback device 904 is used to output the rendered sound and image.

[0241] Please refer to Figure 9 again. The processor 902 may include a parser 921, a decoder 922, and a renderer 923; among them:

[0242] The parser 921 is used to unpack the encapsulated file of the rendered media from the encoding device. Specifically, it unpacks the media file resources according to the file encapsulation format requirements of the point cloud media to obtain the audio bitstream and the video bitstream (point cloud encoded file); and provides the audio bitstream and the video bitstream to the decoder 922.

[0243] The decoder 922 performs audio decoding on the audio bitstream to obtain the audio content and provides it to the renderer for audio rendering. In addition, the decoder 922 decodes the video bitstream to obtain images.

[0244] The renderer 923 is used to render the audio content and images of the point cloud media. Specifically, it renders the audio content and images according to the metadata related to rendering and window in the media presentation description information, and after the rendering is completed, it is handed over to the display / playback device for output.

[0245] In an exemplary embodiment, the processor 902 (specifically, each device included in the processor) executes Figure 5 each step of the data processing method of the point cloud media shown. Specifically, the memory stores one or more first instructions, and the one or more first instructions are adapted to be loaded and executed by the processor 902 to perform the following steps:

[0246] Based on at least two point cloud density levels corresponding to the target point cloud media, obtain the target media file resources associated with the target point cloud density level; the target point cloud density level is used to characterize the media quality of the target point cloud media; the media file resources corresponding to at least two point cloud density levels respectively include the target media file resources, and each media file resource is obtained by performing file encapsulation processing on the point cloud encoded file corresponding to the point cloud density level to which it belongs;

[0247] Obtain the file encapsulation data box; the file encapsulation data box includes at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that the i-th point cloud encoded file of the target point cloud media has a point cloud encoded file with a data dependency relationship, the file identification information indicates the file identification of the point cloud encoded file having a data dependency relationship with the i-th point cloud encoded file, where i is a positive integer and 1 ≤ i ≤ N; N is the number of point cloud encoded files included in the target point cloud media;

[0248] Decode the target media file resources according to the file encapsulation data box to obtain the display data content with the media quality characterized by the target point cloud density level, and output the display data content.

[0249] In the embodiments of the present application, by dividing the point cloud density levels of the point cloud media on the server side (encoding side) of the point cloud media, one point cloud density level corresponds to point cloud data of a certain amount of data, and thus one point cloud density level corresponds to one media quality. The server side performs encoding and encapsulation processing on the corresponding point cloud data according to different point cloud density levels, so as to obtain media file resources corresponding to different point cloud density levels, and one media file resource corresponds to one media quality. Therefore, the user of the user terminal (decoding side) can select the point cloud density level of the target point cloud media according to his own needs, so as to dynamically meet the media quality requirements of the user terminal (decoding side) for the point cloud media, support the user to select different point cloud density levels based on the current network status, and can select a lower point cloud density level when the network is poor. Thus, when the server side transmits data to the user terminal, only the corresponding data (the part of the point cloud data included in the media file resource) corresponding to the lower point cloud density level needs to be transmitted, which can save the bandwidth in the transmission and avoid too long data transmission time. At the same time, because the user terminal only needs to decode the media file resource that receives the corresponding data that meets the media quality requirements, without decoding the complete point cloud data of the point cloud media, the decoding efficiency can be improved. In summary, the present application can dynamically adapt to user needs, save the transmission bandwidth of the point cloud media, reduce the data transmission time, and improve the decoding efficiency.

[0250] In the description of the embodiments of the present application, the terms "first", "second", etc. in the specification, claims and drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally further includes steps or modules not listed, or optionally further includes other step units inherent to these processes, methods, devices, products or equipment.

[0251] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described according to their functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0252] The method and related devices provided by the embodiments of the present application are described with reference to the method flowcharts and / or structural schematic diagrams provided by the embodiments of the present application. Specifically, each process and / or block of the method flowchart and / or structural schematic diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or structural schematic Figure 1 one block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one process or multiple processes and / or structural schematic Figure 1 one block or multiple blocks. These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one process or multiple processes and / or structural schematic one block or multiple blocks.

[0253] The above-disclosed are only the preferred embodiments of the present application. Of course, the scope of the rights of the present application cannot be limited thereby. Therefore, equivalent changes made according to the claims of the present application still fall within the scope covered by the present application.

Claims

1. A data processing method for point cloud media, characterized in that, including: performing point cloud encoding on a target point cloud medium according to at least two point cloud density levels to obtain at least two point cloud encoded files; the point cloud density level is used to characterize the medium quality of the target point cloud medium; generating a file encapsulation data box of the target point cloud medium according to the at least two point cloud density levels and the at least two point cloud encoded files; the file encapsulation data box includes the at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that the i-th point cloud encoded file of the target point cloud medium has a point cloud encoded file with a data dependency relationship, the file identification information includes the file identification of the point cloud encoded file having the data dependency relationship with the i-th point cloud encoded file, where i is a positive integer less than or equal to the number of the at least two point cloud encoded files; performing file encapsulation processing on the at least two point cloud encoded files respectively according to the file encapsulation data box to obtain media file resources corresponding to the at least two point cloud density levels respectively.

2. The method according to claim 1, wherein The at least two point cloud density levels are represented as K i ; i is an integer greater than or equal to 1; The performing point cloud encoding on a target point cloud medium according to at least two point cloud density levels to obtain at least two point cloud encoded files includes: Obtain the point cloud density level K i The corresponding associated point cloud density level; the media quality represented by the associated point cloud density level is lower than the point cloud density level K i The represented media quality; Obtain the point cloud density level K i The corresponding point cloud data S i , and the associated point cloud data corresponding to the associated point cloud density level; the point cloud data S i refers to the point cloud data in the target point cloud media that matches the point cloud density level K i ; the associated point cloud data refers to the point cloud data in the target point cloud media that matches the associated point cloud density level; the data volume of the associated point cloud data is lower than that of the point cloud data S i ; Determine the incremental point cloud data between the associated point cloud data and the point cloud data S i Encode the incremental point cloud data to obtain the point cloud density level K i corresponding point cloud encoding file T i .

3. The method according to claim 2, wherein The generating a file encapsulation data box of the target point cloud medium according to the at least two point cloud density levels and the at least two point cloud encoded files includes: Obtain a first dependency identifier; the first dependency identifier is used to characterize the point cloud encoding file T i There are point cloud encoding files with data dependencies; adding the first dependency identifier to the file dependency information in the file encapsulation data box; obtaining the encoded file identifier of the associated point cloud encoded file, and adding the encoded file identifier of the associated point cloud encoded file to the file identification information in the file encapsulation data box; the associated point cloud encoded file refers to the point cloud encoded file obtained after performing point cloud encoding on the associated point cloud data.

4. The method according to claim 1, characterized in that, The at least two point cloud density levels are denoted as K i ; i is an integer greater than or equal to 1; The performing point cloud encoding on a target point cloud medium according to at least two point cloud density levels to obtain at least two point cloud encoded files includes: Obtain the point cloud density level K i The corresponding point cloud data S i , and perform point cloud encoding on the point cloud data S i to obtain the point cloud encoding file T corresponding to the point cloud density level K i ; The point cloud data S i refers to the point cloud data in the target point cloud media that matches the point cloud density level K i i .​ 5. The method according to claim 4, characterized in that, The generating a file encapsulation data box of the target point cloud medium according to the at least two point cloud density levels and the at least two point cloud encoded files includes: Obtain a second dependency identifier, where the second dependency identifier is used to characterize the point cloud encoding file T i There is no point cloud encoding file with a data dependency relationship; adding the second dependency identifier to the file dependency information in the file encapsulation data box; Set the encoding file identifier associated with the point cloud encoding file T in the file identifier information in the file encapsulation data box to an invalid value. i ​ 6. The method according to claim 2 or 4, characterized in that, The method further includes: Obtain the point cloud density level K i The corresponding point cloud encoding file T i The encoding file identifier, as the media file resource M i The media file identifier of; The media file resource M i Refers to the point cloud encoding file T i The media file resource obtained after performing file encapsulation processing on Create a media mapping relationship between the target media identifier of the target point cloud media, the media file identifier, and the point cloud density level K i ; Generate a media signaling file according to the media mapping relationship; the media signaling file is used to instruct the user terminal to request the media file resource M included in the target point cloud media based on the point cloud density level K i Request the media file resource M included in the target point cloud media i .

7. The method according to claim 1, characterized in that, The method further includes: receiving a media file resource acquisition request sent by a user terminal; the media file resource acquisition request is a request generated by the user terminal in response to a point cloud density level selection operation for the target point cloud medium; the media file resource acquisition request carries a target point cloud density level, and the target point cloud density level is the point cloud density level selected by the point cloud density level selection operation; acquiring a target media file resource corresponding to the target point cloud density level according to the media file resource acquisition request; returning the target media file resource to the user terminal, so that the user terminal decodes the target media file resource to obtain display data content with the media quality characterized by the target point cloud density level, and outputs the display data content.

8. A data processing method for point cloud media, characterized in that, including: Based on at least two point cloud density levels corresponding to the target point cloud media, obtain a target media file resource associated with the target point cloud density level; The target point cloud density level is used to characterize the media quality of the target point cloud media; the media file resources respectively corresponding to the at least two point cloud density levels include the target media file resource, and each media file resource is obtained by performing file encapsulation processing on the point cloud encoding file corresponding to the point cloud density level to which it belongs; Obtain a file encapsulation data box; the file encapsulation data box includes the at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that the i-th point cloud encoding file of the target point cloud media has a point cloud encoding file with a data dependency relationship, the file identification information indicates the file identification of the point cloud encoding file having the data dependency relationship with the i-th point cloud encoding file, where i is a positive integer and 1 ≤ i ≤ N; N is the number of point cloud encoding files included in the target point cloud media; Decode the target media file resource according to the file encapsulation data box to obtain display data content with the media quality characterized by the target point cloud density level, and output the display data content.

9. The method according to claim 8, characterized in that The obtaining a target media file resource associated with the target point cloud density level based on at least two point cloud density levels corresponding to the target point cloud media includes: Respond to a point cloud density level selection operation for the target point cloud media, and obtain a media signaling file; the media signaling file includes the media identification of the point cloud media, the at least two point cloud density levels corresponding to the media identification, and the media file identification of the media file resources corresponding to the at least two point cloud density levels; Generate a media file resource acquisition request according to the media signaling file; the target point cloud density level is carried in the media file resource acquisition request; the target point cloud density level is the point cloud density level selected from the at least two point cloud density levels by the point cloud density level selection operation; Send the media file resource acquisition request to the server; Receive the target media file resource returned by the server according to the media file resource acquisition request.

10. The method according to claim 8, characterized in that, The decoding the target media file resource according to the file encapsulation data box to obtain display data content with the media quality characterized by the target point cloud density level includes: If the file dependency information in the file encapsulation data box contains a first dependency identifier, determine an associated media file resource according to the associated media file identifier included in the file identification information in the file encapsulation data box; the associated media file identifier is the media file identifier of the associated media file resource having a data dependency relationship with the target media file resource; the associated media file resource is a file obtained by the server after encoding and encapsulating associated point cloud data; the associated point cloud data refers to the point cloud data in the target point cloud media that matches the associated point cloud density level; the media quality characterized by the associated point cloud density level is lower than the media quality characterized by the target point cloud density level; Jointly decode the associated media file resource and the target media file resource to obtain display data content with the media quality characterized by the target point cloud density level.

11. The method according to claim 10, characterized in that, The jointly decoding the associated media file resource and the target media file resource to obtain display data content with the media quality characterized by the target point cloud density level includes: According to the file encapsulation data box, respectively perform de-encapsulation processing on the associated media file resource and the target media file resource to obtain an associated point cloud encoding file corresponding to the associated media file resource and a target point cloud encoding file corresponding to the target media file resource; Respectively decode the associated point cloud encoding file and the target point cloud encoding file to obtain first display data content corresponding to the associated point cloud encoding file and second display data content corresponding to the target point cloud encoding file; Fuse the first display data content and the second display data content to obtain display data content with the media quality characterized by the target point cloud density level.

12. The method according to claim 8, characterized in that, The decoding the target media file resource according to the file encapsulation data box to obtain display data content with the media quality characterized by the target point cloud density level includes: If the file dependency information in the file encapsulation data box includes a second dependency identifier, perform de-encapsulation processing on the target media file resource according to the file encapsulation data box to obtain a target point cloud encoding file corresponding to the target media file resource; Decode the target point cloud encoding file to obtain display data content with the media quality characterized by the target point cloud density level.

13. A data processing device for point cloud media, characterized in that, Includes: A media encoding module, configured to perform point cloud encoding on a target point cloud media according to at least two point cloud density levels to obtain at least two point cloud encoding files; The point cloud density level is used to characterize the media quality of the target point cloud media; A data box generation module, configured to generate a file encapsulation data box of the target point cloud media according to the at least two point cloud density levels and the at least two point cloud encoding files; the file encapsulation data box includes the at least two point cloud density levels, file identification information, and file dependency information. When the file dependency information indicates that the i-th point cloud encoding file of the target point cloud media has a point cloud encoding file with a data dependency relationship, the file identification information includes the file identification of the point cloud encoding file having the data dependency relationship with the i-th point cloud encoding file, where i is a positive integer less than or equal to the number of the at least two point cloud encoding files; A file encapsulation module, configured to perform file encapsulation processing on the at least two point cloud encoding files respectively according to the file encapsulation data box to obtain media file resources corresponding to the at least two point cloud density levels respectively.

14. A computer device, characterized in that, Includes: A processor, a memory, and a network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide network communication functions, the memory is used to store program codes, and the processor is used to call the program codes to execute the method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1-12 is executed.

Images