A method, device, equipment and medium for processing point cloud media data

Abstract

The embodiments of the present application provide a method, device, equipment and computer-readable storage medium for processing point cloud media data. The method includes: obtaining a window data box of the point cloud media, the window data box is used to define a rotation strategy for the three-dimensional space area corresponding to the point cloud media, and performing a rotation process on the three-dimensional space area corresponding to the point cloud media according to the window data box. It can be seen that the content consumption device can rotate the three-dimensional space area corresponding to the point cloud media in different rotation modes (angles) according to the instructions of the rotation strategy in the window data box, and present the result of the rotation process in the user's current window, so that the presentation form of the point cloud media is more diversified, thereby improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a point cloud media data processing method, a point cloud media data processing device, a point cloud media data processing equipment and a computer-readable storage medium. Background Art

[0002] With the continuous development of science and technology, it is now possible to obtain a large amount of high-precision point cloud data at a lower cost in a shorter period of time. With the continuous accumulation of large-scale point cloud data, how to present point cloud data through point cloud media has become a hot issue in point cloud application research.

[0003] At present, in the transmission technology of point cloud media, there is a three-dimensional space region presentation method based on three-dimensional space region structure data. Through the three-dimensional space region structure data, the content consumption device can present the three-dimensional space region corresponding to the point cloud media. In practice, it is found that the above three-dimensional space region structure data can only indicate the size and anchor point information of the three-dimensional space region, resulting in a single presentation form of the three-dimensional space region of the point cloud media, which cannot adapt to the various application forms of the point cloud media in different situations, and the user experience is poor. Summary of the invention

[0004] The embodiments of the present application provide a point cloud media data processing method, apparatus, device, and computer-readable storage medium, which can enrich the presentation form of the three-dimensional space area of ​​the point cloud media.

[0005] On the one hand, an embodiment of the present application provides a method for processing point cloud media data, including:

[0006] Obtaining a viewport data box of the point cloud media, where the viewport data box is used to define a rotation strategy for a three-dimensional space region corresponding to the point cloud media;

[0007] Perform rotation processing on the three-dimensional space area corresponding to the point cloud media according to the window data box.

[0008] In the embodiment of the present application, a window data box of the point cloud media is obtained, and the window data box is used to define the rotation strategy of the three-dimensional space area corresponding to the point cloud media, and the three-dimensional space area corresponding to the point cloud media is rotated according to the window data box. It can be seen that the content consumption device can rotate the three-dimensional space area corresponding to the point cloud media in different rotation modes (angles) according to the instructions of the rotation strategy in the window data box, and present the result of the rotation processing in the user's current window, so that the presentation form of the point cloud media is more diversified, thereby improving the user experience.

[0009] On the one hand, an embodiment of the present application provides a method for processing point cloud media data, including:

[0010] Obtaining window information during point cloud media presentation, the window information including three-dimensional space area and orientation information;

[0011] The window data box of the point cloud media is configured according to the window information when the point cloud media is presented. The window data box is used to define the rotation strategy of the three-dimensional space area corresponding to the point cloud media.

[0012] In the embodiment of the present application, the window information of the point cloud media when it is presented is obtained, and the window information includes the three-dimensional space area and orientation information. The window data box of the point cloud media is configured according to the window information when the point cloud media is presented, and the window data box is used to define the rotation strategy of the three-dimensional space area corresponding to the point cloud media. It can be seen that the content production device configures the window data box of the point cloud media according to the window information when the point cloud media is presented, and can indicate the presentation of the three-dimensional space area corresponding to the point cloud media on the content consumption device end from multiple rotation angles (making the presentation form on the content consumption device end diversified), thereby enriching the application form of the point cloud media and greatly improving the user experience.

[0013] On the one hand, an embodiment of the present application provides a point cloud media data processing device, including:

[0014] An acquisition unit, used to acquire a view window data box of a point cloud medium, where the view window data box is used to define a rotation strategy of a three-dimensional space region corresponding to the point cloud medium;

[0015] The processing unit is used to perform rotation processing on the three-dimensional space area corresponding to the point cloud media according to the window data box.

[0016] In one embodiment, the view window data box includes point cloud rotation structure data; the processing unit is further used to perform rotation processing on the three-dimensional space area corresponding to the point cloud media according to the view window data box, specifically for:

[0017] The three-dimensional space area corresponding to the point cloud media is rotated according to the point cloud rotation structure data.

[0018] In one embodiment, the point cloud rotation structure data includes a rotation type field. When the rotation type field is an invalid value, the point cloud rotation structure data includes first rotation information. The first rotation information includes a rotation yaw angle field, a rotation pitch angle field, and a rotation roll angle field. The values ​​of the rotation yaw angle field, the rotation pitch angle field, and the rotation roll angle field are all in 2. -16 The unit is [-180°*2 16 , 180°*2 16 ), the value range of the rotation pitch angle field is [-90°*2 16 , 90°*2 16 ];

[0019] The processing unit is further used to perform rotation processing on the three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data, specifically for:

[0020] When the rotation type field is an invalid value, a first type of rotation processing is performed on the three-dimensional space area corresponding to the point cloud media according to the first rotation information.

[0021] In one implementation, the point cloud rotation structure data includes a rotation type field, and when the rotation type field is a valid value, the point cloud rotation structure data includes second rotation information, and the second rotation information includes a first rotation element number field, a second rotation element number field, a third rotation element number field, and a fourth rotation element number field;

[0022] The processing unit is further used to perform rotation processing on the three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data, specifically for:

[0023] When the rotation type field is a valid value, a second type of rotation processing is performed on the three-dimensional space area corresponding to the point cloud media according to the second rotation information.

[0024] In one embodiment, the view window data box further includes a point cloud initial view window data box, which is used to define a rotation strategy for an initial three-dimensional space region corresponding to the point cloud media; the point cloud initial view window data box includes a rotation flag field and initial three-dimensional space region structure data corresponding to the point cloud media;

[0025] The processing unit is further used to perform rotation processing on the three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data, specifically for:

[0026] When the rotation flag field in the point cloud initial view window data box is a valid value, the initial three-dimensional space region corresponding to the point cloud media is determined according to the initial three-dimensional space region structure data corresponding to the point cloud media in the point cloud initial view window data box;

[0027] The initial three-dimensional space area corresponding to the point cloud media is rotated according to the point cloud rotation structure data.

[0028] In one embodiment, the browsing mode of the point cloud media includes a first browsing mode,

[0029] The processing unit is also used to:

[0030] In the first browsing mode, an initial three-dimensional space area corresponding to the rotated point cloud media is displayed.

[0031] In one embodiment, the browsing mode of the point cloud media includes a second browsing mode, and the view window data box also includes a point cloud recommended view window data box, which is used to define a rotation strategy of a target three-dimensional space region corresponding to the point cloud media in the second browsing mode; the point cloud recommended view window data box includes a rotation flag field and three-dimensional space region structure data corresponding to the point cloud media in the second browsing mode;

[0032] The processing unit is further used to perform rotation processing on the three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data, specifically for:

[0033] When the rotation flag field in the point cloud recommendation window data box is a valid value, determining the target three-dimensional space region corresponding to the point cloud media in the second browsing mode according to the target three-dimensional space region structure data corresponding to the point cloud media in the point cloud recommendation window data box in the second browsing mode;

[0034] In the second browsing mode, rotation processing is performed on the target three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data.

[0035] In one implementation, the point cloud recommended view data box further includes a recommended view type field. When the recommended view type field is an invalid value, the point cloud recommended view data box is used to define a rotation strategy of the three-dimensional space region corresponding to the point cloud media in the second browsing mode and under the first recommended view type.

[0036] When the recommended view type field is a valid value, the point cloud recommended view data box is used to define a rotation strategy of the three-dimensional space region corresponding to the point cloud media in the second browsing mode and under the second recommended view type;

[0037] The point cloud recommended view data box includes a recommended view description field, which includes description information of the recommended view. The description information is an 8-bit string ending with a null character.

[0038] In one embodiment, the processing unit is further configured to:

[0039] Displaying the initial three-dimensional space area corresponding to the rotated point cloud media in the second browsing mode; and

[0040] Switch to display the target three-dimensional space area corresponding to the point cloud media after the rotation processing in the second browsing mode.

[0041] In one implementation, the processing unit is further configured to obtain the view window data box of the point cloud media through the obtaining unit, specifically to:

[0042] Obtaining a package file of point cloud media from a content production device;

[0043] The encapsulated file of the point cloud media is decapsulated to obtain the window data box of the point cloud media.

[0044] On the one hand, an embodiment of the present application provides a point cloud media data processing device, including:

[0045] An acquisition unit, used to acquire window information during point cloud media presentation, the window information including three-dimensional space area and orientation information;

[0046] The processing unit is used to configure the window data box of the point cloud media according to the window information when the point cloud media is presented, and the window data box is used to define the rotation strategy of the three-dimensional space area corresponding to the point cloud media.

[0047] In one embodiment, the processing unit is further configured to:

[0048] The viewport data box is encapsulated into a package file of the point cloud media, and the package file of the point cloud media is sent to a content consumption device.

[0049] In one aspect, the present application provides a point cloud media data processing device, the device comprising:

[0050] a processor adapted to execute a computer program;

[0051] A computer-readable storage medium having a computer program stored therein, wherein when the computer program is executed by a processor, the above-mentioned point cloud media data processing method is implemented.

[0052] On the one hand, the present application provides a computer-readable storage medium, which stores a computer program, and the computer program is suitable for being loaded by a processor and executing the above-mentioned point cloud media data processing method.

[0053] On the one hand, the present application provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned point cloud media data processing method.

[0054] In an embodiment of the present application, a window data box of the point cloud media is configured according to the window information when the point cloud media is presented, and rotation processing is performed on the three-dimensional space area corresponding to the point cloud media according to the window data box. The three-dimensional space area corresponding to the point cloud media rotated at different angles can be presented in the window, thereby diversifying the presentation form of the content consumption device, thereby enriching the application form of the point cloud media and greatly improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1a A data processing architecture diagram of a point cloud media provided by an exemplary embodiment of the present application is shown;

[0057] Figure 1b A schematic diagram of the structure of a packaging package provided by an exemplary embodiment of the present application is shown;

[0058] Figure 1c A schematic diagram of a multi-track container provided by an exemplary embodiment of the present application is shown;

[0059] Figure 1d A schematic diagram of the structure of another packaging package provided by an exemplary embodiment of the present application is shown;

[0060] Figure 2 A flowchart of a point cloud media data processing method provided by an exemplary embodiment of the present application is shown;

[0061] Figure 3a A top view of a three-dimensional space region corresponding to a point cloud media provided by an exemplary embodiment of the present application before rotation processing is shown;

[0062] Figure 3b A top view of a three-dimensional space region corresponding to a point cloud media provided by an exemplary embodiment of the present application after rotation processing is shown;

[0063] Figure 4 A flowchart showing another method for processing point cloud media data provided by an exemplary embodiment of the present application is shown;

[0064] Figure 5 A flowchart showing another method for processing point cloud media data provided by an exemplary embodiment of the present application is shown;

[0065] Figure 6 A schematic diagram of the structure of a point cloud media data processing device provided by an exemplary embodiment of the present application is shown;

[0066] Figure 7 A schematic diagram showing the structure of another point cloud media data processing device provided by an exemplary embodiment of the present application is shown;

[0067] Figure 8A schematic structural diagram of a content consumption device provided by an exemplary embodiment of the present application is shown;

[0068] Fig. 9 A schematic structural diagram of a content production device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0070] The embodiments of the present application relate to data processing technology for point cloud media. A point cloud refers to a set of irregularly distributed discrete points in space that express the spatial structure and surface properties of a three-dimensional object or scene. Each point in a point cloud has at least three-dimensional position information, and may also have color, material or other information depending on the application scenario. Typically, each point in a point cloud has the same number of additional attributes.

[0071] Point cloud media can flexibly and conveniently express the spatial structure and surface properties of three-dimensional objects or scenes. Therefore, it is widely used in virtual reality (VR) games, computer-aided design (CAD), geographic information system (GIS), autonomous navigation system (ANS), digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive remote presentation, three-dimensional reconstruction of biological tissues and organs, and other projects.

[0072] With the continuous development of science and technology, it is now possible to obtain a large amount of high-precision point cloud data at a relatively low cost in a relatively short period of time. For example, point cloud data can be obtained by collecting visual scenes in the real world through acquisition equipment (a group of cameras or a camera device with multiple lenses and sensors). Through 3D scanning, point clouds of static real-world three-dimensional objects or scenes can be obtained, and millions of point clouds can be obtained per second; through 3D photography, point clouds of dynamic real-world three-dimensional objects or scenes can be obtained, and tens of millions of point clouds can be obtained per second; in addition, in the medical field, point cloud data of biological tissues and organs can be obtained through magnetic resonance imaging (MRI), electronic computer tomography (CT), and electromagnetic positioning information. For another example, point cloud data can also be directly generated by a computer based on virtual three-dimensional objects and scenes. With the continuous accumulation of large-scale point cloud data, presenting point cloud data through point cloud media has become the key to point cloud applications.

[0073] Figure 1a FIG. 1 shows a data processing architecture diagram of a point cloud media provided by an exemplary embodiment of the present application. Figure 1a As shown, the data processing process at the content production device end mainly includes: (1) the process of acquiring the media content of the point cloud data; (2) the process of encoding and encapsulating the point cloud data. The data processing process at the content consumption device end mainly includes: (3) the process of decapsulating and decoding the point cloud data file; (4) the process of rendering the point cloud data. In addition, the transmission process of the point cloud media between the content production device and the content consumption device can be based on various transmission protocols. The transmission protocols here may include but are not limited to: DASH (Dynamic Adaptive Streaming over HTTP, dynamic adaptive streaming media transmission) protocol, HLS (HTTP Live Streaming, dynamic bit rate adaptive transmission) protocol, SMTP (Smart Media Transport Protocol, smart media transmission protocol), TCP (Transmission Control Protocol, transmission control protocol), etc.

[0074] The following is a detailed description of the data processing process of point cloud media:

[0075] (1) Obtain media content of point cloud data.

[0076] From the point of view of the acquisition method of the media content of point cloud data, it can be divided into two methods: acquisition through the capture device to collect the sound-visual scene of the real world and generation through the computer. In one implementation, the capture device may refer to a hardware component provided in the content production device, for example, the capture device refers to the microphone, camera, sensor, etc. of the terminal. In another implementation, the capture device may also be a hardware device connected to the content production device, such as a camera connected to the server; used to provide the content production device with a service for acquiring the media content of point cloud data. The capture device may include but is not limited to: audio equipment, camera equipment and sensor equipment. Among them, the audio equipment may include an audio sensor, a microphone, etc. The camera equipment may include an ordinary camera, a stereo camera, a light field camera, etc. The sensor equipment may include a laser device, a radar device, etc. The number of capture devices may be multiple, and these capture devices are deployed at some specific locations in the real space to simultaneously capture the audio content and video content of different angles in the space, and the captured audio content and video content are synchronized in time and space. Due to different acquisition methods, the compression encoding methods corresponding to the media content of different point cloud data may also be different.

[0077] (2) The process of encoding the media content of point cloud data and encapsulating the files.

[0078] At present, the acquired point cloud data is usually encoded using a geometry-based point cloud compression (GPCC) encoding method or a traditional video coding-based point cloud compression (VPCC) encoding method. Taking GPCC as an example, when the GPCC bitstream is transmitted in a single track, it requires the GPCC bitstream to be declared and represented according to the single track transmission rules. The GPCC bitstream encapsulated in a single track does not require further processing and can be encapsulated using ISOBMFF (ISO Base Media File Format).

[0079] Each encapsulation packet in the track contains one or more GPCC components, that is, each encapsulation packet is composed of one or more type-length-value bytestream format (TLV) encapsulation structures. Figure 1b A schematic diagram of the structure of a packaging package provided by an exemplary embodiment of the present application is shown in FIG. Figure 1b As shown, during single track transmission, the bit stream of the GPCC parameter set, geometric information and attributes are encapsulated in one encapsulation packet.

[0080] When the encoded GPCC geometry information bitstream and the encoded GPCC attribute bitstream are transmitted in different tracks, each encapsulation packet in the track contains at least one TLV encapsulation structure that carries a single GPCC component data without containing both geometry information and attribute data. Figure 1c A schematic diagram of a multi-track container provided by an exemplary embodiment of the present application is shown. Figure 1c As shown, encapsulation package 1 transmitted in track 1 contains the encoded GPCC geometric information bit stream, but does not contain the encoded GPCC attribute bit stream; encapsulation package 2 transmitted in track 2 contains the encoded GPCC attribute bit stream, but does not contain the encoded GPCC geometric information bit stream.

[0081] Since the encoded GPCC geometry information bitstream should be decoded first during decoding, and the decoding of the encoded GPCC attribute bitstream depends on the decoded geometry, different GPCC component bitstreams are stored in separate tracks so that the player carried by the content consumption device can access the track carrying the encoded GPCC geometry bitstream before the encoded GPCC attribute bitstream. Figure 1d FIG. 1 shows a schematic diagram of the structure of another packaging package provided by an exemplary embodiment of the present application. Figure 1d As shown, the GPCC parameter set and geometric information are encapsulated in a package.

[0082] (3) The process of decapsulating and decoding point cloud data files;

[0083] The content consumption device can obtain the media file resources and corresponding media presentation description information of the point cloud data through the content production device. The media file resources and media presentation description information of the point cloud data are transmitted from the content production device to the content consumption device through a transmission mechanism (such as DASH, SMT). The file decapsulation process on the content consumption device side is the opposite of the file encapsulation process on the content production device side. The content consumption device decapsulates the media file resources according to the file format requirements of the point cloud media to obtain the encoded bit stream (GPCC bit stream or VPCC bit stream). The decoding process on the content consumption device side is the opposite of the encoding process on the content production device side. The content consumption device decodes the encoded bit stream to restore the point cloud data.

[0084] (4) Rendering process of point cloud data.

[0085] The content consumption device renders the point cloud data obtained by decoding the GPCC bit stream according to the metadata related to rendering and window in the media presentation description information. Once the rendering is completed, the visual scene corresponding to the point cloud data is presented.

[0086] In one embodiment, at the content production device end: first, a real-world visual scene is sampled through a collection device to obtain point cloud data corresponding to the real-world visual scene; then, the acquired point cloud data is encoded through a geometry-based point cloud compression (GPCC) or a traditional video-based point cloud compression (VPCC) encoding method to obtain a GPCC bit stream (including an encoded geometry bit stream and an attribute bit stream) or a VPCC bit stream; then, the GPCC bit stream or the VPCC bit stream is encapsulated to obtain a media file (i.e., point cloud media) corresponding to the point cloud data. Specifically, the content production device synthesizes one or more encoded bit streams into a media file for file playback, or a sequence of initialization segments and media segments for streaming, according to a specific media container file format; wherein the media container file format refers to the ISO basic media file format specified in the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) 14496-12. In one embodiment, the content production device also encapsulates the metadata into a media file or a sequence of initialization / media segments, and transmits the sequence of initialization / media segments to the content consumption device via a transmission mechanism (such as a dynamic adaptive streaming media transmission interface).

[0087] On the content consumption device side: first, the point cloud media file sent by the content production device is received, including: a media file for file playback, or a sequence of initialization segments and media segments for streaming; then the point cloud media file is decapsulated to obtain an encoded GPCC bitstream or VPCC bitstream and metadata related to the presentation of the point cloud media file; then the encoded GPCC bitstream is parsed (i.e., the encoded GPCC bitstream is decoded to obtain point cloud data); finally, based on the viewing (viewing window) direction of the current user, the decoded point cloud data is rendered and displayed on the screen of the head-mounted display or any other display device carried by the content consumption device. It should be noted that the viewing (viewing window) direction of the current user is determined by head tracking and possibly visual tracking functions. In addition to the point cloud data used by the renderer to render the viewing (viewing window) direction of the current user, the audio decoder can also be used to decode and optimize the audio in the viewing (viewing window) direction of the current user.

[0088] From the data processing process of the above point cloud media, it can be seen that after decoding the GPCC bitstream or VPCC bitstream of the point cloud encoding, the decoded point cloud data needs to be rendered and presented. At present, in the transmission technology of point cloud media, there is a three-dimensional space region presentation method based on three-dimensional space region structure data (3DSpatialRegionStruct). Through this three-dimensional space region structure data, the content consumption device can present the three-dimensional space region corresponding to the point cloud media.

[0089] The syntax of the three-dimensional spatial region structure data (3DSpatialRegionStruct) can be found in Table 1 below:

[0090] Table 1

[0091]

[0092] The semantics of the syntax shown in Table 1 above are as follows: 3d_region_id indicates the identifier of the three-dimensional space region corresponding to the point cloud media; anchor_included indicates whether the three-dimensional space region corresponding to the point cloud media contains anchor point information; if the three-dimensional space region corresponding to the point cloud media contains anchor point information, anchor_x, anchor_y and anchor_z respectively indicate the x, y, and z axis coordinates of the anchor point in the three-dimensional space region corresponding to the point cloud media; if the three-dimensional space region corresponding to the point cloud media does not contain anchor point information, the default anchor point is the origin (0,0,0); dimension_included indicates whether the three-dimensional space region corresponding to the point cloud media contains size information; if the three-dimensional space region corresponding to the point cloud media contains size information, region_dx, region_dy, and region_dz respectively indicate the scales of the three-dimensional space region corresponding to the point cloud media along the x, y, and z axes, that is, the width, height, and depth of the three-dimensional space.

[0093] Combined with the above Table 1, it can be seen that the current three-dimensional spatial region structure data can only indicate the identification, size and anchor point information of the three-dimensional spatial region corresponding to the point cloud media, and does not contain the rotation information when the point cloud is presented, resulting in a single presentation form of the three-dimensional spatial region of the point cloud media (for example, the three-dimensional spatial region corresponding to the point cloud media after rotation processing cannot be presented in the initial window), and cannot adapt to the various application forms of the point cloud media in different situations, resulting in a poor user experience.

[0094] Based on this, the embodiment of the present application expands the track group data box to obtain the viewport data box (PointCloudViewportBox) of the point cloud media, and adds the rotation information of the three-dimensional space area corresponding to the point cloud media during presentation on the basis of the three-dimensional space area structure data of the point cloud media. Among them, the rotation strategy of the three-dimensional space area corresponding to the point cloud media is indicated by the point cloud rotation structure data (PointCloudRotationStruct); whether the three-dimensional space area corresponding to the point cloud media needs to be rotated during the initial presentation is indicated by the point cloud initial viewport data box (PointCloudInitialViewportBox); the recommended browsing mode of the three-dimensional space area corresponding to the point cloud media is indicated by the point cloud recommended viewport data box (PointCloudRecommendedViewportBox). The semantics of the syntax of the viewport data box of the point cloud media can be found in Table 2 below:

[0095] Table 2

[0096]

[0097]

[0098] The semantics of the grammar in Table 2 above are as follows ①-⑥:

[0099] ①3D_rotation_type is the rotation type field, which is used to indicate the representation type of the rotation information of the three-dimensional space area corresponding to the point cloud media. The value of this field is 0 (invalid value) to indicate that the rotation information is defined in the form of Euler angles; the value of this field is 1 (valid value) to indicate that the rotation information is defined in the form of quaternions, and the rest of the values ​​are reserved.

[0100] ② When the rotation information is defined in the form of Euler angles: rotation_yaw is the rotation yaw angle field, which is used to indicate the yaw angle of the three-dimensional space area corresponding to the point cloud media along the X-axis; rotation_pitch is the rotation pitch angle field, which is used to indicate the pitch angle of the three-dimensional space area corresponding to the point cloud media along the Y-axis; rotation_roll is the rotation roll angle field, which is used to indicate the roll angle of the three-dimensional space area corresponding to the point cloud media along the Y-axis. The values ​​of the rotation yaw angle field, the rotation pitch angle field, and the rotation roll angle field are all in 2. -16 The value range of the rotation yaw angle field and the rotation roll angle field is [-180°*2 16 , 180°*2 16 ), the value range of the rotation pitch angle field is [-90°*2 16 , 90°*216 ].

[0101] ③ The rotation information is defined in the form of quaternion: rotation_x is the first rotation element field, which is used to indicate the value of the quaternion x component; rotation_y is the second rotation element field, which is used to indicate the value of the quaternion y component; rotation_z is the third rotation element field, which is used to indicate the value of the quaternion z component; rotation_w is the fourth rotation element field, which is used to indicate the value of the w component. The rotation angle of the three-dimensional space area corresponding to the point cloud media is jointly indicated by the quaternion x, y, z and w components.

[0102] ④rotation_flag is the rotation flag field, which is used to indicate whether the three-dimensional space area corresponding to the current point cloud media contains rotation information. When the value of this field is 0 (invalid value), it means that the three-dimensional space area corresponding to the current point cloud media does not contain rotation information; when the value of this field is 1 (valid value), it means that the three-dimensional space area corresponding to the current point cloud media contains rotation information.

[0103] ⑤viewport_type is the recommended viewport type field, which is used to indicate the type of the recommended viewport. The corresponding relationship between the value of the recommended viewport type field and the recommended viewport type is shown in the following recommended viewport type table:

[0104] Recommended window type table

[0105] Value describe 0 The viewport recommended by the creator 1 Recommended windows based on big data statistics 2…255 reserve

[0106] ⑥viewport_description is the recommended viewport description field, which is used to indicate the description information of the recommended viewport. The description information is an 8-bit (UTF-8) string ending with a null character.

[0107] According to the window data box of the point cloud media shown in Table 2 of the embodiment of the present application, the content production device configures the window data box of the point cloud media according to the window information when the point cloud media is presented, and can indicate the presentation of the three-dimensional space area corresponding to the point cloud media on the content consumption device end from multiple rotation angles (making the presentation form on the content consumption device end diversified), thereby enriching the application form of the point cloud media and greatly improving the user experience.

[0108] Figure 2 A flowchart of a point cloud media data processing method provided by an exemplary embodiment of the present application is shown; the method can be executed by a content consumption device in a point cloud media system, and the method includes the following steps S201 and S202:

[0109] S201. Obtain a view window data box of a point cloud media, where the view window data box is used to define a rotation strategy for a three-dimensional space region corresponding to the point cloud media.

[0110] S202, performing rotation processing on the three-dimensional space area corresponding to the point cloud media according to the window data box.

[0111] The syntax of the three-dimensional space region structure data corresponding to the point cloud media in step S201 and step S202 can be found in the above Table 1, and the syntax of the window data box of the point cloud media can be found in the above Table 2. The rotation strategy is used to indicate the rotation method of the three-dimensional space region corresponding to the point cloud media. For example, rotation strategy 1 is used to indicate that the three-dimensional space region corresponding to the point cloud media is rotated 30° clockwise along the Z axis.

[0112] In one implementation, the viewport data box of the point cloud media may refer to an ISOBMFF (ISO Base Media File Format) data box. After obtaining the viewport data box of the point cloud media, the content consumption device performs a rotation process on the three-dimensional space region corresponding to the point cloud media according to the viewport data box, and displays the three-dimensional space region corresponding to the point cloud media after the rotation process in the current viewport.

[0113] Figure 3a FIG. 1 shows a top view of a three-dimensional space region corresponding to a point cloud media provided by an exemplary embodiment of the present application before rotation processing. Figure 3a As shown, point A indicates the user's location, and area S ACD is the user's current window area, and point B is the reference point. Figure 3b FIG. 1 shows a top view of a three-dimensional space region corresponding to a point cloud media provided by an exemplary embodiment of the present application after rotation processing. Figure 3b As shown, the user's current window area S ACD What is presented is that the content consumption device will Figure 3a The three-dimensional space region corresponding to the point cloud media shown is the three-dimensional space region after being rotated 30° clockwise along the Z axis.

[0114] In the embodiment of the present application, the content consumption device obtains the window data box of the point cloud media, and the window data box is used to define the rotation strategy of the three-dimensional space area corresponding to the point cloud media, and performs rotation processing on the three-dimensional space area corresponding to the point cloud media according to the window data box. It can be seen that the content consumption device can rotate the three-dimensional space area corresponding to the point cloud media in different rotation modes (angles) according to the instructions of the rotation strategy in the window data box, and present the result of the rotation processing in the user's current window, so that the presentation form of the point cloud media is more diversified, thereby improving the user experience.

[0115] Figure 4A flowchart of another point cloud media data processing method provided by an exemplary embodiment of the present application is shown; the method can be executed by a content consumption device in a point cloud media system, and the method includes the following steps S401-S403:

[0116] S401: Obtain a package file of point cloud media from a content production device.

[0117] The production and packaging process of point cloud media can be referred to Figure 1a The data processing processes (1) and (2) of the point cloud media in will not be repeated here.

[0118] S402, decapsulating the encapsulated file of the point cloud media to obtain a view window data box of the point cloud media, wherein the view window data box includes point cloud rotation structure data.

[0119] The decapsulation process of the point cloud media package file can be referred to Figure 1a The data processing process of the point cloud media in (3) will not be repeated here.

[0120] S403, performing rotation processing on the three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data.

[0121] In one embodiment, the point cloud rotation structure data includes a rotation type field, which is used to indicate the representation type of the rotation information of the three-dimensional space region corresponding to the point cloud media. The value of this field is 0 (invalid value) to indicate that the rotation information is defined in the form of Euler angles; the value of this field is 1 (valid value) to indicate that the rotation information is defined in the form of quaternions, and the other values ​​are reserved.

[0122] Furthermore, when the rotation type field is an invalid value, the point cloud rotation structure data includes first rotation information, and the content consumption device performs a first type of rotation processing on the three-dimensional space area corresponding to the point cloud media according to the first rotation information. The first rotation information includes a rotation yaw angle field, a rotation pitch angle field, and a rotation roll angle field, and the values ​​of the rotation yaw angle field, the rotation pitch angle field, and the rotation roll angle field are all in 2. -16 The unit is [-180°*2 16 , 180°*2 16 ), the value range of the rotation pitch angle field is [-90°*2 16 , 90°*2 16 ].

[0123] When the rotation type field is a valid value, the point cloud rotation structure data includes the second rotation information, and the content consumption device performs the second type of rotation processing on the three-dimensional space area corresponding to the point cloud media according to the second rotation information. The second rotation information includes the first rotation element field, the second rotation element field, the third rotation element field and the fourth rotation element field, and these four rotation element fields respectively indicate the values ​​of the quaternion x, y, z and w components, and the rotation angle of the three-dimensional space area corresponding to the point cloud media is jointly indicated by the quaternion x, y, z and w components.

[0124] In another embodiment, the view window data box of the point cloud media also includes a point cloud initial view window data box, which is used to define the rotation strategy of the initial three-dimensional space region corresponding to the point cloud media; the point cloud initial view window data box includes a rotation flag field and initial three-dimensional space region structure data; wherein, the syntax of the initial three-dimensional space region structure data can be found in the above Table 1, the rotation flag field is used to indicate whether the initial three-dimensional space region corresponding to the point cloud media contains rotation information, when the value of the field is 0 (invalid value), it indicates that the initial three-dimensional space region corresponding to the point cloud media does not contain rotation information, and when the value of the field is 1 (valid value), it indicates that the initial three-dimensional space region corresponding to the point cloud media contains rotation information. When the rotation flag field in the point cloud initial view window data box is a valid value, the content consumption device determines the initial three-dimensional space region corresponding to the point cloud media according to the initial three-dimensional space region structure data corresponding to the point cloud media in the point cloud initial view window data box, and performs rotation processing on the initial three-dimensional space region corresponding to the point cloud media according to the point cloud rotation structure data.

[0125] For example, suppose the relevant fields of the three-dimensional spatial region structure data 3DspatialRegionStruct, the point cloud rotation structure data PointCloudRotationStruct and the point cloud initial viewport data box PointCloudInitialViewportBox in the viewport data box of the point cloud media sent by the content production device are:

[0126] 3d_region_id=1;

[0127] unsingedint(16)anchor_x=10;

[0128] unsingedint(16)anchor_y=0;

[0129] unsingedint(16)anchor_z=0;

[0130] unsingedint(16)region_dx=100;

[0131] unsingedint(16)region_dy=100;

[0132] unsingedint(16)region_dz=100;

[0133] 3D_rotation_type=0;

[0134] signedint(32)rotation_yaw=0;

[0135] signedint(32)rotation_pitch=0;

[0136] signed int(32)rotation_roll=45*2 16 ;

[0137] rotation_flag=1;

[0138] The content consumption device then determines the initial three-dimensional space area corresponding to the point cloud media that needs to be presented based on the above information in the window data box of the point cloud media. The initial three-dimensional space area is centered at (10,0,0) and has dimensions (length, width, and height) of 100,100,100. When presented, the initial three-dimensional space area needs to be rotated according to the point cloud rotation structure data corresponding to the point cloud initial window data box. The rotation method is to rotate the initial three-dimensional space area corresponding to the point cloud media 45° clockwise along the Z axis. That is, the point cloud media presented by the content consumption device is the three-dimensional space area after the initial three-dimensional space area corresponding to the point cloud media is rotated 45° clockwise along the Z axis.

[0139] In an embodiment of the present application, the browsing mode of the point cloud media may include a first browsing mode or a second browsing mode. The first browsing mode may refer to a free browsing mode. The so-called free browsing mode means that the user can freely browse the three-dimensional space area corresponding to the point cloud media by rotating the head / body. The second browsing mode may refer to a recommended browsing mode. The recommended browsing mode refers to browsing the three-dimensional space area corresponding to the point cloud media in a recommended window (such as a window recommended by the director / content producer). Regardless of the browsing mode used by the content consumption device to browse the point cloud media, the first thing presented is the initial three-dimensional space area of ​​the point cloud media after the point cloud rotation structure data corresponding to the point cloud initial window data box is rotated.

[0140] Furthermore, if the content consumption device chooses to browse the point cloud media in the first browsing mode (i.e., free browsing mode), the content consumption device first displays the initial three-dimensional space area of ​​the point cloud media after the point cloud rotation structure data corresponding to the point cloud initial window data box is rotated in this first browsing mode. On this basis, the user can freely browse the three-dimensional space area of ​​the point cloud media by rotating the head / body. If the content message device chooses to browse the point cloud media in the second browsing mode (i.e., recommended browsing mode), the window data box of the point cloud media includes not only the point cloud initial window data box but also the point cloud recommended window data box. The point cloud recommended window data box is used to define the rotation strategy of the target three-dimensional space area corresponding to the point cloud media in the second browsing mode. The point cloud recommended window data box includes a rotation flag field and the target three-dimensional space area structure data corresponding to the point cloud media in the second browsing mode. If the content consumption device chooses to browse the point cloud media in the second browsing mode (i.e., the recommended browsing mode), the content consumption device first displays the initial three-dimensional space area of ​​the point cloud media after rotation processing according to the point cloud rotation structure data corresponding to the point cloud initial window data box in the second browsing mode, and then switches to display the target three-dimensional space area corresponding to the point cloud media after rotation processing according to the point cloud rotation structure data corresponding to the point cloud recommended window data box in the second browsing mode.

[0141] Furthermore, the point cloud recommended window data box also includes a recommended window type field. When the value of this field is 0 (invalid value), it indicates that the currently recommended window is a window recommended by the producer (i.e., the first recommended window type). When the value of this field is 1 (valid value), it indicates that the currently recommended window is a recommended window obtained based on big data statistics (i.e., the second recommended window type). The remaining values ​​2-255 are reserved and can be configured according to actual conditions. For example, in one embodiment, when the value of this field is 2, it indicates that the currently recommended window is a window recommended by a certain director. In addition, the point cloud recommended window data box also includes a recommended window description field, which is used to indicate the description information of the recommended window (such as the first recommended window type is recommended by XX author). The description information is an 8-bit (UTF-8) string ending with a null character.

[0142] For example, suppose the relevant fields of the three-dimensional spatial region structure data 3DspatialRegionStruct, the point cloud rotation structure data PointCloudRotationStruct, the point cloud initial viewport data box PointCloudInitialViewportBox and the recommended viewport data box PointCloudRecommendedViewportBox in the viewport data box of the point cloud media sent by the content production device are:

[0143] 3d_region_id=1;

[0144] unsingedint(16)anchor_x=10;

[0145] unsingedint(16)anchor_y=0;

[0146] unsingedint(16)anchor_z=0;

[0147] unsingedint(16)region_dx=100;

[0148] unsingedint(16)region_dy=100;

[0149] unsingedint(16)region_dz=100;

[0150] 3D_rotation_type=0;

[0151] signedint(32)rotation_yaw=0;

[0152] signedint(32)rotation_pitch=0;

[0153] signed int(32)rotation_roll=45*2 16 ;

[0154] rotation_flag=1;

[0155] 3d_region_id=2;

[0156] unsingedint(16)anchor_x=10;

[0157] unsingedint(16)anchor_y=10;

[0158] unsingedint(16)anchor_z=0;

[0159] unsingedint(16)region_dx=50;

[0160] unsingedint(16)region_dy=50;

[0161] unsingedint(16)region_dz=50;

[0162] rotation_flag=0;

[0163] viewport_type=0;

[0164] Then the content consumption device first displays the initial three-dimensional space area corresponding to the point cloud media after the point cloud rotation structure data corresponding to the point cloud initial view data box is rotated in the second browsing mode. For example, in the above example, the initial three-dimensional space area is centered at (10,0,0) and has a size (length, width and height) of 100,100,100. The initial three-dimensional space area needs to be rotated when presented. The rotation method is to rotate the initial three-dimensional space area corresponding to the point cloud media 45° clockwise along the Z axis. That is, the content consumption device first presents the initial three-dimensional space area corresponding to the point cloud media after the initial three-dimensional space area is rotated 45° clockwise along the Z axis. Then, according to the above information in the view data box of the point cloud media, the content consumption device presents the target three-dimensional space area corresponding to the point cloud media in the second browsing mode to the user. The target three-dimensional space area is centered at (10,10,0) and has a size (length, width and height) of 50,50,50. In this example, since rotaon_flag=0, the target three-dimensional space area does not need to be rotated when presented and can be presented directly.

[0165] In the embodiment of the present application, the content consumption device obtains the encapsulation file of the point cloud media from the content production device, decapsulates the encapsulation file of the point cloud media, obtains the window data box of the point cloud media, the window data box includes point cloud rotation structure data, and performs rotation processing on the three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data. It can be seen that the content consumption device can rotate the three-dimensional space area corresponding to the point cloud media in different rotation modes (angles) according to the instructions of the rotation strategy in the window data box, and present the result of the rotation processing in the user's current window, so that the presentation form of the point cloud media is more diversified, thereby improving the user experience. In addition, when consuming the window data box of the point cloud media, the content consumption device can present the effect specified by the producer of the point cloud media recommendation mode according to the rotation strategy corresponding to different recommended window types, so that the user can obtain the best viewing experience.

[0166] Figure 5 A flowchart of another point cloud media data processing method provided by an exemplary embodiment of the present application is shown; the method can be executed by a content production device in a point cloud media system, and the method includes the following steps S501 and S502:

[0167] S501. Obtain window information during point cloud media presentation, where the window information includes three-dimensional space area and orientation information.

[0168] S502: configuring a window data box of the point cloud media according to the window information when the point cloud media is presented, where the window data box is used to define a rotation strategy for a three-dimensional space region corresponding to the point cloud media.

[0169] The syntax of the three-dimensional space region structure data corresponding to the point cloud media in step S501 and step S502 can be found in the above Table 1, and the syntax of the viewport data box of the point cloud media can be found in the above Table 2. After executing the above steps S501 and S502, the content production device encapsulates the viewport data box into the encapsulation file of the point cloud media, and sends the encapsulation file of the point cloud media to the content consumption device.

[0170] The scheme of the embodiment of the present application is described in detail below through a complete example: the content production device configures the window data box of the point cloud media according to the three-dimensional space area and orientation information of the collected point cloud data file A when it is presented. The window data box includes the three-dimensional space area structure data 3DSpatialRegionStruct, the point cloud rotation structure data PointCloudRotationStruct, the initial window data box PointCloudInitialViewportBox and the recommended window data box PointCloudRecommendedViewportBox. Assume that the identifier of the three-dimensional space area corresponding to the point cloud media is 1; the anchor point coordinates are (10,0,0); the dimensions (length, width and height) of the three-dimensional space area are 100,100,100. Then the content production device configures the relevant fields of the three-dimensional space area structure data 3DSpatialRegionStruct as follows:

[0171] 3d_region_id=1;

[0172] unsingedint(16)anchor_x=10;

[0173] unsingedint(16)anchor_y=0;

[0174] unsingedint(16)anchor_z=0;

[0175] unsingedint(16)region_dx=100;

[0176] unsingedint(16)region_dy=100;

[0177] unsingedint(16)region_dz=100;

[0178] Similarly, assuming that the three-dimensional space area corresponding to the point cloud media contains rotation information, and the rotation information is defined in the form of Euler angles, the three-dimensional space area when the point cloud media is presented is the three-dimensional space area corresponding to the point cloud media rotated 45° clockwise along the Z axis, then the content production device configures the relevant fields of the point cloud rotation structure data PointCloudRotationStruct as follows:

[0179] 3D_rotation_type=0;

[0180] signedint(32)rotation_yaw=0;

[0181] signedint(32)rotation_pitch=0;

[0182] signed int(32)rotation_roll=45*2 16 ;

[0183] And configure the relevant fields of the initial viewport data box PointCloudInitialViewportBox as follows:

[0184] rotation_flag=1;

[0185] Assume that the identifier of the three-dimensional space area corresponding to the recommended cloud media is 2, and the anchor point coordinates are (10, 10, 0); the size (length, width and height) of the three-dimensional space area is 50, 50, 50; the three-dimensional space area when the point cloud media is presented is the three-dimensional space area corresponding to the point cloud media (i.e., no rotation is required); the recommended viewport is the viewport recommended by the producer and does not include description information (i.e., viewport_description is empty), then the content production device configures the relevant fields of the recommended viewport data box PointCloudRecommendedViewportBox and the corresponding three-dimensional space region structure data 3DSpatialRegionStruct as follows:

[0186] 3d_region_id=2;

[0187] unsingedint(16)anchor_x=10;

[0188] unsingedint(16)anchor_y=10;

[0189] unsingedint(16)anchor_z=0;

[0190] unsingedint(16)region_dx=50;

[0191] unsingedint(16)region_dy=50;

[0192] unsingedint(16)region_dz=50;

[0193] rotation_flag=0;

[0194] viewport_type=0;

[0195] After the configuration is completed, the content production device encapsulates the window data box into the encapsulation file of the point cloud media, and sends the encapsulation file of the point cloud media to the content consumption device.

[0196] In the embodiment of the present application, the content production device obtains the window information when the point cloud media is presented, and the window information includes the three-dimensional space area and orientation information. The window data box of the point cloud media is configured according to the window information when the point cloud media is presented. The window data box is used to define the rotation strategy of the three-dimensional space area corresponding to the point cloud media. The window data box is encapsulated into the encapsulation file of the point cloud media, and the encapsulation file of the point cloud media is sent to the content consumption device. It can be seen that the content production device configures the window data box of the point cloud media according to the window information when the point cloud media is presented, and can indicate the presentation of the three-dimensional space area corresponding to the point cloud media on the content consumption device side from multiple rotation angles (making the presentation form on the content consumption device side diversified), thereby enriching the application form of the point cloud media and greatly improving the user experience.

[0197] The method of the embodiment of the present application is described in detail above. In order to facilitate better implementation of the above scheme of the embodiment of the present application, the device of the embodiment of the present application is provided below accordingly.

[0198] See also Figure 6 , Figure 6 A schematic diagram of the structure of a point cloud media data processing device provided by an exemplary embodiment of the present application is shown; the point cloud media data processing device may be a computer program (including program code) running in a content consumption device, for example, the point cloud media data processing device may be an application software in the content consumption device. Figure 6 As shown, the data processing device of the point cloud media includes an acquisition unit 601 and a processing unit 602 . Figure 6 The data processing device of the cloud media shown can be used to perform the above Figure 2 or Figure 4 Part or all of the functions of the method embodiment described. The detailed description of each unit is as follows:

[0199] An acquisition unit 601 is used to acquire a viewport data box of a point cloud medium, where the viewport data box is used to define a rotation strategy for a three-dimensional space region corresponding to the point cloud medium;

[0200] The processing unit 602 is used to perform rotation processing on the three-dimensional space area corresponding to the point cloud media according to the window data box.

[0201] In one implementation, the view window data box includes point cloud rotation structure data; the processing unit 602 is further configured to perform rotation processing on the three-dimensional space region corresponding to the point cloud media according to the view window data box, specifically for:

[0202] The three-dimensional space area corresponding to the point cloud media is rotated according to the point cloud rotation structure data.

[0203] In one embodiment, the point cloud rotation structure data includes a rotation type field. When the rotation type field is an invalid value, the point cloud rotation structure data includes first rotation information. The first rotation information includes a rotation yaw angle field, a rotation pitch angle field, and a rotation roll angle field. The values ​​of the rotation yaw angle field, the rotation pitch angle field, and the rotation roll angle field are all in 2. -16 The unit is [-180°*2 16 , 180°*2 16 ), the value range of the rotation pitch angle field is [-90°*2 16 , 90°*2 16 ];

[0204] The processing unit 602 is further configured to perform rotation processing on the three-dimensional space region corresponding to the point cloud media according to the point cloud rotation structure data, specifically, to:

[0205] When the rotation type field is an invalid value, a first type of rotation processing is performed on the three-dimensional space area corresponding to the point cloud media according to the first rotation information.

[0206] In one implementation, the point cloud rotation structure data includes a rotation type field, and when the rotation type field is a valid value, the point cloud rotation structure data includes second rotation information, and the second rotation information includes a first rotation element number field, a second rotation element number field, a third rotation element number field, and a fourth rotation element number field;

[0207] The processing unit 602 is further configured to perform rotation processing on the three-dimensional space region corresponding to the point cloud media according to the point cloud rotation structure data, specifically, to:

[0208] When the rotation type field is a valid value, a second type of rotation processing is performed on the three-dimensional space area corresponding to the point cloud media according to the second rotation information.

[0209] In one embodiment, the view window data box further includes a point cloud initial view window data box, which is used to define a rotation strategy for an initial three-dimensional space region corresponding to the point cloud media; the point cloud initial view window data box includes a rotation flag field and initial three-dimensional space region structure data corresponding to the point cloud media;

[0210] The processing unit 602 is further configured to perform rotation processing on the three-dimensional space region corresponding to the point cloud media according to the point cloud rotation structure data, specifically, to:

[0211] When the rotation flag field in the point cloud initial view window data box is a valid value, the initial three-dimensional space region corresponding to the point cloud media is determined according to the initial three-dimensional space region structure data corresponding to the point cloud media in the point cloud initial view window data box;

[0212] The initial three-dimensional space area corresponding to the point cloud media is rotated according to the point cloud rotation structure data.

[0213] In one embodiment, the browsing mode of the point cloud media includes a first browsing mode,

[0214] The processing unit 602 is further configured to:

[0215] In the first browsing mode, an initial three-dimensional space area corresponding to the rotated point cloud media is displayed.

[0216] In one embodiment, the browsing mode of the point cloud media includes a second browsing mode, and the view window data box also includes a point cloud recommended view window data box, which is used to define a rotation strategy of a target three-dimensional space region corresponding to the point cloud media in the second browsing mode; the point cloud recommended view window data box includes a rotation flag field and three-dimensional space region structure data corresponding to the point cloud media in the second browsing mode;

[0217] The processing unit 602 is further configured to perform rotation processing on the three-dimensional space region corresponding to the point cloud media according to the point cloud rotation structure data, specifically, to:

[0218] When the rotation flag field in the point cloud recommendation window data box is a valid value, determining the target three-dimensional space region corresponding to the point cloud media in the second browsing mode according to the target three-dimensional space region structure data corresponding to the point cloud media in the point cloud recommendation window data box in the second browsing mode;

[0219] In the second browsing mode, rotation processing is performed on the target three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data.

[0220] In one implementation, the point cloud recommended view data box further includes a recommended view type field. When the recommended view type field is an invalid value, the point cloud recommended view data box is used to define a rotation strategy of the three-dimensional space region corresponding to the point cloud media in the second browsing mode and under the first recommended view type.

[0221] When the recommended view type field is a valid value, the point cloud recommended view data box is used to define a rotation strategy of the three-dimensional space region corresponding to the point cloud media in the second browsing mode and under the second recommended view type;

[0222] The point cloud recommended view data box includes a recommended view description field, which includes description information of the recommended view. The description information is an 8-bit string ending with a null character.

[0223] In one implementation, the processing unit 602 is further configured to:

[0224] Displaying the initial three-dimensional space area corresponding to the rotated point cloud media in the second browsing mode; and

[0225] Switch to display the target three-dimensional space area corresponding to the point cloud media after the rotation processing in the second browsing mode.

[0226] In one implementation, the processing unit 602 is further configured to obtain the view window data box of the point cloud media through the obtaining unit 601, specifically:

[0227] Obtaining a package file of point cloud media from a content production device;

[0228] The encapsulated file of the point cloud media is decapsulated to obtain the window data box of the point cloud media.

[0229] According to one embodiment of the present application, Figure 6 The various units in the data processing device of the point cloud media shown can be individually or completely combined into one or several other units to form, or one (some) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the data processing device of the point cloud media may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, it can be executed by running on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM). Figure 2 or Figure 4 A computer program (including program code) for each step involved in the corresponding method shown in FIG. Figure 6 The data processing device of the point cloud media shown in the figure, and the data processing method of the point cloud media to implement the embodiment of the present application. The above computer program can be recorded on, for example, a computer readable recording medium, and loaded into the above computing device through the computer readable recording medium, and run therein.

[0230] Based on the same inventive concept, the principles and beneficial effects of solving the problems provided by the data processing device for point cloud media in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems provided by the data processing method for point cloud media in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0231] See also Figure 7 , Figure 7 A schematic diagram of the structure of another point cloud media data processing device provided by an exemplary embodiment of the present application is shown; the point cloud media data processing device may be a computer program (including program code) running in a content production device, for example, the point cloud media data processing device may be an application software in the content production device. Figure 7 As shown, the data processing device of the point cloud media includes an acquisition unit 701 and a processing unit 702 . Figure 7 The data processing device of the cloud media shown can be used to perform the above Figure 5 Part or all of the functions of the method embodiment described. The detailed description of each unit is as follows:

[0232] An acquisition unit 701 is used to acquire window information during point cloud media presentation, where the window information includes three-dimensional space area and orientation information;

[0233] The processing unit 702 is used to configure a window data box of the point cloud media according to the window information when the point cloud media is presented, and the window data box is used to define a rotation strategy of a three-dimensional space area corresponding to the point cloud media.

[0234] In one implementation, the processing unit 702 is further configured to:

[0235] The viewport data box is encapsulated into a package file of the point cloud media, and the package file of the point cloud media is sent to a content consumption device.

[0236] According to one embodiment of the present application, Figure 7The various units in the data processing device of the point cloud media shown can be individually or completely combined into one or several other units to form, or one (some) of the units can be further divided into multiple functionally smaller units to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In practical applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the data processing device of the point cloud media may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, it can be executed by running on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM). Figure 5 A computer program (including program code) for each step involved in the corresponding method shown in FIG. Figure 7 The data processing device of the point cloud media shown in the figure, and the data processing method of the point cloud media to implement the embodiment of the present application. The above computer program can be recorded on, for example, a computer readable recording medium, and loaded into the above computing device through the computer readable recording medium, and run therein.

[0237] Based on the same inventive concept, the principles and beneficial effects of solving the problems provided by the data processing device for point cloud media in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems provided by the data processing method for point cloud media in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0238] Figure 8 The following is a schematic diagram showing the structure of a content consumption device provided by an exemplary embodiment of the present application; the content consumption device may be a computer device used by a user of a point cloud media, and the computer device may be a terminal (such as a PC, a smart mobile device (such as a smart phone), a VR device (such as a VR helmet, VR glasses, etc.)). Figure 8 As shown, the content consumption device includes a receiver 801, a processor 802, a memory 803, and a display / playing device 804. Among them:

[0239] The receiver 801 is used to implement decoding and transmission interaction with other devices, and is specifically used to implement the transmission of point cloud media between the content production device and the content consumption device. That is, the content consumption device receives the relevant media resources of the point cloud media transmitted by the content production device through the receiver 801.

[0240] The processor 802 (or CPU (Central Processing Unit)) is the processing core of the content production device. The processor 802 is suitable for implementing one or more program instructions, and is specifically suitable for loading and executing one or more program instructions to achieve Figure 2 or Figure 4 The flow of the point cloud media data processing method is shown.

[0241] The memory 803 is a memory device in the content consumption device, which is used to store programs and media resources. It is understandable that the memory 803 here can include both the built-in storage medium in the content consumption device and the extended storage medium supported by the content consumption device. It should be noted that the memory 803 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one memory located away from the aforementioned processor. The memory 803 provides a storage space, which is used to store the operating system of the content consumption device. In addition, the storage space is also used to store a computer program, which includes program instructions, and the program instructions are suitable for being called and executed by the processor to execute the steps of the data processing method of the point cloud media. In addition, the memory 803 can also be used to store the three-dimensional image of the point cloud media formed after processing by the processor, the audio content corresponding to the three-dimensional image, and the information required for rendering the three-dimensional image and audio content.

[0242] The display / playback device 804 is used to output the rendered sound and three-dimensional image.

[0243] See also Figure 8 , the processor 802 may include a parser 821, a decoder 822, a converter 823 and a renderer 824; wherein:

[0244] The parser 821 is used to decapsulate the encapsulated files of the rendering media from the content production device, specifically to decapsulate the media file resources according to the file format requirements of the point cloud media to obtain audio code streams and video code streams; and provide the audio code streams and video code streams to the decoder 822.

[0245] The decoder 822 performs audio decoding on the audio code stream to obtain audio content and provides it to the renderer for audio rendering. In addition, the decoder 822 decodes the video code stream to obtain a 2D image. According to the metadata provided by the media presentation description information, if the metadata indicates that the point cloud media has performed a region encapsulation process, the 2D image refers to a packaged image; if the metadata indicates that the point cloud media has not performed a region encapsulation process, the plane image refers to a projected image.

[0246] The converter 823 is used to convert a 2D image into a 3D image. If the point cloud media has performed a regional encapsulation process, the converter 823 will first perform regional decapsulation on the encapsulated image to obtain a projection image. The projection image is then reconstructed to obtain a 3D image. If the rendering media has not performed a regional encapsulation process, the converter 823 will directly reconstruct the projection image to obtain a 3D image.

[0247] The renderer 824 is used to render the audio content and 3D images of the point cloud media. Specifically, the audio content and 3D images are rendered according to the metadata related to the rendering and the window in the media presentation description information, and the rendering is output to the display / playback device after completion.

[0248] In an exemplary embodiment, the processor 802 (specifically, each device included in the processor) executes by calling one or more instructions in the memory 803. Figure 2 or Figure 4 Specifically, the memory 803 stores one or more first instructions, which are suitable for being loaded by the processor 802 and executing the following steps:

[0249] Acquire a view window data box of the point cloud media through the receiver 801, where the view window data box is used to define a rotation strategy of a three-dimensional space region corresponding to the point cloud media;

[0250] Perform rotation processing on the three-dimensional space area corresponding to the point cloud media according to the window data box.

[0251] As an optional implementation, the specific implementation of the processor 802 performing the rotation processing on the three-dimensional space area corresponding to the point cloud media according to the window data box is as follows:

[0252] The three-dimensional space area corresponding to the point cloud media is rotated according to the point cloud rotation structure data.

[0253] As an optional implementation, the point cloud rotation structure data includes a rotation type field. When the rotation type field is an invalid value, the point cloud rotation structure data includes first rotation information. The first rotation information includes a rotation yaw angle field, a rotation pitch angle field, and a rotation roll angle field. The values ​​of the rotation yaw angle field, the rotation pitch angle field, and the rotation roll angle field are all in 2. -16 The unit is [-180°*2 16 , 180°*2 16 ), the value range of the rotation pitch angle field is [-90°*2 16 , 90°*2 16 ];

[0254] The specific implementation method of the processor 802 performing rotation processing on the three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data is as follows:

[0255] When the rotation type field is an invalid value, a first type of rotation processing is performed on the three-dimensional space area corresponding to the point cloud media according to the first rotation information.

[0256] As an optional implementation manner, the point cloud rotation structure data includes a rotation type field. When the rotation type field is a valid value, the point cloud rotation structure data includes second rotation information. The second rotation information includes a first rotation element number field, a second rotation element number field, a third rotation element number field, and a fourth rotation element number field.

[0257] The specific implementation method of the processor 802 performing rotation processing on the three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data is as follows:

[0258] When the rotation type field is a valid value, a second type of rotation processing is performed on the three-dimensional space area corresponding to the point cloud media according to the second rotation information.

[0259] As an optional implementation, the view window data box further includes a point cloud initial view window data box, which is used to define a rotation strategy of an initial three-dimensional space region corresponding to the point cloud media; the point cloud initial view window data box includes a rotation flag field and initial three-dimensional space region structure data corresponding to the point cloud media;

[0260] The specific implementation method of the processor 802 performing rotation processing on the three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data is as follows:

[0261] When the rotation flag field in the point cloud initial view window data box is a valid value, the initial three-dimensional space region corresponding to the point cloud media is determined according to the initial three-dimensional space region structure data corresponding to the point cloud media in the point cloud initial view window data box;

[0262] The initial three-dimensional space area corresponding to the point cloud media is rotated according to the point cloud rotation structure data.

[0263] As an optional implementation, the browsing mode of the point cloud media includes a first browsing mode;

[0264] The processor 802 further performs the following operations by running the executable program code in the memory 803:

[0265] In the first browsing mode, an initial three-dimensional space area corresponding to the rotated point cloud media is displayed.

[0266] As an optional implementation, the browsing mode of the point cloud media includes a second browsing mode, and the view window data box also includes a point cloud recommended view window data box, which is used to define a rotation strategy of a target three-dimensional space region corresponding to the point cloud media in the second browsing mode; the point cloud recommended view window data box includes a rotation flag field and three-dimensional space region structure data corresponding to the point cloud media in the second browsing mode;

[0267] The specific implementation method of the processor 802 performing rotation processing on the three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data is as follows:

[0268] When the rotation flag field in the point cloud recommendation window data box is a valid value, determining the target three-dimensional space region corresponding to the point cloud media in the second browsing mode according to the target three-dimensional space region structure data corresponding to the point cloud media in the point cloud recommendation window data box in the second browsing mode;

[0269] In the second browsing mode, rotation processing is performed on the target three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data.

[0270] As an optional implementation, the point cloud recommended view data box further includes a recommended view type field. When the recommended view type field is an invalid value, the point cloud recommended view data box is used to define a rotation strategy of the three-dimensional space area corresponding to the point cloud media in the second browsing mode and under the first recommended view type.

[0271] When the recommended view type field is a valid value, the point cloud recommended view data box is used to define a rotation strategy of the three-dimensional space region corresponding to the point cloud media in the second browsing mode and under the second recommended view type;

[0272] The point cloud recommended view data box includes a recommended view description field, which includes description information of the recommended view. The description information is an 8-bit string ending with a null character.

[0273] As an optional implementation, the processor 802 further performs the following operations by running the executable program code in the memory 803:

[0274] Displaying the initial three-dimensional space area corresponding to the rotated point cloud media in the second browsing mode; and

[0275] Switch to display the target three-dimensional space area corresponding to the point cloud media after the rotation processing in the second browsing mode.

[0276] As an optional implementation, the specific implementation of the processor 802 obtaining the window data box of the point cloud media through the receiver 801 is:

[0277] Obtaining a package file of the point cloud media from the content production device via the receiver 801;

[0278] The encapsulated file of the point cloud media is decapsulated to obtain the window data box of the point cloud media.

[0279] Based on the same inventive concept, the principles and beneficial effects of solving the problems of the point cloud media processing device provided in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems of the point cloud media processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0280] Fig. 9 The schematic diagram of the structure of a content production device provided by an exemplary embodiment of the present application is shown; the content production device may refer to a computer device used by a provider of point cloud media, and the computer device may be a terminal (such as a PC, a smart mobile device (such as a smart phone), etc.) or a server. Fig. 9 As shown, the content production device includes a capture device 901, a processor 902, a memory 903 and a transmitter 904. Among them:

[0281] The capture device 901 is used to collect the sound-visual scene of the real world to obtain the original data of the point cloud media (including audio content and video content synchronized in time and space). The capture device 901 may include but is not limited to: audio equipment, camera equipment and sensor equipment. Among them, the audio equipment may include an audio sensor, a microphone, etc. The camera equipment may include an ordinary camera, a stereo camera, a light field camera, etc. The sensor equipment may include a laser device, a radar device, etc.

[0282] The processor 902 (or CPU (Central Processing Unit)) is the processing core of the content production device. The processor 902 is suitable for implementing one or more program instructions, and is specifically suitable for loading and executing one or more program instructions to achieve Figure 5 The flow of the point cloud media data processing method is shown.

[0283] The memory 903 is a memory device in the content production device, which is used to store programs and media resources. It is understandable that the memory 903 here can include both the built-in storage medium in the content production device and the extended storage medium supported by the content production device. It should be noted that the memory can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one memory located away from the aforementioned processor. The memory provides a storage space, which is used to store the operating system of the content production device. In addition, the storage space is also used to store a computer program, which includes program instructions, and the program instructions are suitable for being called and executed by the processor to execute the various steps of the data processing method of the point cloud media. In addition, the memory 903 can also be used to store point cloud media files formed after processing by the processor, and the point cloud media files include media file resources and media presentation description information.

[0284] The transmitter 904 is used to realize the transmission interaction between the content production device and other devices, and is specifically used to realize the transmission of point cloud media between the content production device and the content consumption device. That is, the content production device transmits the relevant media resources of the point cloud media to the content consumption device through the transmitter 904.

[0285] See also Fig. 9 , the processor 902 may include a converter 921, an encoder 922 and a packager 923; wherein:

[0286] The converter 921 is used to perform a series of conversion processes on the captured video content, so that the video content becomes content suitable for executing point cloud media video encoding. The conversion process may include: splicing and projection, and optionally, the conversion process also includes area packaging. The converter 921 can convert the captured 3D video content into a 2D image and provide it to the encoder for video encoding.

[0287] The encoder 922 is used to perform audio encoding on the captured audio content to form an audio code stream of the point cloud media, and is also used to perform video encoding on the 2D image converted by the converter 921 to obtain a video code stream.

[0288] The encapsulator 923 is used to encapsulate the audio code stream and the video code stream in a file container according to the file format of the point cloud media (such as ISOBMFF) to form a media file resource of the point cloud media. The media file resource can be a media file or a media segment to form a media file of the point cloud media; and according to the file format requirements of the point cloud media, the media presentation description information is used to record the metadata of the media file resource of the point cloud media. The encapsulated file of the point cloud media processed by the encapsulator will be stored in the storage and provided to the content consumption device as needed for the presentation of the point cloud media.

[0289] In an exemplary embodiment, the processor 902 (specifically, each device included in the processor) executes by calling one or more instructions in the memory 903. Figure 5 Specifically, the memory 903 stores one or more first instructions, which are suitable for being loaded by the processor 902 and executing the following steps:

[0290] Acquiring, by means of a capture device 901, viewport information during point cloud media presentation, the viewport information including three-dimensional space area and orientation information;

[0291] The window data box of the point cloud media is configured according to the window information when the point cloud media is presented. The window data box is used to define the rotation strategy of the three-dimensional space area corresponding to the point cloud media.

[0292] As an optional implementation, the processor 902 further performs the following operations by running the executable program code in the memory 903:

[0293] The viewport data box is encapsulated into a package file of the point cloud media, and the package file of the point cloud media is sent to the content consumption device via the transmitter 904 .

[0294] Based on the same inventive concept, the principles and beneficial effects of solving the problems of the point cloud media processing device provided in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems of the point cloud media processing method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0295] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded by a processor and executing a processing method for an application program of the above method embodiment.

[0296] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method for processing the above-mentioned application program.

[0297] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0298] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0299] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0300] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the readable storage medium can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0301] What is disclosed above is only a preferred embodiment of the present application, and it certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiment and equivalent changes made according to the claims of the present application are still within the scope covered by the application.

Claims

1. A method for processing point cloud media data, characterized in that: The method comprises: Obtaining a viewport data box of a point cloud media, wherein the viewport data box is used to define a rotation strategy of a three-dimensional space region corresponding to the point cloud media; Performing rotation processing on the three-dimensional space area corresponding to the point cloud media according to the window data box; Rendering point cloud data corresponding to the point cloud media based on the viewing direction for display on a display device; Wherein, the view data box includes point cloud rotation structure data, and the point cloud rotation structure data includes a rotation type field, and the rotation type field is used to indicate the representation type of the rotation information of the three-dimensional space area corresponding to the point cloud media; the rotation type field has an invalid value, indicating that the rotation information is defined in the form of Euler angles; the rotation type field has a valid value, indicating that the rotation information is defined in the form of quaternions; The performing rotation processing on the three-dimensional space area corresponding to the point cloud media according to the window data box includes: When the rotation type field is an invalid value, performing a first type of rotation processing on the three-dimensional space area corresponding to the point cloud media according to first rotation information, wherein the first rotation information includes: a rotation yaw angle field, a rotation pitch angle field, and a rotation roll angle field; When the rotation type field is a valid value, a second type of rotation processing is performed on the three-dimensional space area corresponding to the point cloud media according to second rotation information, and the second rotation information includes: the second rotation information includes a first rotation element number field, a second rotation element number field, a third rotation element number field and a fourth rotation element number field.

2. The method according to claim 1, characterized in that The values ​​of the rotation yaw angle field, the rotation pitch angle field and the rotation roll angle field are all in 2. -16 The value range of the rotation yaw angle field and the rotation roll angle field is [-180°*2 16 , 180°*2 16 ), the value range of the rotation pitch angle field is [-90°*2 16 , 90°*2 16 ].

3. The method according to claim 1 or 2, characterized in that The view window data box also includes a point cloud initial view window data box, which is used to define a rotation strategy for an initial three-dimensional space region corresponding to the point cloud media; the point cloud initial view window data box includes a rotation flag field and initial three-dimensional space region structure data corresponding to the point cloud media; The performing rotation processing on the three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data includes: When the rotation flag field in the point cloud initial view window data box is a valid value, determining the initial three-dimensional space region corresponding to the point cloud media according to the initial three-dimensional space region structure data corresponding to the point cloud media in the point cloud initial view window data box; A rotation process is performed on an initial three-dimensional space region corresponding to the point cloud media according to the point cloud rotation structure data.

4. The method according to claim 3, characterized in that The browsing mode of the point cloud media includes a first browsing mode, and the method further includes: An initial three-dimensional space area corresponding to the point cloud media after the rotation processing is displayed in the first browsing mode.

5. The method according to claim 3, characterized in that The browsing mode of the point cloud media includes a second browsing mode, and the window data box further includes a point cloud recommended window data box, and the point cloud recommended window data box is used to define a rotation strategy of a target three-dimensional space area corresponding to the point cloud media in the second browsing mode; The point cloud recommendation window data box includes a rotation flag field and target three-dimensional space region structure data corresponding to the point cloud media in the second browsing mode; The performing of rotation processing on the three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data also includes: When the rotation flag field in the point cloud recommendation window data box is a valid value, determining the target three-dimensional space region corresponding to the point cloud media in the second browsing mode according to the target three-dimensional space region structure data corresponding to the point cloud media in the second browsing mode in the point cloud recommendation window data box; In the second browsing mode, rotation processing is performed on the target three-dimensional space area corresponding to the point cloud media according to the point cloud rotation structure data.

6. The method according to claim 5, characterized in that The point cloud recommended view data box further includes a recommended view type field. When the recommended view type field is an invalid value, the point cloud recommended view data box is used to define a rotation strategy of the three-dimensional space region corresponding to the point cloud media in the second browsing mode and under the first recommended view type; When the recommended view type field is a valid value, the point cloud recommended view data box is used to define a rotation strategy of the three-dimensional space region corresponding to the point cloud media in the second browsing mode and under the second recommended view type; The point cloud recommended view data box includes a recommended view description field, and the recommended view description field includes description information of the recommended view, and the description information is an 8-bit string ending with a null character.

7. The method according to claim 5, characterized in that The method further comprises: Displaying the initial three-dimensional space area corresponding to the point cloud media after the rotation processing in the second browsing mode; and Switching to display the target three-dimensional space area corresponding to the point cloud media after the rotation processing in the second browsing mode.

8. The method according to claim 1, characterized in that The step of obtaining the view window data box of the point cloud media includes: Obtaining a package file of point cloud media from a content production device; The encapsulated file of the point cloud media is decapsulated to obtain a window data box of the point cloud media.

9. A method for processing point cloud media data, characterized in that: The method comprises: Obtaining window information during point cloud media presentation, wherein the window information includes three-dimensional space area and orientation information; Configure a window data box of the point cloud media according to the window information when the point cloud media is presented, wherein the window data box is used to define a rotation strategy of a three-dimensional space region corresponding to the point cloud media; Wherein, the view data box includes point cloud rotation structure data, and the point cloud rotation structure data includes a rotation type field, and the rotation type field is used to indicate the representation type of the rotation information of the three-dimensional space area corresponding to the point cloud media; the rotation type field has an invalid value, indicating that the rotation information is defined in the form of Euler angles; the rotation type field has a valid value, indicating that the rotation information is defined in the form of quaternions; Based on the rotation strategy of the three-dimensional space area corresponding to the point cloud media defined by the window data box, when the rotation type field is an invalid value, the first type of rotation processing will be performed on the three-dimensional space area corresponding to the point cloud media according to the first rotation information, and the first rotation information includes: a rotation yaw angle field, a rotation pitch angle field, and a rotation roll angle field; when the rotation type field is a valid value, the second type of rotation processing will be performed on the three-dimensional space area corresponding to the point cloud media according to the second rotation information, and the second rotation information includes: the second rotation information includes a first rotation element number field, a second rotation element number field, a third rotation element number field, and a fourth rotation element number field.

10. The method according to claim 9, characterized in that The method further comprises: The window data box is encapsulated into an encapsulation file of the point cloud media, and the encapsulation file of the point cloud media is sent to a content consumption device.

11. A point cloud media data processing device, characterized in that: include: An acquisition unit, used to acquire a view window data box of a point cloud medium, wherein the view window data box is used to define a rotation strategy of a three-dimensional space region corresponding to the point cloud medium; a processing unit, configured to perform a rotation process on the three-dimensional space region corresponding to the point cloud media according to the viewport data box, and to render the point cloud data corresponding to the point cloud media based on the viewing direction for display on a display device; Wherein, the view data box includes point cloud rotation structure data, and the point cloud rotation structure data includes a rotation type field, and the rotation type field is used to indicate the representation type of the rotation information of the three-dimensional space area corresponding to the point cloud media; the rotation type field has an invalid value, indicating that the rotation information is defined in the form of Euler angles; the rotation type field has a valid value, indicating that the rotation information is defined in the form of quaternions; The processing unit is used to perform a first type of rotation processing on the three-dimensional space area corresponding to the point cloud media according to first rotation information when the rotation type field is an invalid value, and the first rotation information includes: a rotation yaw angle field, a rotation pitch angle field, and a rotation roll angle field; when the rotation type field is a valid value, perform a second type of rotation processing on the three-dimensional space area corresponding to the point cloud media according to second rotation information, and the second rotation information includes: the second rotation information includes a first rotation element number field, a second rotation element number field, a third rotation element number field, and a fourth rotation element number field.

12. A point cloud media data processing device, characterized in that: include: An acquisition unit, used to acquire window information during point cloud media presentation, wherein the window information includes three-dimensional space area and orientation information; A processing unit, configured to configure a window data box of the point cloud media according to the window information when the point cloud media is presented, wherein the window data box is used to define a rotation strategy of a three-dimensional space region corresponding to the point cloud media; Wherein, the view data box includes point cloud rotation structure data, and the point cloud rotation structure data includes a rotation type field, and the rotation type field is used to indicate the representation type of the rotation information of the three-dimensional space area corresponding to the point cloud media; the rotation type field has an invalid value, indicating that the rotation information is defined in the form of Euler angles; the rotation type field has a valid value, indicating that the rotation information is defined in the form of quaternions; Based on the rotation strategy of the three-dimensional space area corresponding to the point cloud media defined by the window data box, when the rotation type field is an invalid value, the first type of rotation processing will be performed on the three-dimensional space area corresponding to the point cloud media according to the first rotation information, and the first rotation information includes: a rotation yaw angle field, a rotation pitch angle field, and a rotation roll angle field; when the rotation type field is a valid value, the second type of rotation processing will be performed on the three-dimensional space area corresponding to the point cloud media according to the second rotation information, and the second rotation information includes: the second rotation information includes a first rotation element number field, a second rotation element number field, a third rotation element number field, and a fourth rotation element number field.

13. A point cloud media data processing device, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by the processor, the data processing method of the point cloud media as described in any one of claims 1 to 8 is implemented, or the data processing method of the point cloud media as described in claim 9 or 10 is implemented.

14. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the data processing method of the point cloud media described in any one of claims 1 to 8 is implemented, or the data processing method of the point cloud media described in claim 9 or 10 is implemented.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the point cloud media data processing method described in any one of claims 1 to 8, or implements the point cloud media data processing method described in claim 9 or 10.

Images