Immersive media data processing method, device, and computer-readable storage medium

Abstract

The present application provides a data processing method, device, and computer-readable storage medium for immersive media files, which can enrich the presentation form of immersive media. The method includes: obtaining a recommended window data box of the immersive media file, where the recommended window data box is used to define the switching information of the viewpoint of the immersive media file and the switching information of the recommended window; determining a target viewpoint and a target recommended window based on the switching information of the current viewpoint and the switching information of the current recommended window; switching to the target viewpoint and the target recommended window to present the immersive media file.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly, to an immersive media data processing method, device, and computer-readable storage medium. Background Art

[0002] In the prior art, the content of immersive media is divided into multiple samples (for example, image frames or video frames), which are encapsulated in track groups according to their relevance. The so-called relevance means that all samples in the track group correspond to the same viewpoint, or samples in a track group correspond to one viewpoint, or samples in a track group correspond to different viewpoints. However, the samples in a track group can only be presented in a certain time sequence. Such relevance limits the packaging flexibility of immersive media to a certain extent, and thus limits the flexibility of the presentation form of immersive media. Therefore, how to enrich the presentation form of immersive media is an issue that needs to be solved urgently. Summary of the Invention

[0003] The present application provides an immersive media data processing method, device, and computer-readable storage medium, which can enrich the presentation form of immersive media.

[0004] In a first aspect, a method for processing immersive media data is provided, comprising: obtaining a recommended viewport data box of an immersive media file, the recommended viewport data box being used to define viewpoint switching information of the immersive media file and recommended viewport switching information;

[0005] Determine the target viewpoint and the target recommended window according to the switching information of the current viewpoint and the switching information of the current recommended window;

[0006] Switch to the target viewpoint and the target recommendation window to present the immersive media file.

[0007] In a second aspect, a method for processing immersive media data is provided, comprising: obtaining viewpoint information and window information when presenting an immersive media file;

[0008] A recommended window data box is configured according to viewpoint information and window information when the immersive media file is presented, wherein the recommended window data box is used to define viewpoint switching information of the immersive media file and switching information of the recommended window.

[0009] In a third aspect, an immersive media data processing device is provided, comprising:

[0010] an acquiring unit, configured to acquire a recommended viewport data box of an immersive media file, wherein the recommended viewport data box is used to define switching information of a viewpoint of the immersive media file and switching information of a recommended viewport;

[0011] a determining unit, configured to determine a target viewpoint and a target recommended window according to the switching information of the current viewpoint and the switching information of the current recommended window;

[0012] A presentation unit is configured to switch to the target viewpoint and the target recommendation window to present the immersive media file.

[0013] In a fourth aspect, an immersive media data processing device is provided, comprising:

[0014] An acquisition unit, configured to acquire viewpoint information and window information when presenting an immersive media file;

[0015] The encapsulation unit is configured to encapsulate a recommended window data box according to viewpoint information and window information when the immersive media file is presented, wherein the recommended window data box is used to define viewpoint switching information of the immersive media file and switching information of the recommended window.

[0016] In a fifth aspect, an immersive media data processing device is provided, comprising a processor and a memory. The memory is configured to store a computer program, and the processor is configured to call and execute the computer program stored in the memory to perform the method of the first aspect or its implementations.

[0017] In a sixth aspect, an immersive media data processing device is provided, comprising a processor and a memory. The memory is configured to store a computer program, and the processor is configured to call and execute the computer program stored in the memory to perform the method of the second aspect or its implementations.

[0018] In a seventh aspect, a computer-readable storage medium is provided for storing a computer program, which enables a computer to execute the method of any one of the first to second aspects or their respective implementations.

[0019] In an eighth aspect, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions enable a computer to execute the method of any one of the first to second aspects or their respective implementations.

[0020] In a ninth aspect, a computer program is provided, which, when executed on a computer, enables the computer to execute the method in any one of the first to second aspects or their respective implementations.

[0021] Through the above technical solution, the media production device can encapsulate the recommended window data box of the immersive media according to the viewpoint information and window information of the immersive media. Furthermore, the media playback device can switch the viewpoint and window according to the switching method of the viewpoint and window defined in the recommended window data box, thereby diversifying the presentation form of the media playback device, thereby enriching the application form of the immersive media and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is an architectural diagram of an immersive media system provided by an exemplary embodiment of the present application.

[0023] Figure 2 This is a flow chart of an immersive media transmission solution provided by an exemplary embodiment of the present application.

[0024] Figure 3 This is a schematic flowchart of an immersive media data processing method provided according to an embodiment of the present application.

[0025] Figure 4 This is a schematic flowchart of another immersive media data processing method provided according to an embodiment of the present application.

[0026] Figure 5 It is a structural diagram of an immersive media data processing device provided by an exemplary embodiment of the present application.

[0027] Figure 6 A schematic structural diagram of another immersive media data processing device provided by an exemplary embodiment of the present application.

[0028] Figure 7 It is a structural diagram of a media production device provided by an exemplary embodiment of the present application.

[0029] Figure 8 It is a structural diagram of a media playback device provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0030] The following will describe the technical solutions in the embodiments of this application in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. With respect to the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0031] The embodiments of the present application relate to data processing technology for immersive media. The so-called immersive media refers to a media file that can provide immersive media content, so that users immersed in the media content can obtain visual, auditory and other sensory experiences in the real world. Specifically, the immersive media can be 3DoF (Degree of Freedom) immersive media, 3DoF+ immersive media or 6DoF immersive media. Immersive media content includes video content represented in three-dimensional (3-Dimension, 3D) space in various forms, such as three-dimensional video content represented in a spherical form. Specifically, immersive media content can be VR (Virtual Reality) video content, panoramic video content, spherical video content or 360-degree video content; therefore, immersive media can also be called VR video, panoramic video, spherical video or 360-degree video. In addition, the immersive media content also includes audio content synchronized with the video content represented in the three-dimensional space.

[0032] Figure 1 FIG. 1 shows an architecture diagram of an immersive media system provided by an exemplary embodiment of the present application; FIG. Figure 1 As shown, the immersive media system includes a media production device and a media playback device (or media consumption device). The media production device may refer to a computer device used by the provider of immersive media (e.g., the media producer of immersive media), and the computer device may be a terminal (e.g., PC (Personal Computer), smart mobile device (e.g., smart phone), etc.) or a server. The media playback device may refer to a computer device used by the user of immersive media (e.g., user), and the computer device may be a terminal (e.g., PC (Personal Computer), smart mobile device (e.g., smart phone), VR device (e.g., VR helmet, VR glasses, etc.)). The data processing process of immersive media includes the data processing process on the media production device side and the data processing process on the media playback device side.

[0033] The data processing process on the media production device side mainly includes: (1) the process of acquiring and producing the media content of the immersive media; (2) the process of encoding and encapsulating the immersive media files. The data processing process on the media playback device side mainly includes: (3) the process of decapsulating and decoding the immersive media files; (4) the process of rendering the immersive media. In addition, the transmission process of the immersive media between the media production device and the media playback device can be based on various transmission protocols. The transmission protocols here may include but are not limited to: DASH (Dynamic Adaptive Streaming over HTTP, dynamic adaptive streaming media transmission) protocol, HLS (HTTP Live Streaming, dynamic bit rate adaptive transmission) protocol, SMTP (Smart Media Transport Protocol, smart media transmission protocol), TCP (Transmission Control Protocol, transmission control protocol), etc.

[0034] Figure 2 FIG1 shows a flow chart of an immersive media transmission solution provided by an exemplary embodiment of the present application. Figure 2 As shown, in order to solve the transmission bandwidth load problem caused by the excessive amount of data in immersive media itself, in the processing of immersive media, it is usually chosen to spatially split the original video into multiple block videos, encode them separately, encapsulate them, and then transmit them to the client for consumption.

[0035] Combine Figure 1 and Figure 2 ,The following sections describe each process involved in the data ,processing of immersive media.

[0036] 1. Data processing on the media production device side:

[0037] (1) Obtain media content of immersive media.

[0038] The acquisition method for immersive media content can be categorized into two approaches: capturing real-world audio and visual scenes through a capture device and generating content through a computer. In one implementation, the capture device can refer to a hardware component within the media production device, such as a microphone, camera, or sensor on a terminal. In another implementation, the capture device can also be a hardware device connected to the media production device, such as a camera connected to a server, providing immersive media content acquisition services to the media production device. The capture device may include, but is not limited to, audio equipment, video equipment, and sensor equipment. Audio equipment may include audio sensors and microphones. Video equipment may include standard cameras, stereo cameras, and light field cameras. Sensor equipment may include laser equipment and radar equipment. There may be multiple capture devices, deployed at specific locations in real space to simultaneously capture audio and video content from different angles within that space, with the captured audio and video content synchronized in both time and space. Due to different acquisition methods, the compression encoding methods corresponding to the media content of different immersive media may also vary.

[0039] (2) The production process of media content for immersive media.

[0040] The captured audio content itself is suitable for audio encoding for immersive media. The captured video content undergoes a series of production processes before it becomes video encoding suitable for immersive media. The production process includes:

[0041] ① Stitching. Since the captured video content is shot by the capture device at different angles, stitching refers to stitching the video content shot at various angles into a complete video that can reflect the 360-degree visual panorama of the real space. In other words, the stitched video is a panoramic video (or spherical video) represented in three-dimensional space.

[0042] ② Projection. Projection refers to the process of mapping a spliced ​​3D video onto a 2D image. The resulting 2D image is called a projected image. Projection methods include, but are not limited to, latitude and longitude projection and regular hexahedron projection.

[0043] (3) The encoding process of media content of immersive media.

[0044] The projected image can be encoded directly or after region encapsulation. Modern mainstream video coding technologies, such as the international video coding standards HEVC (High Efficiency Video Coding), VVC (Versatile Video Coding), and AVS (Audio Video Coding Standard), use a hybrid coding framework to perform a series of operations and processing on the input raw video signal: block partition structure, predictive coding, transform coding and quantization, entropy coding or statistical coding, and loop filtering.

[0045] (4) The packaging process of immersive media.

[0046] The audio code stream and the video code stream are encapsulated in a file container according to the file format of immersive media (such as ISOBMFF (ISO Base Media File Format, ISO base media file format)) to form the media file resource of immersive media. The media file resource can be a media file or a media fragment to form the media file of immersive media; and according to the file format requirements of immersive media, the media presentation description information (Media presentation description, MPD) is used to record the metadata of the media file resource of immersive media. The metadata here is a general term for information related to the presentation of immersive media. The metadata may include description information of the media content, description information of the window, and signaling information related to the presentation of media content, etc. Figure 1 As shown, the media production device will store the media presentation description information and media file resources formed after the data processing process.

[0047] A sample is a unit of media file encapsulation. A media file is composed of many samples. For example, in the case of video media, a sample is typically a video frame.

[0048] 2. Data processing on the media player device:

[0049] (1) The process of decapsulating and decoding immersive media files;

[0050] The media playback device can dynamically and adaptively obtain immersive media media file resources and corresponding media presentation description information from the media production device based on recommendations from the media production device or based on user needs on the media playback device. For example, the media playback device can determine the user's orientation and position based on head / eye / body tracking information, and then dynamically request the corresponding media file resources from the media production device based on the determined orientation and position. The media file resources and media presentation description information are transmitted from the media production device to the media playback device via a transport mechanism (such as DASH and SMT). The file decapsulation process on the media playback device is the inverse of the file encapsulation process on the media production device. The media playback device decapsulates the media file resources according to the immersive media file format requirements to obtain audio and video streams. The decoding process on the media playback device is the inverse of the encoding process on the media production device. The media playback device decodes the audio stream to restore the audio content. In addition, the decoding process of the video stream on the media playback device includes the following: ① Decoding the video stream to obtain a two-dimensional projected image. ② Reconstruct the projected image according to the media presentation description information to convert it into a 3D image. The reconstruction process here refers to the process of re-projecting the two-dimensional projected image into a 3D space.

[0051] (2) Rendering process of immersive media.

[0052] The media player renders the audio content obtained from audio decoding and the 3D image obtained from video decoding based on the rendering, viewpoint, and viewport metadata in the media presentation description. Once rendering is complete, the 3D image is played and output. Viewpoint refers to the immersive media content captured by the capture device during immersive media production, and viewport refers to the portion of the immersive media content viewed by the user during immersive media presentation.

[0053] The immersive media system supports data boxes (Boxes). Data boxes are data blocks or objects that contain metadata. That is, data boxes contain metadata about the corresponding media content. In related technologies, viewport metadata is recorded using the recommended viewport data box (RcvpInfoBox). The syntax of the recommended viewport data box (RcvpInfoBox) for immersive media is shown in Table 1 below:

[0054] Table 1

[0055]

[0056] The semantics of the syntax shown in the above expression 1 are as follows:

[0057] viewport_type: indicates the type of recommended viewport. A value of 0 indicates that the recommended viewport is edited by the media producer; a value of 1 indicates that the recommended viewport is obtained based on big data statistics.

[0058] viewport_description: indicates the description of the recommended viewport, which is an octet (UTF-8) string terminated by a null character.

[0059] viewpoint_idc: When the value of this field is 0, it indicates that all media tracks associated with the current media track belong to the same viewpoint. When the value is 1, it indicates that the ID of the viewpoint associated with the entry of the current recommended track is determined according to rvif_viewpoint_id. When the value is 2, it indicates that some samples in the media track correspond to a specific viewpoint.

[0060] rvif_viewpoint_id: indicates the ID of the viewpoint to which all samples corresponding to the sample entry belong.

[0061] viewpoint_id: Indicates the ID of the viewpoint to which the sample belongs.

[0062] In the embodiments of this application, a track refers to a series of samples with time attributes that are encapsulated in the ISO base media file format (ISOBMFF). For example, a video track is obtained by encapsulating the bitstream generated by each frame of the video encoder according to the ISOBMFF specification. A track can correspond to an entry, or a sample entry.

[0063] As can be seen from the above semantics, in RcvpInfoBox, when viewpoint_idc is 1, each viewpoint corresponds to one recommended window, and different viewpoints have different recommended windows. When viewpoint_idc is 2, a recommended window can switch between multiple viewpoints, meaning that a single recommended window can correspond to multiple viewpoints. In other words, in both scenarios, the viewpoint must switch with the recommended window, failing to provide a flexible user experience for switching viewpoints and windows.

[0064] In view of this, an embodiment of the present application provides an immersive media data processing method, which can encapsulate viewpoint switching information and window switching information in a recommended video data box. In this way, the media playback device can flexibly switch viewpoints and windows according to the viewpoint switching information and window switching information, thereby improving the user experience.

[0065] Figure 3 This is a schematic flow chart of an immersive media data processing method provided according to an exemplary embodiment of the present application. This method can be executed by a media playback device (or media consumption device) in an immersive media system. The media consumption device can be, for example, a server, drone, handheld terminal, or other device with point cloud media encoding and decoding capabilities, which is not limited by this application.

[0066] like Figure 3 As shown, the method may include at least part of the following:

[0067] S301, obtaining a recommended viewport data box of an immersive media file, where the recommended viewport data box is used to define viewpoint switching information of the immersive media file and recommended viewport switching information;

[0068] S302, determining a target viewpoint and a target recommended window based on the switching information of the current viewpoint and the switching information of the current recommended window;

[0069] S303: Switch to the target viewpoint and the target recommendation window to present the immersive media file.

[0070] Optionally, in some embodiments, when the media consumption device chooses to browse immersive media in the recommended browsing mode, the media consumption device switches the viewpoint and the window according to the switching of the viewpoint and the window in the recommended window data box.

[0071] Optionally, the recommended browsing mode refers to browsing various samples of the immersive media in a recommended viewing window (such as a viewing window recommended by a director / content producer).

[0072] Optionally, in some embodiments, the recommended window data box includes:

[0073] The viewpoint switch field (denoted as viewpoint_switch) is used to indicate whether it is necessary to switch to another viewpoint after presenting a specific recommended viewport corresponding to the current viewport, wherein the current viewport corresponds to at least one recommended viewport.

[0074] For example, a value of 1 in the viewpoint switching field indicates that after the specific recommended window corresponding to the current viewpoint is presented, it is necessary to switch to another viewpoint; a value of 0 in the viewpoint switching field indicates that after the specific recommended window corresponding to the current viewpoint is presented, it is not necessary to switch to another viewpoint.

[0075] Optionally, in some embodiments, the recommended window data box includes:

[0076] The target viewpoint field (denoted as des_viewpoint_id) is used to indicate the target viewpoint to switch to after presenting the current recommended viewport;

[0077] The target recommendation viewport field (denoted as des_viewport_id) is used to indicate the target recommendation viewport to be switched to after the current recommendation viewport is presented and switched to the target viewport.

[0078] In some embodiments, when the viewpoint switching field indicates that switching to another viewpoint is required, the target viewpoint and the target recommended view window are determined according to the target viewpoint field and the target recommended view window field.

[0079] In some embodiments, the value of the target viewpoint field may be an identifier of the target viewpoint.

[0080] In some embodiments, the value of the target recommendation window field may be an identifier of a recommended video.

[0081] Optionally, in some embodiments, the recommended viewport data box includes an independent viewport field (independent_viewport) for indicating whether each viewport corresponds to an independent set of recommended viewports.

[0082] As an example, the independent window field takes the first value, such as 1, indicating that each viewpoint corresponds to an independent set of recommended windows. In other words, each set of recommended windows corresponds to an independent viewpoint, that is, the two are one-to-one corresponding.

[0083] The independent viewpoint corresponding to each group of recommended view windows is determined according to the viewpoint identification ID field in the sample entry of each recommended view window.

[0084] As another example, the independent window field takes a second value, such as 0, indicating that each viewpoint corresponds to the same set of recommended windows. In this case, there is only one recommended window entry, and all viewpoints correspond to this recommended window entry.

[0085] In other words, when the value of the Independent Window field is 1, it means that each set of recommended windows (i.e., a recommended window entry) only corresponds to the content of a certain viewpoint. When the value of the Independent Window field is 0, it means that there is only one set of recommended windows, and this set of recommended windows corresponds to the content of all viewpoints.

[0086] In the related art, the viewpoint information in the recommended window data box only includes the tag information of the viewpoint. When a viewpoint includes multiple media contents, it is impossible to achieve diversified presentation of the viewpoint.

[0087] Therefore, in the embodiment of the present application, the package file of the immersive media may further include a viewpoint entity group data box (ViewpointEntityGroupBox) for recording labels of the various entities constituting the viewpoint.

[0088] Optionally, the entities constituting a viewpoint may refer to various media components, such as video, audio, background video, etc., which is not limited in this application.

[0089] Specifically, when switching to a certain viewpoint, the media consumption device can obtain the labels of each entity of the viewpoint and perform rendering based on the entities of the viewpoint, thereby facilitating diversified presentation of viewpoints.

[0090] In one embodiment, the recommended viewport data box and the view entity group data box may be implemented by extending the ISO Base Media File Format (ISOBMFF) data box.

[0091] As an example, the recommended video data box may include the viewpoint information data box (AssoRcViewpointInfoBox) in Table 2 and the recommended viewport sample data box (RecommendedViewportSample) in Table 3, where the viewpoint information data box and the recommended viewport sample data box are respectively used to define the above fields.

[0092] Table 2

[0093]

[0094] Table 3

[0095]

[0096] The meaning of each field in Table 2 and Table 3 is as described above.

[0097] As an example, the viewpoint entity group data box can be defined in the manner shown in Table 4.

[0098] Table 4

[0099]

[0100] The semantics of the syntax shown in Table 4 above are as follows:

[0101] viewpoint_label: indicates the label information of the viewpoint, for example, it can be a string ending with a null character.

[0102] group_id: indicates the group ID of the current entity group.

[0103] num_entities_in_group: Indicates the number of entities in the current entity group.

[0104] entity_id: Indicates the ID of the entity, which corresponds to the track ID or item ID.

[0105] viewpoint_entity_label: indicates the labels of the entities that make up the current viewpoint. For example, it can be a string terminated by a null character.

[0106] It should be understood that Table 2, Table 3 and Figure 4 The encapsulation method in the example is only an example. In other embodiments, the above fields can also be encapsulated in other ways, and this application does not limit this.

[0107] In addition, the recommended window data box also includes a recommended window description field for indicating description information of the recommended window (eg, the first recommended window type is recommended by XX author). The description information is an octet (UTF-8) string ending with a null character.

[0108] Optionally, in some embodiments, the step S301 may specifically include:

[0109] Obtain a package file of the immersive media file from a media production device;

[0110] The encapsulated file of the immersive media file is decapsulated to obtain a recommended window data box of the immersive media file.

[0111] It should be understood that the transmission and decapsulation process of the encapsulated file of the immersive media can refer to Figure 1 and Figure 2 The description of the illustrated embodiment will not be repeated here.

[0112] Therefore, in an embodiment of the present application, a media consumption device obtains an encapsulated file of immersive media from a media production device, decapsulates the encapsulated file of the immersive media, and obtains a recommended window data box of the immersive media. The recommended window data box is used to define switching information of the viewpoint of the immersive media and switching information of the recommended window. Furthermore, the media consumption device can switch viewpoints and windows according to the switching method defined by the recommended window data box, thereby making the presentation form of the immersive media more diversified, thereby improving the user experience.

[0113] Figure 4 This is a schematic flow chart of an immersive media data processing method provided according to an exemplary embodiment of the present application. This method can be executed by a media production device in an immersive media system. The media production device can be, for example, a server, drone, handheld terminal, or other device with point cloud media encoding and decoding capabilities, although this application does not limit this.

[0114] like Figure 4 As shown, the method may include at least part of the following:

[0115] S401, obtaining viewpoint information and window information of an immersive media file;

[0116] S402 : Encapsulating a recommended window data box according to the viewpoint information and the window information of the immersive media file, wherein the recommended window data box is used to define viewpoint switching information of the immersive media file and switching information of the recommended window.

[0117] Optionally, in some embodiments, the method further comprises:

[0118] The recommended window data box is encapsulated into an encapsulation file of the immersive media file, and the encapsulation file of the immersive media file is sent to the media playback device.

[0119] The production and packaging process of immersive media can be referred to Figure 1 and Figure 2 For the sake of brevity, the relevant descriptions in the embodiments are not repeated here.

[0120] Optionally, in some embodiments, the recommended window data box includes:

[0121] The viewpoint switching field is used to indicate whether it is necessary to switch to another viewpoint after presenting a specific recommended viewpoint corresponding to the current viewpoint, wherein the current viewpoint corresponds to at least one recommended viewpoint.

[0122] Optionally, in some embodiments, the recommended window data box includes:

[0123] The target viewpoint field is used to indicate the target viewpoint to switch to after presenting the current recommendation window;

[0124] The target recommendation window field is used to indicate the target recommendation window to be switched to after the current recommendation window is presented and switched to the target viewpoint.

[0125] Optionally, in some embodiments, the recommended view window data box includes: an independent view window field, used to indicate whether the viewpoint corresponds to an independent set of recommended view windows.

[0126] Optionally, in some embodiments, the independent window field takes a first value, indicating that each viewpoint corresponds to an independent set of recommended windows;

[0127] The independent window field takes the second value, indicating that each viewpoint corresponds to the same set of recommended windows.

[0128] Optionally, in some embodiments, the recommended viewport data box further includes: a viewpoint entity group data box for recording labels of various entities constituting a viewpoint.

[0129] Therefore, in an embodiment of the present application, the media production device can encapsulate a recommended window data box based on the viewpoint information and window information of the immersive media, and how to encapsulate the recommended window data box in the encapsulation file of the immersive media, and further send the encapsulation file of the immersive media to the media consumption device, so that the media consumption device decapsulates the encapsulation file of the immersive media to obtain the window data box of the immersive media, and switches the viewpoint and window according to the switching method defined by the window data box, thereby making the presentation form of the immersive media more diversified, thereby improving the user experience.

[0130] The solution of the embodiment of the present application is described in detail below through a complete example.

[0131] First, the media production device encapsulates a recommended view data box based on the viewpoint information of the captured media file A and the recommended view windows corresponding to the viewpoints. It also encapsulates a view entity group data box based on the entities that make up the viewpoints. Media file A has two viewpoints, VPI1 and VPI2, with VPI1 being the initial viewpoint. VPI1 corresponds to recommended view windows vp11, vp12, and vp13, with vp11 being the initial view. VPI2 corresponds to recommended view windows vp21 and vp22, with vp21 being the initial view.

[0132] The information recorded in the recommended window data box is as follows:

[0133] VPI1 (independent_viewport=1; viewpoint_switch=1): vp11, vp12 (des_viewpoint_id=2; des_viewport_id=21), vp13;

[0134] VPI2 (independent_viewport=1; viewpoint_switch=1): vp21, vp22 (des_viewpoint_id=1; des_viewport_id=13).

[0135] Generate a viewpoint entity group data box corresponding to each viewpoint based on the constituent entities of VPI1 and VPI2. VPI1 contains two entity components, "video" and "audio", and VPI2 contains three entity components, "video", "audio", and "backgroundvideo".

[0136] Furthermore, after configuring the recommended window data box, the media production device encapsulates the recommended window data box into an immersive media encapsulation file and sends the immersive media encapsulation file to the media consumption device. For example, upon receiving a request from the media consumption device, the media production device may send the immersive media encapsulation file to the media consumption device.

[0137] On the media consumption device side, if the user chooses to enter the recommended browsing mode, since VPI1 is the initial viewpoint, consumption starts from VPI1. At this time, the video track and audio track are obtained and rendered separately according to the tags of the constituent entities of VPI1.

[0138] In VPI1, recommended view vp11 is the initial view, so it is presented first. Each recommended view in that track is then rendered sequentially according to the sample information for the recommended view. When rendering reaches recommended view vp12, by parsing the metadata in the sample corresponding to vp12, it is determined that after consuming vp12, the view should jump to viewpoint VPI2 and begin consuming recommended view vp21.

[0139] The media consumption device switches to viewpoint VPI2 and recommended window vp21, and obtains the video track, audio track, and background video track according to the labels of the constituent entities of VPI2 and renders them separately.

[0140] After consuming the recommendation window vp21, the media consumption device renders each recommendation window in sequence according to the sample information of the recommendation window track. Specifically, after presenting the recommendation window vp22, according to the metadata information in the recommendation window vp22 sample, after consuming the recommendation window vp22, it jumps to the recommendation window vp13 to continue consumption.

[0141] Therefore, the embodiments of the present application can achieve flexible and independent switching of viewpoints and windows, thereby improving the user experience.

[0142] Combined with the above Figures 3 and 4 , the method embodiment of the embodiment of the present application is elaborated in detail. In order to facilitate better implementation of the above scheme of the embodiment of the present application, the following is combined with Figures 5 to 8 , introduces the device embodiment according to the present application. It should be understood that the device embodiment and the method embodiment correspond to each other. Similar descriptions can refer to the method embodiment. For the sake of brevity, they will not be repeated here.

[0143] See Figure 5 , Figure 5 The following is a schematic diagram showing the structure of an immersive media data processing device provided by an exemplary embodiment of the present application; the immersive media data processing device may be a computer program (including program code) running on a media consumption device, for example, the immersive media data processing device may be an application software in the media consumption device. Figure 5 As shown, the data processing device includes an acquiring unit 501 , a determining unit 502 and a presenting unit 503 . Figure 5 The data processing device shown can be used to perform the above Figure 3 Some or all of the described method embodiments.

[0144] In some embodiments, the acquiring unit 501 is configured to acquire a recommended viewport data box of an immersive media file, where the recommended viewport data box is configured to define viewpoint switching information and recommended viewport switching information of the immersive media file.

[0145] a determining unit 502 for determining a target viewpoint and a target recommended window based on the switching information of the current viewpoint and the switching information of the current recommended window;

[0146] The presenting unit 503 is configured to switch to the target viewpoint and the target recommendation window to present the immersive media file.

[0147] In some embodiments, the recommended window data box includes:

[0148] The viewpoint switching field is used to indicate whether it is necessary to switch to another viewpoint after presenting a specific recommended viewpoint corresponding to the current viewpoint, wherein the current viewpoint corresponds to at least one recommended viewpoint.

[0149] In some embodiments, the recommended window data box includes:

[0150] The target viewpoint field is used to indicate the target viewpoint to switch to after presenting the current recommendation window;

[0151] The target recommendation window field is used to indicate the target recommendation window to be switched to after the current recommendation window is presented and switched to the target viewpoint.

[0152] In some embodiments, the recommended view window data box includes: an independent view window field, used to indicate whether the viewpoint corresponds to an independent set of recommended view windows.

[0153] In some embodiments, the independent window field takes a first value, indicating that each viewpoint corresponds to an independent set of recommended windows;

[0154] The value of the independent window field is the second value, indicating that each viewpoint corresponds to the same set of recommended windows.

[0155] In some embodiments, the recommended viewport data box further includes a viewpoint entity group data box for recording labels of the entities constituting the viewpoint.

[0156] In some implementations, the presenting unit 503 is further configured to:

[0157] Rendering each entity of the target viewpoint according to the label of each entity of the target viewpoint.

[0158] In some implementations, the acquiring unit 501 is specifically configured to:

[0159] Obtain a package file of the immersive media file from a media production device;

[0160] The encapsulated file of the immersive media file is decapsulated to obtain a recommended window data box of the immersive media file.

[0161] According to one embodiment of the present application, Figure 5The various units in the data processing device of the immersive media shown can be individually or completely combined into one or several other units to form a whole, or one (or some) of the units can be further divided into multiple functionally smaller units to form a whole, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present application, the data processing device of the immersive media may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, the following can be executed by running on a general-purpose computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM). Figure 3 A computer program (including program code) for each step of the corresponding method shown in FIG. Figure 3 The immersive media data processing device shown, and the immersive media data processing method for implementing the embodiments of the present application. The computer program can be recorded on, for example, a computer-readable recording medium, and loaded into the above-mentioned computing device via the computer-readable recording medium and executed therein.

[0162] Based on the same inventive concept, the principles and beneficial effects of solving problems provided by the data processing device for immersive media in the embodiment of the present application are similar to the principles and beneficial effects of solving problems provided by the data processing method for immersive media in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0163] See Figure 6 , Figure 6 The following is a schematic diagram showing the structure of another immersive media data processing device provided by an exemplary embodiment of the present application; the immersive media data processing device may be a computer program (including program code) running in a media production device, for example, the immersive media data processing device may be an application software in the media production device. Figure 6 As shown, the immersive media data processing device includes an acquisition unit 601 and an encapsulation unit 602. The detailed description of each unit is as follows:

[0164] An acquiring unit 601 is configured to acquire viewpoint information and window information when presenting an immersive media file;

[0165] The encapsulation unit 602 is configured to encapsulate a recommended window data box according to the viewpoint information and the window information when the immersive media file is presented, wherein the recommended window data box is used to define the viewpoint switching information of the immersive media file and the recommended window switching information.

[0166] In some embodiments, the recommended window data box includes:

[0167] The viewpoint switching field is used to indicate whether it is necessary to switch to another viewpoint after presenting a specific recommended viewpoint corresponding to the current viewpoint, wherein the current viewpoint corresponds to at least one recommended viewpoint.

[0168] In some embodiments, the recommended window data box includes:

[0169] The target viewpoint field is used to indicate the target viewpoint to switch to after presenting the current recommendation window;

[0170] The target recommendation window field is used to indicate the target recommendation window to be switched to after the current recommendation window is presented and switched to the target viewpoint.

[0171] In some embodiments, the recommended view window data box includes: an independent view window field, used to indicate whether the viewpoint corresponds to an independent set of recommended view windows.

[0172] In some embodiments, the independent window field takes a first value, indicating that each viewpoint corresponds to an independent set of recommended windows;

[0173] The value of the independent window field is the second value, indicating that each viewpoint corresponds to the same set of recommended windows.

[0174] In some embodiments, the recommended viewport data box further includes a viewpoint entity group data box for recording labels of the entities constituting the viewpoint.

[0175] In some embodiments, the encapsulation unit 602 is further configured to: encapsulate the recommended window data box into an encapsulation file of the immersive media file;

[0176] The data processing apparatus further includes: a transmission unit configured to send an encapsulated file of the immersive media file to a media playback device.

[0177] According to one embodiment of the present invention, Figure 6The various units in the data processing device of the immersive media shown can be individually or completely combined into one or several other units to form a whole, or one (or some) of the units can be further divided into multiple functionally smaller units to form a whole, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present application, the data processing device of the immersive media may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units. According to another embodiment of the present application, the following can be executed by running on a general-purpose computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM). Figure 4 A computer program (including program code) for each step of the corresponding method shown in FIG. Figure 4 The immersive media data processing device shown, and the immersive media data processing method for implementing the embodiments of the present application. The computer program can be recorded on, for example, a computer-readable recording medium, and loaded into the above-mentioned computing device via the computer-readable recording medium and executed therein.

[0178] Based on the same inventive concept, the principles and beneficial effects of solving problems provided by the data processing device for immersive media in the embodiment of the present application are similar to the principles and beneficial effects of solving problems provided by the data processing method for immersive media in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0179] Figure 7 The following is a schematic diagram showing the structure of a media production device provided by an exemplary embodiment of the present application; the media production device may refer to a computer device used by a provider of immersive media, and the computer device may be a terminal (such as a PC, a smart mobile device (such as a smartphone), etc.) or a server. Figure 7 As shown, the media production device includes a capture device 801, a processor 802, a memory 803 and a transmitter 804.

[0180] Capture device 801 is used to capture real-world audio-visual scenes to obtain raw data for immersive media (including audio and video content synchronized in time and space). Capture device 801 may include, but is not limited to, audio equipment, video equipment, and sensor equipment. Audio equipment may include audio sensors, microphones, etc. Video equipment may include standard cameras, stereo cameras, light field cameras, etc. Sensor equipment may include laser equipment, radar equipment, etc.

[0181] The processor 802 (also called CPU (Central Processing Unit)) is the processing core of the media production device. The processor 802 is suitable for implementing one or more program instructions, and is specifically suitable for loading and executing one or more program instructions to achieve Figure 3 The flow of the immersive media data processing method is shown.

[0182] The memory 803 is a memory device in the media production device, which is used to store programs and media resources. It is understandable that the memory 803 here can include both the built-in storage medium in the media production device and the extended storage medium supported by the media production device. It should be noted that the memory can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory; optionally, it can be at least one memory located away from the aforementioned processor. The memory provides a storage space, which is used to store the operating system of the media production device. In addition, the storage space is also used to store a computer program, which includes program instructions, and the program instructions are suitable for being called and executed by the processor to perform the various steps of the immersive media data processing method. In addition, the memory 803 can also be used to store immersive media files formed after processing by the processor, and the immersive media files include media file resources and media presentation description information.

[0183] The transmitter 804 is used to implement transmission interaction between the media production device and other devices, specifically to implement the transmission of immersive media between the media production device and the media playback device. That is, the media production device transmits the relevant media resources of the immersive media to the media playback device through the transmitter 804.

[0184] See also Figure 7 , the processor 802 may include a converter 821, an encoder 822 and a packager 823;

[0185] Converter 821 is configured to perform a series of conversions on the captured video content, making it suitable for immersive media video encoding. This conversion process may include splicing and projection, and optionally, region packing. Converter 821 can convert the captured 3D video content into 2D images and provide them to the encoder for video encoding.

[0186] The encoder 822 is used to perform audio encoding on the captured audio content to form an audio code stream for immersive media, and is also used to perform video encoding on the 2D image converted by the converter 821 to obtain a video code stream.

[0187] The encapsulator 823 is used to encapsulate the audio and video streams into a file container according to the immersive media file format (such as ISOBMFF) to form an immersive media media file resource. The media file resource can be a media file or a media segment to form an immersive media media file. The encapsulator 823 is also used to record the metadata of the immersive media media file resource using media presentation description information according to the immersive media file format requirements. The encapsulated immersive media file generated by the encapsulator is stored in the memory and provided to the media playback device on demand for immersive media presentation.

[0188] In an exemplary embodiment, the processor 802 (specifically, each device included in the processor) executes the following instructions by calling one or more instructions in the memory 803: Figure 4 Specifically, the memory 803 stores one or more first instructions, which are suitable for being loaded by the processor 802 and executing the following steps:

[0189] Obtaining viewpoint information and window information of immersive media files;

[0190] A recommended window data box is encapsulated according to the viewpoint information and the window information of the immersive media file, wherein the recommended window data box is used to define switching information of the viewpoint of the immersive media file and switching information of the recommended window.

[0191] As an optional implementation, the processor 802 further performs the following operations by running the executable program code in the memory 803: encapsulating the recommended window data box into the encapsulation file of the immersive media file, and sending the encapsulation file of the immersive media file to the media playback device.

[0192] Based on the same inventive concept, the principles and beneficial effects of solving problems provided by the data processing device for immersive media in the embodiment of the present application are similar to the principles and beneficial effects of solving problems provided by the data processing method for immersive media in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0193] Figure 8 The following is a schematic diagram showing the structure of a media playback device provided by an exemplary embodiment of the present application; the media playback device may refer to a computer device used by a user of immersive media, and the computer device may be a terminal (such as a PC, a smart mobile device (such as a smartphone), a VR device (such as a VR helmet, VR glasses, etc.)). Figure 8 As shown, the media playback device includes a receiver 901, a processor 902, a memory 903, and a display / playback device 904.

[0194] The receiver 901 is used to implement decoding and transmission interaction with other devices, specifically to implement the transmission of immersive media between the media production device and the media playback device. That is, the media playback device receives the relevant media resources of the immersive media transmitted by the media production device through the receiver 901.

[0195] The processor 902 (also called CPU (Central Processing Unit)) is the processing core of the media production device. The processor 902 is suitable for implementing one or more program instructions, and is specifically suitable for loading and executing one or more program instructions to achieve Figure 3 The flow of the immersive media data processing method is shown.

[0196] The memory 903 is a memory device in the media playback device, which is used to store programs and media resources. It is understandable that the memory 903 here can include both the built-in storage medium in the media playback device and the extended storage medium supported by the media playback device. It should be noted that the memory 903 can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory; optionally, it can also be at least one memory located away from the aforementioned processor. The memory 903 provides a storage space, which is used to store the operating system of the media playback device. In addition, the storage space is also used to store a computer program, which includes program instructions, and the program instructions are suitable for being called and executed by the processor to perform the various steps of the data processing method of immersive media. In addition, the memory 903 can also be used to store the three-dimensional image of the immersive media formed after processing by the processor, the audio content corresponding to the three-dimensional image, and the information required for rendering the three-dimensional image and audio content.

[0197] The display / playback device 904 is used to output the rendered sound and three-dimensional image.

[0198] See also Figure 8 , the processor 902 may include a parser 921, a decoder 922, a converter 923 and a renderer 924; wherein:

[0199] The parser 921 is used to decapsulate the encapsulated files of the rendered media from the media production device. Specifically, it decapsulates the media file resources according to the file format requirements of the immersive media to obtain audio code streams and video code streams; and provides the audio code streams and video code streams to the decoder 922.

[0200] Decoder 922 decodes the audio stream to obtain audio content, which it then provides to the renderer for audio rendering. Decoder 922 also decodes the video stream to obtain a 2D image. Based on the metadata provided by the media presentation description, if the metadata indicates that the immersive media has undergone a region encapsulation process, the 2D image is a encapsulated image; if the metadata indicates that the immersive media has not undergone a region encapsulation process, the planar image is a projected image.

[0201] Converter 923 converts 2D images into 3D images. If the immersive media has performed a region packing process, converter 923 will first perform region packing on the packed image to obtain a projected image. The projected image is then reconstructed to obtain a 3D image. If the rendering media has not performed a region packing process, converter 923 will directly reconstruct the projected image to obtain a 3D image.

[0202] The renderer 924 is used to render the audio content and 3D images of the immersive media. Specifically, the renderer renders the audio content and 3D images based on the metadata related to rendering and viewport in the media presentation description information. After the rendering is completed, the audio content and 3D images are output to the display / playback device.

[0203] In an exemplary embodiment, the processor 902 (specifically, each device included in the processor) executes by calling one or more instructions in the memory 803. Figure 3 Specifically, the memory 903 stores one or more second instructions, which are suitable for being loaded by the processor 902 and executing the following steps:

[0204] Obtaining a recommended viewport data box of an immersive media file, where the recommended viewport data box is used to define viewpoint switching information of the immersive media file and recommended viewport switching information;

[0205] Determine the target viewpoint and the target recommended window according to the switching information of the current viewpoint and the switching information of the current recommended window;

[0206] Switch to the target viewpoint and the target recommendation window to present the immersive media file.

[0207] As an optional implementation, the processor 902 further performs the following operations by running the executable program code in the memory 903: obtaining a package file of the immersive media file from the media production device;

[0208] The encapsulated file of the immersive media file is decapsulated to obtain a recommended window data box of the immersive media file.

[0209] Based on the same inventive concept, the principles and beneficial effects of solving problems provided by the data processing device for immersive media in the embodiment of the present application are similar to the principles and beneficial effects of solving problems provided by the data processing method for immersive media in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0210] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded by a processor and executing the above method embodiment.

[0211] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above-described method embodiment.

[0212] It should be noted that for the aforementioned various method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0213] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0214] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0215] A person skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0216] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiment may be completed by hardware integrated logic circuits in the processor or software instructions.

[0217] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A method for processing data of an immersive media file, characterized in that: include: Obtaining a recommended viewport data box for an immersive media file, the recommended viewport data box being used to define viewpoint switching information for the immersive media file and recommended viewport switching information. The viewpoint refers to the immersive media content captured by a capture device during the creation of the immersive media file, and the recommended viewport refers to the portion of the immersive media content recommended for viewing by a user during the presentation of the immersive media file. Determine the target viewpoint and the target recommended window according to the switching information of the current viewpoint and the switching information of the current recommended window; Switching to the target viewpoint and the target recommendation window to present the immersive media file; The recommended viewport data box includes a viewpoint entity group data box for recording labels of entities constituting a viewpoint. Before presenting the immersive media file, the recommended viewport data box further includes: Rendering the entities of the target viewpoint according to the labels of the entities of the target viewpoint, where the entities of the target viewpoint are the media components of the target viewpoint; Render each target recommended window in the recommended window track in sequence according to the sample information of the recommended window track.

2. The method according to claim 1, characterized in that The recommended window data box includes: The viewpoint switching field is used to indicate whether it is necessary to switch to another viewpoint after presenting a specific recommended viewpoint corresponding to the current viewpoint, wherein the current viewpoint corresponds to at least one recommended viewpoint.

3. The method according to claim 1 or 2, characterized in that The recommended window data box includes: The target viewpoint field is used to indicate the target viewpoint to switch to after presenting the current recommendation window; The target recommendation window field is used to indicate the target recommendation window to be switched to after the current recommendation window is presented and switched to the target viewpoint.

4. The method according to claim 1 or 2, characterized in that The recommended window data box includes: an independent window field, which is used to indicate whether the viewpoint corresponds to an independent set of recommended windows.

5. The method according to claim 4, characterized in that The value of the independent window field is the first value, indicating that each viewpoint corresponds to an independent set of recommended windows; The independent window field takes the second value, indicating that each viewpoint corresponds to the same set of recommended windows.

6. The method according to claim 1, wherein The step of obtaining the recommended viewport data box of the immersive media file includes: Obtaining a package file of the immersive media file from a media production device; The encapsulated file of the immersive media file is decapsulated to obtain a recommended window data box of the immersive media file.

7. A method for processing data of an immersive media file, characterized in that: include: Obtaining viewpoint information and window information of immersive media files; Encapsulating a recommended window data box based on the viewpoint information and viewport information of the immersive media file, wherein the recommended window data box is used to define viewpoint switching information and recommended window switching information of the immersive media file, the viewpoint refers to the immersive media content captured by a capture device during the production of the immersive media file, and the recommended window refers to the portion of the immersive media content that is recommended and viewable by the user during the presentation of the immersive media file; encapsulating the recommended view window data box into an encapsulation file of the immersive media file, and sending the encapsulation file of the immersive media file to a media playback device, so that the media playback device determines a target viewpoint and a target recommended window based on the switching information of the current viewpoint and the switching information of the current recommended window, switches to the target viewpoint and the target recommended window, and presents the immersive media file; Among them, the recommended window data box includes a viewpoint entity group data box, which is used to record the labels of the various entities that constitute the viewpoint. Before the media playback device presents the immersive media file, it also includes: rendering the various entities of the target viewpoint according to the labels of the various entities of the target viewpoint, and the various entities of the target viewpoint are the various media components of the target viewpoint; rendering the various target recommended windows in the recommended window track in sequence according to the sample information of the recommended window track.

8. The method according to claim 7, characterized in that The recommended window data box includes: A viewpoint switching field is used to indicate whether to switch to another viewpoint after presenting a specific recommended viewpoint corresponding to the current viewpoint, wherein the current viewpoint corresponds to at least one recommended viewpoint; The target viewpoint field is used to indicate the target viewpoint to switch to after presenting the current recommendation window; The target recommendation window field is used to indicate the target recommendation window to be switched to after the current recommendation window is presented and switched to the target viewpoint.

9. A data processing device for immersive media files, characterized in that: include: an acquisition unit, configured to acquire a recommended viewport data box of an immersive media file, the recommended viewport data box defining viewpoint switching information of the immersive media file and recommended viewport switching information, the viewpoint being immersive media content captured by a capture device during the creation of the immersive media file, and the recommended viewport being a portion of the immersive media content recommended for viewing by a user during the presentation of the immersive media file; a determining unit, configured to determine a target viewpoint and a target recommended window according to the switching information of the current viewpoint and the switching information of the current recommended window; a presenting unit, configured to switch to the target viewpoint and the target recommendation window, and present the immersive media file; Among them, the recommended window data box includes a viewpoint entity group data box, which is used to record the labels of the various entities that constitute the viewpoint. Before presenting the immersive media file, the presentation unit is also used to: render the various entities of the target viewpoint according to the labels of the various entities of the target viewpoint, and the various entities of the target viewpoint are the various media components of the target viewpoint; and render the various target recommended windows in the recommended window track in sequence according to the sample information of the recommended window track.

10. A data processing device for immersive media files, characterized in that: include: An acquisition unit, configured to acquire viewpoint information and window information when presenting an immersive media file; an encapsulation unit for encapsulating a recommended window data box based on viewpoint information and window information during presentation of the immersive media file, wherein the recommended window data box is used to define viewpoint switching information and recommended window switching information of the immersive media file, the viewpoint refers to the immersive media content correspondingly captured by a capture device during production of the immersive media file, and the recommended window refers to the portion of the immersive media content that is recommended and viewable by a user during presentation of the immersive media file; encapsulating the recommended window data box into an encapsulation file of the immersive media file, and sending the encapsulation file of the immersive media file to a media playback device, so that the media playback device determines a target viewpoint and a target recommended window based on the switching information of the current viewpoint and the switching information of the current recommended window, and switches to the target viewpoint and the target recommended window to present the immersive media file; Among them, the recommended window data box includes a viewpoint entity group data box, which is used to record the labels of the various entities that constitute the viewpoint. Before the media playback device presents the immersive media file, it also includes: rendering the various entities of the target viewpoint according to the labels of the various entities of the target viewpoint, and the various entities of the target viewpoint are the various media components of the target viewpoint; rendering the various target recommended windows in the recommended window track in sequence according to the sample information of the recommended window track.

11. A data processing device for immersive media files, characterized in that: include: a processor adapted to execute a computer program; A computer-readable storage medium having a computer program stored therein, wherein when the computer program is executed by the processor, the computer program implements the data processing method for an immersive media file according to any one of claims 1 to 6, or the data processing method for an immersive media file according to any one of claims 7 to 8.

12. A computer-readable storage medium, characterized in that Used to store a computer program, wherein the computer program enables a computer to execute the data processing method of an immersive media file according to any one of claims 1 to 6, or implement the data processing method of an immersive media file according to any one of claims 7 to 8.

Images