Information processing device, information processing method

Abstract

To provide a technology for optimally performing 3D data rendering and reducing processing load. [Solution] 3D data and metadata related to the 3D data are stored in a file in a specified format. The priority level for displaying each object, or at least a portion of each object, contained in the 3D data is stored in the file.

Description

Technical Field

[0001] This disclosure relates to information processing technology.

Background Art

[0002] As a method for generating 3D (three-dimensional) data, a method for generating 3D data using computer graphics has been conventionally known. However, recently, a method of scanning the shape and texture of an object such as an actual object or person using a dedicated device or studio and obtaining 3D data has been carried out.

[0003] In recent years, there has been a trend to use the 3D data thus obtained for autonomous driving or driving support, or as data for free-viewpoint video to be displayed on a display device such as a head-mounted display.

[0004] On the other hand, since 3D data generally has a large amount of data, under the umbrella of ISO (International Organization for Standardization) and IEC (International Electrotechnical Commission), the MPEG (Moving Picture Experts Group) is promoting the standardization of a standard for encoding 3D data and a file format standard for storing the encoded 3D data.

[0005] 3D data is generally handled in data formats such as point cloud data and 3D mesh data. In MPEG, for example, as an example of a standard for encoding point cloud data, ISO / IEC 23090-5 Visual Volumetric Video-based Coding (V3C) and Video-based Point Cloud Compression (V-PCC) (hereinafter referred to as V3C / V-PCC) has been standardized, which uses a video codec for encoding. ISO / IEC 23090-10 Carriage of visual volumetric video-based coding data (hereinafter referred to as Carriage of V3C) has been standardized as a standard for storing point cloud data encoded according to the above standard into a file.

[0006] Incidentally, when using the aforementioned 3D data as data for free-viewpoint video, the rendering processing load for displaying the 3D data can become a problem. For example, when viewing team sports such as basketball or soccer in free-viewpoint video, it is necessary to simultaneously render 3D data of multiple players scanned during the game. Furthermore, 3D data obtained by scanning people tends to require high-quality data with high resolution of shape data and high detail of textures, and since higher-quality data results in a larger data volume, the processing load during rendering tends to increase as well. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2022-103836 [Overview of the project] [Problems that the invention aims to solve]

[0008] Therefore, there is a need for methods to reduce the processing load of 3D data display. This disclosure provides a technology for suitably performing 3D data display processing and reducing the processing load. [Means for solving the problem]

[0009] One aspect of this disclosure is a storage control means for storing 3D data and metadata related to the 3D data in a file in a specified format, wherein the storage control means stores in the file the priority level for displaying each object, or at least a portion of each object, included in the 3D data. [Effects of the Invention]

[0010] According to this disclosure, it is possible to provide a technology for suitably performing 3D data display processing and reducing processing load. [Brief explanation of the drawing]

[0011] [Figure 1] A block diagram showing an example of the hardware configuration of the information processing device 100. [Figure 2] A flowchart of the processing performed by the information processing device 100. [Figure 3] A diagram showing an example of the structure of a 3D media file. [Figure 4] A diagram showing an example of metadata description that sets a priority level for an object. [Figure 5A] A diagram illustrating how to view an object from a specific viewpoint. [Figure 5B] A diagram illustrating how to view an object from a specific viewpoint. [Figure 5C] A diagram illustrating how to view an object from a specific viewpoint. [Figure 6] This diagram shows an example of metadata description that sets the priority level of an object in relation to the viewport. [Figure 7] A flowchart of the processing performed by the information processing device 100. [Figure 8] A schematic diagram illustrating a case where only a portion of an object that can be divided into multiple sub-objects is viewed. [Figure 9] A schematic diagram illustrating the case of viewing an object that has been divided into many small tiles (divided object). [Modes for carrying out the invention]

[0012] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the disclosures relating to the claims. While the embodiments describe multiple features, not all of these features are essential to the disclosure, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0013] [First Embodiment] First, an example of the hardware configuration of the information processing device 100 according to this embodiment will be explained using the block diagram in Figure 1. The information processing device 100 according to this embodiment can be a computer device such as a PC (personal computer), a tablet terminal, or a smartphone.

[0014] The CPU 102 executes various processes using computer programs and data stored in the RAM 103. In doing so, the CPU 102 controls the overall operation of the information processing device 100, and also executes or controls the various processes described as those performed by the information processing device 100.

[0015] The RAM 103 has an area for storing computer programs and data loaded from the ROM 104 and the non-volatile memory 113, and an area for storing computer programs and data received from an external device via the communication unit 107. Further, the RAM 103 has a work area used when the CPU 102, the acquisition unit 108, the analysis unit 109, the setting unit 110, the processing unit 111, and the encoding unit 112 execute various processes. Thus, the RAM 103 can appropriately provide various areas.

[0016] The ROM 104 stores setting data of the information processing apparatus 100, computer programs and data related to the startup of the information processing apparatus 100, computer programs and data related to the basic operations of the information processing apparatus 100, and the like.

[0017] The operation input unit 105 is a user interface such as a keyboard, a mouse, a touch panel, etc., and various instructions and information can be input to the information processing apparatus 100 by the user's operation.

[0018] The display unit 106 has a liquid crystal screen or a touch panel screen, and can display the processing result by the CPU 102 as an image, characters, etc. Note that the display unit 106 may be a projection device such as a projector that projects images and characters.

[0019] Also, the display unit 106 may be a touch panel and the operation input unit 105 may include a touch panel sensor. In that case, when the operation input unit 105 detects that an operation input has been made on the user interface screen displayed on the display unit 106, it outputs a control signal indicating that to the CPU 102.

[0020] The communication unit 107 performs data communication with external devices via a network such as a LAN or the Internet. For example, the communication unit 107 is the PHY and MAC (Transmission Media Control Processor) of Ethernet® for a wired LAN. Also, if the information processing device 100 can connect to a wireless LAN, the communication unit 107 includes a controller that performs wireless LAN control such as IEEE802.11a / b / g / n / ac, an RF circuit, and an antenna.

[0021] The operation of each of the acquisition unit 108, analysis unit 109, setting unit 110, processing unit 111, and encoding unit 112 will be described later. In this embodiment, we will describe the case in which the acquisition unit 108, analysis unit 109, setting unit 110, processing unit 111, and encoding unit 112 are all implemented in hardware. Note that the acquisition unit 108, analysis unit 109, setting unit 110, processing unit 111, and encoding unit 112 may each be separate hardware components, or two or more functional units may be implemented in a single hardware component. Furthermore, one or more of the acquisition unit 108, analysis unit 109, setting unit 110, processing unit 111, and encoding unit 112 may be implemented in software (computer program). In that case, the CPU 102 executes the computer program corresponding to that functional unit to realize the function corresponding to that functional unit.

[0022] Non-volatile memory 113 is, for example, flash memory such as an SD card or SSD, or a magnetic recording device such as a hard disk drive. Non-volatile memory 113 stores the OS (operating system), computer programs and data for causing the CPU 102 to execute or control various processes described as processes performed by the information processing device 100.

[0023] The CPU 102, RAM 103, ROM 104, operation input unit 105, display unit 106, communication unit 107, acquisition unit 108, analysis unit 109, setting unit 110, processing unit 111, encoding unit 112, and non-volatile memory 113 are all connected to the system bus 101.

[0024] Next, a series of processes for generating a 3D media file by detecting objects contained in 3D data and storing metadata related to the detected objects and encoded data obtained by encoding the 3D data in a specified format will be explained according to the flowchart in Figure 2.

[0025] In step S201, the acquisition unit 108 acquires 3D data, which includes point clouds and 3D meshes that define an object. The method by which the acquisition unit 108 acquires 3D data is not limited to a specific method. For example, the acquisition unit 108 may acquire 3D data received by the communication unit 107 from an external device, or it may acquire 3D data that has been pre-stored in the non-volatile memory 113. Note that 3D data is dynamic data whose content can change over time and is generated for each frame, so in step S201, the acquisition unit 108 acquires 3D data for multiple frames.

[0026] In step S202, the acquisition unit 108 loads (stores) the 3D data of the first frame from the 3D data for each frame acquired in step S201 into the RAM 103. Note that the method and configuration for acquiring the 3D data of each frame into the RAM 103 are not limited to a specific method or configuration, as long as the processing in steps S203 to S209 can be performed for the 3D data of each frame in multiple frames.

[0027] In step S203, the analysis unit 109 analyzes the 3D data of the frame stored in the RAM 103 (hereinafter referred to as the target frame) and determines whether or not the 3D data contains an object. The object is not limited to a specific type of object. The analysis unit 109 may use AI technology, such as machine learning, to determine whether or not the 3D data contains an object.

[0028] As a result of this determination, if the 3D data of the target frame contains an object (if an object is detected from the 3D data of the target frame), the process proceeds to step S205 via step S204.

[0029] On the other hand, if the 3D data of the target frame does not contain an object (if no object is detected from the 3D data of the target frame), the process proceeds to step S208 via step S204.

[0030] In step S205, the processing unit 111 assigns a unique identifier to each object included in the 3D data of the target frame. Each object in the 3D data is assigned a different identifier, and the same object is assigned the same identifier across frames.

[0031] In step S206, the analysis unit 109 sets a bounding box for each object included in the 3D data of the target frame. "Setting a bounding box" means setting the position (3D coordinates) and size (length, width, height) of the bounding box in three-dimensional space. For example, the analysis unit 109 sets the center coordinates of the object as the position of the bounding box, and defines the bounding box as the area that extends from this position to surround the object in the directions of the three axes X, Y, and Z, and defines the size of the bounding box as the length of the bounding box in each of the X, Y, and Z axes. The analysis unit 109 may also define the rotation angles of the bounding box with respect to the three axes X, Y, and Z. This means that the bounding box does not necessarily have to be composed of planes orthogonal to the three axes X, Y, and Z. Also, the bounding box does not have to be an area that includes the entire object, but may be an area that surrounds only a part of the object.

[0032] In step S207, the setting unit 110 sets the display priority level for each object, or the bounding box, included in the 3D data of the target frame. The method for setting the priority level for an object or bounding box is not limited to a specific method; for example, the user may input a priority level by operating the operation input unit 105.

[0033] In this embodiment, the priority level may represent either priority or recommendation level. Generally, higher priority is assigned to items that are given priority. In addition, the creator's intent for the content (object) is taken into consideration, and content that is of higher importance as content is assigned a higher recommendation level. Note that the priority level is not limited to being set in step S207, but may be set before step S207.

[0034] The processing unit 111 then performs data processing in accordance with the specifications of a specified file format on the identifier assigned to the object, the position and size of the bounding box, and the priority level set for the object or bounding box, and generates "metadata in a form compliant with the specifications of a specified file format" that includes this information. This metadata is metadata related to the 3D data acquired in step S201.

[0035] In step S208, the analysis unit 109 determines whether or not it has analyzed the 3D data for all frames. If it determines that it has analyzed the 3D data for all frames, the process proceeds to step S210. If there is still 3D data for frames that have not yet been analyzed, the process proceeds to step S209.

[0036] In step S209, the acquisition unit 108 loads (stores) the 3D data of the frame following the target frame from the 3D data for each frame acquired in step S201 into the RAM 103.

[0037] In step S210, the encoding unit 112 generates encoded data of the 3D data of all frames by encoding the 3D data in a manner compliant with an encoding standard such as V3C / V-PCC.

[0038] In step S211, the processing unit 111 stores the encoded data generated in step S210 and the metadata generated in step S207 in a file conforming to a file format standard such as Carriage of V3C (storage control). Then, the processing unit 111 outputs such a file as a 3D media file. The output destination of the 3D media file is not limited to a specific destination; for example, the processing unit 111 may output the 3D media file to the non-volatile memory 113, or it may transmit it to an external device via the communication unit 107.

[0039] Next, an example of the configuration of a 3D media file generated by the information processing device 100 will be explained using Figure 3. The file format explained in Figure 3 is based on the ISO Base Media File Format (hereinafter referred to as ISOBMFF), which is the basic specification for media files standardized by MPEG.

[0040] In Figure 3, moov301 includes an atlas track 302, a geometry track 310, an attribute track 311, an occupancy track 312, and a metadata track 313.

[0041] Atlas Track 302 stores metadata such as the region where encoded data (encoded 3D data) stored in the 3D media file exists, and information about the configuration of the encoded data.

[0042] The geometry track 310 is a track for managing coordinate information to indicate the shape of an object, and the attribute track 311 is a track for managing attribute information, including information such as the color and light reflectivity of the object's surface. The occupancy track 312 is a track for managing information that identifies the three-dimensional space in which the object exists, and the metadata track 313 is a track that can store attribute information of 3D data that changes over time.

[0043] Furthermore, these four tracks—Geometry Track 310, Attribute Track 311, Occupancy Track 312, and Metadata Track 313—are linked by Atlas Track 302 and Track Reference 305. The v3vg306, v3va307, v3vo308, and cdsc309 contained in Track Reference 305 indicate the reference type of each associated track.

[0044] Figure 3 shows an example of track configuration when the point cloud defining an object is encoded using the V3C / V-PCC encoding standard, and the mdat315 stores data managed by each track, such as encoded data.

[0045] In this embodiment, metadata track 313 is used to store the priority level of an object or bounding box as metadata associated with the object or bounding box in a 3D media file. Figure 4 shows an example of metadata description for setting a priority level for an object.

[0046] In Figure 4, DynamicVolumetricMetadataSampleEntry401 is one of the sample entries defined in Carriage of V3C, and this sample entry can be included in metadata track 313 as metadata sample entry 314 in Figure 3. In this case, the spatial region where the 3D data transmitted by atlas track 302 resides is considered a dynamic region.

[0047] According to Carriage of V3C, when scene object information, which indicates the state of an object in 3D space, changes over time, V3CVolumetricMetadataSample402 can show changes in the position and size of the bounding box surrounding the detected object. This sample is stored in mdat315 as metadata sample 316 in Figure 3, and if the scene object information changes, there will be one or more V3CVolumetricMetadataSample402 instances.

[0048] In this embodiment, with respect to V3CVolumetricMetadataSample402, for example, a parameter called priority_value403 is added. In V3CVolumetricMetadataSample402, num_regions indicates the number of bounding boxes surrounding the detected object, and region includes the position and size of the bounding box, the identifier of the object contained within the bounding box, etc. Therefore, the value of priority_value403 is a numerical value representing the priority level for each bounding box. Note that the value of priority_value403 may be defined such that, for example, 0 (zero) is the highest priority level, and the priority level decreases as the value increases.

[0049] Up to this point, we have described the case in which DynamicVolumetricMetadataSampleEntry401 is stored as metadata sample entry 314 in metadata track 313. On the other hand, we will now describe the case in which V3CSpatialRegionCollectionBox404, which is included in DynamicVolumetricMetadataSampleEntry401, is directly stored as V3C atlas sample entry 304 in sample entry 303 of atlas track 302.

[0050] According to Carriage of V3C, in this case, the spatial region where the 3D data transmitted by the Atlas Track 302 exists is considered a static region, so the scene object information indicating the state of an object in 3D space does not change over time. Therefore, in this embodiment, a priority level for each bounding box is defined by adding a parameter such as priority_value405 to the V3CSpatialRegionCollectionBox404.

[0051] By defining a priority level for each bounding box surrounding a detected object in this way, when reproducing objects, it becomes possible to display them in order from those with higher priority levels, for example, depending on the processing capacity of the device performing the reproduction.

[0052] Furthermore, if there are objects that you want to exclude from the playback process, you can, for example, set a value such as `priority_value403` or `priority_value405` to exclude them from the playback process.

[0053] In the explanation of the file structure using Figure 3, moov301 was described as a multi-track configuration containing multiple tracks: Atlas Track 302, Geometry Track 310, Attribute Track 311, Occupancy Track 312, and Metadata Track 313. However, according to Carriage of V3C, it is also possible to have a single-track configuration in which multiple tracks managing 3D encoded data are combined into one, excluding the Metadata Track 313.

[0054] In a single-track configuration, only one track called a V3C bitstream track is generated, which combines the atlas track 302, geometry track 310, attribute track 311, and occupancy track 312 into one. Multiple different samples are defined in the sample entries within the V3C bitstream track, and different types of samples are managed by treating a composite sample, which combines different types of samples with the same presentation time, as a single sample within the V3C bitstream track.

[0055] In this type of single-track configuration, the multi-track configuration described above can be replaced by associating the V3C bitstream and the track metadata track using a cdsc type track reference.

[0056] Next, we will explain how to view an object from a specific viewpoint using Figures 5A to 5C. Figure 5A shows a view of a group of objects defined by 3D data (objects 504, 505, and 506) from viewpoint 501, in the line of sight direction 502, with a field of view angle 503. Hereafter, the combination of viewpoint 501, line of sight direction 502, and field of view angle 503 will be referred to as viewport A.

[0057] The three objects shown above, which are being viewed in viewport A, are objects detected from the 3D data through the analysis process in step S203. Furthermore, bounding boxes 507-509 in Figure 5B are the bounding boxes set for objects 504-506 in step S206.

[0058] Here, if we consider objects 504 and 505 to be important objects for viewing, but object 506 to be not, then, for example, as shown in the field of view 510 in Figure 5C, the important objects 504 and 505 may be obstructed by the unimportant object 506. In other words, when viewing in viewport A, it is desirable to remove object 506 which interferes with viewing, so that objects 504 and 505, which are not obstructed by object 506, can be viewed, as shown in the field of view 511.

[0059] On the other hand, in Figure 5A, when viewing the above group of objects in viewport B, which differs from viewport A in terms of viewpoint, line of sight, and field of view, object 506 does not interfere with viewing and therefore does not need to be removed. In other words, when viewing the same group of objects in different viewports, even at the same time, the objects that are recommended to be viewed and the objects that are not recommended may differ depending on the viewport.

[0060] Therefore, we will explain how to associate viewports with object priority levels using Figure 6. Figure 6 is a diagram showing an example of metadata description that associates object priority levels with viewports. In Figure 6, we show an example of metadata description that associates object priority levels with viewport information, which includes information on viewpoint, line of sight, and field of view in 3D data space.

[0061] ViewportInfoSampleEntry601 is one of the sample entries defined in Carriage of V3C, and this sample entry can be included in metadata track 313 as metadata sample entry 314 in Figure 3. Note that the configuration described using Figure 4 can coexist with the configuration described below, but in that case, metadata track 313 must be a different track.

[0062] In this embodiment, description A603 is added to ViewportInfoConfigurationBox602, which is included in ViewportInfoSampleEntry601. Description A603 defines the recommended / deprecated status for each object.

[0063] `num_of_objects` indicates the number of objects in the viewport that are designated as deprecated or deprecated.

[0064] The `object_deprecation_flag` flag indicates whether an object is a deprecated or deprecated object. For example, if `object_deprecation_flag` is 0, it means that the object with the following identifier is a deprecated object. On the other hand, if `object_deprecation_flag` is 1, it means that the object with the following identifier is a deprecated object.

[0065] soi_object_idx[i] is intended to be an identifier for each object included in the Scene object information SEI (Supplemental Enhancement Information) as defined in V3C / V-PCC. In other words, by using object_deprecation_flag, it is possible to reduce the amount of code because it is not necessary to describe the identifier of all objects, especially when there are many objects. The description method described here is just one example, and it is sufficient if it contains information that can identify whether an object is recommended or deprecated.

[0066] Furthermore, by adding description B604 to ViewportInfoConfigurationBox602 instead of description A603, you can define the priority for each object. The priority_value in description B604 is a parameter that indicates the priority.

[0067] Furthermore, defining recommended / deprecated behaviors for each bounding box surrounding an object can be achieved by adding a description like C605 to ViewportInfoConfigurationBox602.

[0068] Furthermore, defining the priority of each bounding box surrounding an object can be achieved by adding a description like description D606 to ViewportInfoConfigurationBox602.

[0069] Furthermore, according to Carriage of V3C, if the viewport changes dynamically, viewport information can be defined as a sample in metadata track 313. ViewportInfoSample607 is a sample that stores viewport information defined in Carriage of V3C, and the viewport information is included in ViewportInfo608.

[0070] In this embodiment, the recommendation / deprecation status for each object can be indicated by adding description E609 to ViewportInfo608. The roles of the parameters described in description E609 are as explained in description A. Furthermore, by replacing description E609 with descriptions B604, C605, and D606, it is also possible to define the priority for each object, the recommendation / deprecation status for each bounding box surrounding an object, and the priority for each bounding box surrounding an object, respectively.

[0071] Another way to define the priority of objects or the bounding boxes surrounding them is to list the identifiers of the objects or bounding boxes in order of priority (highest to lowest or lowest to highest). In this case, objects or bounding boxes whose identifiers are not listed may be implicitly defined as having the lowest (or highest) priority level. The same applies to recommendation levels. For example, if unlisted objects are not displayed because they have the lowest priority level, there is no need to re-identify the objects included in the viewport. Only the listed objects need to be displayed according to the specified priority level, which can reduce the processing load during display, especially when there are many objects.

[0072] Furthermore, in free-viewpoint video, the viewport generally changes dynamically over time. However, as explained using Figure 6, viewport information can be described as a sample in metadata track 313. In other words, by generating a timed metadata track that stores viewport information, the changes in the viewport over time can be defined. Therefore, if there are multiple viewports, a timed metadata track storing viewport information can be generated for each viewport.

[0073] Furthermore, the timed metadata track that stores viewport information may also be used to track specific objects. For example, in sports content where multiple players are playing simultaneously, a viewport focusing on a specific player may be defined. In that case, you can set soi_object_idx[i] or the region defined in V3CSpatialRegion to track the specific player.

[0074] ViewportInfoSample607 allows you to define as many viewports as there are num_viewports, so you could define a viewport for each specific player. However, in the case of delivering sports content, for example, separating tracks for each player you're focusing on allows viewers to receive only the track containing viewport information for that player, thus prioritizing viewing that player. Furthermore, by setting a priority level that indicates a higher priority for a specific player and a lower priority level for other players or objects, it becomes possible to prioritize playback of that specific player.

[0075] Furthermore, to facilitate the control of showing / hiding each object, the 3D encoded data for each object may be treated as independent 3D encoded data. When storing multiple independent encoded data in a file, in the file structure explained using Figure 3, the track managing the 3D encoded data consists of five tracks, including the atlas track 302, the geometry track 310 associated with the atlas track 310, the attribute track 311, the occupancy track 312, and the metadata track 313. However, to manage independent encoded data for each object, these five tracks may be generated for each object.

[0076] Furthermore, when displaying multiple objects consisting of independent 3D encoded data in the same 3D space, it is necessary to align the coordinate origin, axis direction, tilt, and scale, which serve as the reference coordinates for the 3D space. Generally, 3D encoded data sets reference coordinates in 3D space, and the position, shape, and movement of an object can be represented by the position information at those coordinates. Therefore, a simple method for aligning the coordinate origin is to capture multiple objects in an environment where the same coordinate origin has been set in advance. This allows objects to have the same coordinate origin and scale, even if they are independent as 3D encoded data. This is possible in studios dedicated to 3D capture, or in 3D capture systems installed in stadiums, etc., which are mainly used to capture athletes.

[0077] On the other hand, when displaying objects captured in different environments or computer-generated CG objects in the same 3D space, it is necessary to define common global coordinates among multiple tracks that manage the 3D encoded data to be displayed in the same 3D space. In other words, this can be achieved by generating or acquiring information such as the origin and axial offset of each object relative to the common global coordinates, as well as tilt and scale information for each object, and storing it, for example, in the metadata track of each object.

[0078] In this way, when 3D encoded data is stored independently for each object as separate tracks, priority or recommendation level can be defined at the track level rather than the object level. That is, priority or recommendation level can be set for each track ID, which is the track identification information. However, as explained above, if there are multiple tracks managing the same 3D encoded data, for example in the example in Figure 3, the atlas track 302, which stores the reference information for each track, can be designated as the representative track, and the track ID and priority or recommendation level can be set for the representative track.

[0079] Incidentally, viewport information is intended to be used to define the viewing point recommended by the content creator when viewing content as free-viewpoint video, and may be used as the default viewing coordinates when viewing content with a head-mounted display, especially when the viewing coordinates are not specified from the head-mounted display side. In such cases, if there are multiple timed metadata tracks storing viewport information, it is unclear which track is recommended or has a high degree of recommendation.

[0080] Therefore, the simplest method of determination is to implicitly define that a smaller track ID number in a timed metadata track that stores viewport information indicates a higher or lower recommendation level. Alternatively, this can be achieved by creating a group of timed metadata tracks that store viewport information within a track-level meta box, for example, as an EntityToGroupBox, and describing information that identifies the recommendation level or priority within that group.

[0081] Next, we will explain in more detail how to construct objects and bounding boxes using Figure 8. Figure 8 is a schematic diagram showing a case where a portion of an object that can be divided into multiple sub-objects is viewed.

[0082] In Figure 8, viewport C consists of viewpoint 801, line of sight direction 802, and field of view 803, with field of view 804 showing an example of an image viewable at field of view 803. Furthermore, object 810 consists of sub-objects 811 and 812, and both sub-objects can be decoded and rendered independently.

[0083] Here, the field of view of viewport C shown in Figure 8 contains only sub-object 811. In other words, since the data for the object necessary for viewing is only that of sub-object 811, the bounding box 805 defines only the region that contains only sub-object 811. By making it possible to acquire only the data of sub-object 811, it is possible to suppress the acquisition of unnecessary data that is not used for viewing.

[0084] Therefore, in order to obtain only the data necessary for viewing, it is desirable to be able to divide an object into multiple regions and encode them. For example, V3C / V-PCC, one of the point cloud coding standards, supports a function to divide an object into tiles and encode them for purposes such as improving parallel processing and spatial random access, and each divided tile can be decoded independently. Here, "tiled" means a state in which the object is divided into multiple regions by a plane horizontal or perpendicular to the coordinate axes in the 3D space in which the object exists, and this method of dividing and encoding in a tiled manner so that it can be decoded independently will be called tile coding from now on.

[0085] In Figure 8, by dividing the sub-objects 811 and 812 that constitute object 810 into independent tiles and tile encoding them, the sub-object 812, which is not included in the field of view 803 of viewport C, does not require processing such as acquiring encoded data, decoding, and rendering.

[0086] Next, we will explain the case where the object is further subdivided and tiled, using Figure 9. Figure 9 is a schematic diagram showing the case of viewing an object that has been finely tiled (divided object).

[0087] In Figure 9, viewport D consists of viewpoint 901, line of sight direction 902, and field of view 903, viewing object 910 from slightly above. Object 910 is divided into multiple regions in 3D space, and each of these regions is encoded as a tile. Here, the tiles necessary for viewing from viewport D are those that are included in the field of view, but tiles located in occlusion regions (blind spots) that do not contain data to be rendered do not need to be acquired. In other words, even if an occlusion region is included in the field of view, encoded data that is not used for rendering is not necessary for viewing.

[0088] In Figure 9, object 910 is shown to be divided into tiles with each edge divided into three sections, and each section is tile-encoded. Sub-object 911 is the set of tiles necessary for viewing at the field of view 903 of viewport D, while sub-object 912 is the set of tiles included in the occlusion region of viewport D. Therefore, although object 910 contains all tiles within the field of view 903 of viewport D, only sub-object 911 is needed for rendering, and sub-object 912 does not need to be obtained.

[0089] In this way, by utilizing encoding mechanisms that subdivide objects, such as tile coding, and allow for partial decoding, it becomes possible to omit data from areas not included in the field of view when viewed from a specific viewport, as well as data from occlusion areas, enabling efficient processing of only the data necessary for viewing.

[0090] In tile coding, an identifier is generally assigned to each tile to identify it, and this is also the case with V3C / V-PCC. Here, when viewing from a desired viewport, in order to obtain only the tiles corresponding to sub-object 911 in Figure 9, it is strictly necessary to know the identifiers of the tiles that make up sub-object 911. This can be done by associating a list of identifiers of one or more tiles necessary for viewing (rendering) with ViewportInfoSampleEntry 601 or ViewportInfoSample 607, which store viewport information as explained using Figure 6.

[0091] Additionally, recommended / deprecated or priority level information, such as descriptions A603 and B604 in Figure 6, may be specified for each sub-object. By explicitly specifying sub-objects that are not needed for viewing, the playback side can easily identify which sub-objects need to be displayed, thereby reducing the processing load.

[0092] The embodiments described so far have used V3C / V-PCC as the encoding standard for 3D data, but the effects do not depend on a specific encoding standard in the implementation. That is, in addition to V3C / V-PCC, other standards such as ISO / IEC 23090-9 Geometry-based Point Cloud Compression (G-PCC), which is also standardized in MPEG, or ISO / IEC 23090-29 Video-based dynamic mesh coding (V-DMC), which is an encoding standard for 3D meshes, may also be used. Other 3D data encoding standards such as Gaussian Splat may also be used. Similarly, while the Carriage of V3C has been described as a file format standard in the embodiments described so far, implementations do not depend on a specific file format standard. For example, the same storage method is possible with other storage standards depending on the encoding format of the 3D encoded data to be stored, such as ISO / IEC 23090-18 Carriage of Geometry-based Point Cloud Compression Data, which is a standard for storing the above-mentioned G-PCC encoded data. Furthermore, common information that is not dependent on encoding standards, such as viewport information in 3D space, is defined in ISO / IEC 23090-7 Immersive media metadata and is referenced by file format standards such as Carriage of V3C and Carriage of Geometry-based Point Cloud Compression Data. In other words, viewport information can also be considered generalized information and is not dependent on any particular standard.

[0093] Next, the playback process of 3D data extracted from 3D media files generated by the methods described in the embodiments described so far will be explained according to the flowchart in Figure 7. Figure 7 is a flowchart of an example of a process that plays back 3D data from a 3D media file containing 3D data using object priority.

[0094] The following describes a case where the information processing device 100 performs processing according to the flowchart in Figure 7, but it is not limited to this case; a device other than the information processing device 100 may acquire the 3D media file and perform the playback processing described above.

[0095] In step S701, the acquisition unit 108 loads (acquires) the 3D media file stored in the non-volatile memory 113 into the RAM 103.

[0096] In step S702, the analysis unit 109 analyzes the track configuration and other information contained in the 3D media file acquired in RAM 103.

[0097] In step S703, the analysis unit 109 determines, based on the results of the analysis in step S702, whether or not the 3D media file contains a track that stores viewport information (viewport track).

[0098] As a result of this determination, if the 3D media file contains a viewport track, the process proceeds to step S704; otherwise, the process proceeds to step S707.

[0099] In step S704, the analysis unit 109 determines whether the 3D media file contains multiple viewport tracks. If the 3D media file contains multiple viewport tracks, the process proceeds to step S705; otherwise, the process proceeds to step S706.

[0100] In step S705, the analysis unit 109 selects the viewport track with the highest recommendation level from among multiple viewport tracks. The recommendation level can be determined using the methods defined in some of the examples mentioned above, or, if there is no specific definition for recommendation level, the viewport track with the smallest track ID value may be selected.

[0101] In step S706, the analysis unit 109 analyzes the viewport track selected in step S705 or a single viewport track included in the 3D media file.

[0102] In step S707, the analysis unit 109 analyzes the priority level of the object. Note that, as explained using Figure 6, if the priority level is defined for each bounding box surrounding an object rather than for the object itself, the analysis unit 109 analyzes the priority level of the bounding box. The following explanation will focus on objects, but the subsequent processing will be the same even if the bounding box is targeted.

[0103] In step S708, CPU 102 obtains the priority level of the object corresponding to the 3D data of the first frame.

[0104] In step S709, the CPU 102 identifies which objects to display and which not to display based on the priority level obtained in step S708.

[0105] In step S710, the CPU 102 extracts encoded data from the 3D media file that includes the object identified in step S709 as the object to be displayed (the object to be displayed). This process extracts the data necessary for displaying the object from the 3D media file for each frame.

[0106] In step S711, the CPU 102 decodes the encoded data extracted from the 3D media file in step S710 to obtain 3D data, and generates (renders) an image of the object to be displayed based on the 3D data.

[0107] In step S712, the CPU 102 determines whether processing has been completed for all frames included in the 3D media file. If processing has been completed for all frames, the process according to the flowchart in Figure 7 is terminated. On the other hand, if there are still frames that have not been processed, the process proceeds to step S713.

[0108] In step S713, the CPU 102 determines whether the object's priority level changes dynamically based on the results of the analysis in step S702. If the object's priority level changes dynamically as a result of this determination, the process proceeds to step S714; otherwise, it proceeds to step S709. In step S714, the CPU 102 obtains the object's priority level for the next frame.

[0109] Up to this point, the explanation has primarily described a system where object priority or recommendation information and viewport information are stored within the file. However, viewing information does not necessarily need to be stored within the file. In other words, the priority or recommendation information and viewport information explained using Figures 4 and 6 can be stored in a different location from the file containing the encoded data.

[0110] For example, metadata information associated with encoded data could be stored as a file and associated with it using a software-based system, or it could be stored in a memory device such as RAM without being stored as a file.

[0111] This configuration is particularly suitable for large-scale systems, as it allows for efficient centralized management by storing priority level information associated with viewports in a single file rather than in individual files.

[0112] Incidentally, ISO / IEC 12113:2022 glTF2.0 is known as a format for transferring 3D content and describing 3D scene information. Furthermore, MPEG has standardized ISO / IEC 23090-14 Scene Description, a technology that extends glTF2.0 to allow data encoded with 3D encoding standards and audio encoding standards standardized by MPEG to be defined as 3D scene information. Therefore, object position information and viewport information may be defined as 3D scene information by applying glTF2.0 or the Scene Description standard. By defining it using such standards, it can be expected to be used in systems that support glTF2.0.

[0113] In the explanations so far, we have used terms that are semantically similar, namely priority and recommendation level. This is because, as mentioned above, the former is intended to represent what is generally prioritized, while the latter emphasizes the intentions of the content creator. These are scales set with different intentions. In this embodiment, however, it is not important which of the two scales was defined with which intention, so the terms priority and recommendation level can be considered synonymous.

[0114] Furthermore, in the explanations above, we described a method of defining priority levels for each object or for each bounding box surrounding an object. However, in cases where multiple objects are in contact or intricately intertwined, it may not be possible to clearly distinguish and define the bounding boxes surrounding each object. For example, there may be cases where multiple objects are included in a single bounding box. In other words, there may be cases where the objects to be reproduced cannot be separated at the 3D data level. Therefore, in such cases, it is desirable to assign an identifier to the "bounding box surrounding one or more objects" and set a priority level accordingly.

[0115] Furthermore, by utilizing priority levels for each object and each bounding box surrounding an object, for example, when playing computationally intensive content using 3D data on a display terminal with low processing power, it becomes possible to display items in order from highest priority to lowest priority within the limits of the display terminal's processing power. This reduces the rendering load of 3D data and enables smooth playback.

[0116] Furthermore, when viewing as a free-viewpoint video, it becomes possible to exclude and display objects with a low priority level, or in some cases, objects that may interfere with viewing, when viewed from a specific field of view. Moreover, instead of simply hiding low-priority objects, it's possible to display only the wireframe of a 3D object, or, in the case of scalable 3D data such as LoD (Level of Detail), control the level of detail displayed according to the priority level, thereby optimizing the rendering process and reducing processing load. It's also conceivable that the priority level of objects to be rendered could be controlled based on the user's permissions when viewing the 3D data.

[0117] Furthermore, various definitions can be applied to the recommendation level and recommendation / non-recommendation used in this embodiment. For example, the recommendation level may be a parameter that takes an integer value from 0 (lowest recommendation level) to 100 (highest recommendation level). Alternatively, a recommendation level above a threshold may represent "recommendation," and a recommendation level below a threshold may represent "non-recommendation." Alternatively, the recommendation level may be represented by two values, "0" and "1," where a value of "0" represents non-recommendation and a value of "1" represents recommendation.

[0118] When rendering high-quality 3D data containing multiple people as free-viewpoint video for devices such as head-mounted displays, smartphones, or tablet PCs, demanding requirements are placed on hardware resources such as CPU and memory. Therefore, services known as cloud rendering, which perform rendering on cloud-based computing devices, have emerged. However, cloud rendering involves sending data such as viewpoint, viewing direction, and field of view from each viewing device, and then transmitting the resulting 2D video to each device. Depending on the communication environment and use case, transmission delays can become a problem. Furthermore, if a large number of viewing devices utilize cloud rendering, the increased load on the cloud-based computing device can become a problem. This embodiment of information processing equipment enables optimal rendering and reduces the rendering load.

[0119] [Second Embodiment] In the first embodiment, a case was described in which the information processing device 100 performs the process of generating 3D media files and the process of playing back objects based on 3D media files. However, these processes may be performed using multiple computer devices. Alternatively, some of these processes may be performed by an external device (e.g., a cloud server), and the information processing device 100 may perform subsequent processes based on the results of that execution. Thus, the main body of the overall processing and the configuration of the system for it are not limited to a specific form.

[0120] Alternatively, computer programs stored in the information processing device 100 may be downloaded to the non-volatile memory 113 by connecting to an internet homepage from the browser of the information processing device 100, which acts as a client computer. These computer programs may be uncompressed computer program files, or compressed computer program files that include an automatic installation function.

[0121] Alternatively, the program code constituting such a computer program may be divided into multiple files, and each file may be downloaded to the information processing device 100 from a different homepage. In other words, a WWW server that allows multiple users to download computer program files to implement the above processing on the information processing device 100 can also be considered as one embodiment.

[0122] Furthermore, it is possible to encrypt such computer programs, store them on a storage medium such as a CD-ROM, distribute them to users, and allow users who meet certain conditions to download decryption key information from a website via the internet. In other words, the user can use that key information to execute the encrypted computer program on the information processing device 100 and install it on the information processing device 100.

[0123] The numerical values, processing timing, processing order, processing entity, data (information) structure / acquisition method / destination / source / storage location, etc., used in the above embodiment are given as examples for the purpose of providing a concrete explanation, and are not intended to limit the scope to such examples.

[0124] Furthermore, some or all of the embodiments described above may be used in appropriate combinations. Alternatively, some or all of the embodiments described above may be used selectively.

[0125] (Other embodiments) This disclosure can also be implemented by supplying a program that implements one or more of the functions of the embodiments described above to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0126] The disclosures herein include the following information processing devices, information processing methods, and computer programs. (Item 1) The system includes storage control means for storing 3D data and metadata related to the 3D data in a file in a specified format. The storage control means stores in the file the priority level for displaying each object, or at least a portion of each object, included in the 3D data. An information processing device characterized by the following: (Item 2) The information processing device according to item 1, characterized in that the storage control means stores viewport information, including the viewpoint, line of sight direction, and field of view in space based on the three-dimensional data, in the file. (Item 3) The information processing device according to item 2, characterized in that the storage control means stores a plurality of viewport pieces of information in the file. (Item 4) The information processing device according to any one of items 1 to 3, characterized in that the storage control means targets each object included in the 3D data, or at least a part of each object, and stores in the file information that can identify the object that is recommended to be displayed among the objects. (Item 5) The information processing device according to any one of items 1 to 4, characterized in that the storage control means targets each object included in the 3D data, or at least a part of each object, and stores in the file information that can identify objects from which display is not recommended. (Item 6) The information processing device according to any one of items 1 to 5, characterized in that the priority level is either priority or a recommendation level set according to the importance of the content. (Item 7) An acquisition means for acquiring the display priority level stored for each object, or at least a portion of each object, contained in the 3D data, from a file in which 3D data and metadata related to said 3D data are stored in a specified format. Control means for controlling the playback of the 3D data based on the priority level, An information processing device characterized by comprising: (Item 8) The acquisition means acquires viewport information, including the viewpoint, line of sight direction, and field of view in space based on the 3D data, from the file. The control means controls the playback based on the viewport information and the priority level associated with the viewport information. The information processing device described in item 7, characterized by the features described herein. (Item 9) The information processing device according to item 8, characterized in that the acquisition means acquires the viewport information with the highest recommendation level among a plurality of viewport information from the file. (Item 10) The information processing device according to any one of items 7 to 9, characterized in that the file contains information that can identify which of the three-dimensional data, or at least a portion of each of the three-dimensional data, is recommended for display. (Item 11) The information processing device according to any one of items 7 to 10, characterized in that the file contains information that can identify objects included in the three-dimensional data, or at least a portion of such objects, that are not recommended to be displayed. (Item 12) The information processing device according to any one of items 7 to 11, characterized in that the priority level is either priority or a recommendation level set according to the importance of the content. (Item 13) The information processing device according to item 3, characterized in that the storage control means stores each of the multiple viewport pieces of information in the file as a different track. (Item 14) The information processing apparatus according to item 4 or 5, characterized in that the storage control means stores each object included in the 3D data as a different track in the file. (Item 15) The information processing device according to item 14, characterized in that the storage control means associates the priority level for display with the identification information of the track storing the three-dimensional data. (Item 16) The information processing device further comprises encoding means for encoding the object using a predetermined encoding method, The encoding means divides the object into multiple regions and encodes each of the divided objects so that it can be decoded independently. The information processing apparatus according to item 1, characterized in that the storage control means sets the priority level for at least a portion of the encoded plurality of objects. (Item 17) The information processing device further comprises encoding means for encoding the object using a predetermined encoding method, The encoding means divides the object into multiple regions and encodes each of the divided objects so that it can be decoded independently. The information processing apparatus according to item 2, characterized in that the storage control means sets a priority level to recommend acquisition for encoded objects located in the area necessary for display from the viewport information, and sets a priority level to not recommend acquisition for encoded objects located in the occlusion area of ​​the viewport information, or does not set a priority level for any of the encoded objects. (Item 18) An information processing method performed by an information processing device, The storage control means of the information processing device includes a storage control step of storing 3D data and metadata related to the 3D data in a file in a specified format. In the storage control step, the priority level for displaying each object, or at least a portion of each object, included in the 3D data is stored in the file. An information processing method characterized by the following: (Item 19) An information processing method performed by an information processing device, The acquisition means of the information processing device includes an acquisition step of acquiring 3D data, metadata related to the 3D data, and the priority level for display stored for each object, or at least a part of each object, contained in the 3D data. The control means for the information processing device includes a control step that controls the playback of the 3D data based on the priority level. An information processing method characterized by comprising: (Item 20) The information processing method according to item 19, characterized in that the priority level is obtained from a file stored in a specified format. (Item 21) In the acquisition process described above, from among multiple tracks, each containing multiple viewport information stored as a different track, the track containing the predetermined viewport information is acquired. In the control step, the objects to be displayed are selected according to the priority level associated with the viewport information. The information processing method described in item 19, characterized by the features described herein. (Item 22) An information processing method performed by an information processing device, The acquisition means of the information processing device includes an acquisition step of acquiring 3D data, metadata related to the 3D data, and the priority level for display stored for each object, or at least a part of each object, contained in the 3D data. The control means for the information processing device includes a control step that controls the playback of the 3D data based on the priority level. Equipped with, The object to be displayed can be obtained as a divided object, which is divided into multiple regions. The aforementioned priority level is set for each divided object, to be applied when reproducing the object to be displayed from a predetermined viewport. In the control step, the segmented objects located in the area necessary to display the object to be displayed from a predetermined viewport are identified from the priority level of each segmented object obtained, In the acquisition step, the identified segmented object is acquired. An information processing method characterized by the following: (Item 23) The information processing method according to item 22, characterized in that the priority level includes information intended to be recommended for display and information intended not to be recommended for display. (Item 24) An information processing device that generates metadata related to 3D data, Equipped with means for acquiring 3D data, The 3D data acquired by the acquisition means includes multiple objects, The system includes setting means for setting the display priority level for each object, or at least a portion of each object, included in the aforementioned 3D data. An information processing device characterized by generating a set priority level as metadata. (Item 25) A computer program that causes a computer to function as one of the means of an information processing device described in any one of items 1 through 17 or 24.

[0127] The disclosure is not limited to the embodiments described above, and various modifications and alterations are possible without departing from the spirit and scope of the disclosure. Accordingly, claims are attached to make the scope of the disclosure public. [Explanation of symbols]

[0128] 100: Information processing unit 101: System bus 102: CPU 103: RAM 104: ROM 105: Operation input unit 106: Display unit 107: Communication unit 108: Acquisition unit 109: Analysis unit 110: Setting unit 111: Processing unit 112: Encoding unit 113: Non-volatile memory

Claims

1. The system includes storage control means for storing 3D data and metadata related to the 3D data in a file in a specified format. The storage control means stores in the file the priority level for displaying each object, or at least a portion of each object, included in the three-dimensional data. An information processing device characterized by the following:

2. The information processing device according to claim 1, characterized in that the storage control means stores viewport information, including the viewpoint, line of sight direction, and field of view in space based on the three-dimensional data, in the file.

3. The information processing apparatus according to claim 2, characterized in that the storage control means stores a plurality of viewport pieces of information in the file.

4. The information processing apparatus according to claim 1, characterized in that the storage control means targets each object included in the three-dimensional data, or at least a part of each object, and stores in the file information that can identify which of the objects is recommended for display.

5. The information processing apparatus according to claim 1, wherein the storage control means targets each object included in the three-dimensional data, or at least a portion of each object, and stores in the file information that can identify objects from which display is not recommended.

6. The information processing device according to claim 1, characterized in that the priority level is either priority or a recommendation level set according to the importance of the content.

7. An acquisition means for acquiring the display priority level stored for each object, or at least a portion of each object, contained in the 3D data, from a file in which 3D data and metadata related to said 3D data are stored in a specified format. Control means for controlling the playback of the 3D data based on the priority level, An information processing device characterized by comprising:

8. The acquisition means acquires viewport information, including the viewpoint, line of sight direction, and field of view in space based on the three-dimensional data, from the file. The control means controls the playback based on the viewport information and the priority level associated with the viewport information. The information processing apparatus according to feature 7.

9. The information processing device according to claim 8, characterized in that the acquisition means acquires the viewport information with the highest recommendation level among the plurality of viewport information from the file.

10. The information processing device according to claim 7, characterized in that the file contains information that can identify which of the three-dimensional data, or at least a portion of each of the three-dimensional data, is recommended for display.

11. The information processing device according to claim 7, characterized in that the file contains information that can identify each object included in the three-dimensional data, or at least a part of each of said objects, that is not recommended to be displayed.

12. The information processing device according to claim 7, characterized in that the priority level is either priority or a recommendation level set according to the importance of the content.

13. The information processing apparatus according to claim 3, characterized in that the storage control means stores each of the plurality of viewport pieces of information in the file as a different track.

14. The information processing apparatus according to claim 4, characterized in that the storage control means stores each object included in the three-dimensional data as a different track in the file.

15. The information processing apparatus according to claim 14, characterized in that the storage control means associates the priority level for display with the identification information of the track storing the three-dimensional data.

16. The information processing device further comprises encoding means for encoding the object using a predetermined encoding method, The encoding means divides the object into multiple regions and encodes each of the divided objects so that it can be decoded independently. The information processing apparatus according to claim 1, wherein the storage control means sets the priority level for at least a portion of the encoded plurality of objects.

17. The information processing device further comprises encoding means for encoding the object using a predetermined encoding method, The encoding means divides the object into multiple regions and encodes each of the divided objects so that it can be decoded independently. The information processing apparatus according to claim 2, wherein the storage control means sets a priority level to recommend acquisition for encoded objects located in the area necessary for display from the viewport information, and sets a priority level to not recommend acquisition for encoded objects located in the occlusion area of ​​the viewport information, or does not set a priority level for such objects.

18. An information processing method performed by an information processing device, The storage control means of the information processing device includes a storage control step of storing three-dimensional data and metadata related to the three-dimensional data in a file in a specified format. In the storage control step, the priority level for displaying each object, or at least a portion of each object, included in the three-dimensional data is stored in the file. An information processing method characterized by the following:

19. An information processing method performed by an information processing device, The acquisition means of the information processing device includes an acquisition step of acquiring three-dimensional data, metadata related to the three-dimensional data, and the priority level for display stored for each object, or at least a part of each object, contained in the three-dimensional data. The control means of the information processing device includes a control step that controls the playback of the three-dimensional data based on the priority level. An information processing method characterized by comprising:

20. The information processing method according to claim 19, characterized in that the priority level is obtained from a file stored in a specified format.

21. In the acquisition process described above, from among multiple tracks, each containing multiple viewport information stored as a different track, the track containing the predetermined viewport information is acquired. In the control step, the objects to be displayed are selected according to the priority level associated with the viewport information. The information processing method according to feature 19.

22. An information processing method performed by an information processing device, The acquisition means of the information processing device includes an acquisition step of acquiring three-dimensional data, metadata related to the three-dimensional data, and the priority level for display stored for each object, or at least a part of each object, contained in the three-dimensional data. The control means of the information processing device includes a control step that controls the playback of the three-dimensional data based on the priority level. Equipped with, The object to be displayed can be obtained as a divided object, which is divided into multiple regions. The aforementioned priority level is set for each divided object, to be applied when reproducing the object to be displayed from a predetermined viewport. In the control step, the segmented objects located in the area necessary to display the object to be displayed from a predetermined viewport are identified from the priority level of each segmented object obtained, In the acquisition step, the identified segmented object is acquired. An information processing method characterized by the following:

23. The information processing method according to claim 22, characterized in that the priority level includes information intended to be recommended for display and information intended not to be recommended for display.

24. An information processing device that generates metadata related to 3D data, Equipped with means for acquiring 3D data, The three-dimensional data acquired by the acquisition means includes multiple objects, The system includes setting means for setting the display priority level for each object, or at least a portion of each object, included in the aforementioned three-dimensional data. An information processing device characterized by generating a set priority level as metadata.

25. A computer program for causing a computer to function as one of the means of an information processing apparatus described in any one of claims 1 to 17 or 24.

Images