Point cloud encapsulation and decapsulation method, device, medium and electronic equipment

Abstract

The application belongs to the technical field of audio and video, and particularly relates to a point cloud encapsulation and decapsulation method, device, medium and electronic equipment. The point cloud encapsulation method in the application embodiment comprises the following steps: obtaining information data boxes of content entities in a point cloud code stream, wherein the information data boxes comprise an entity identifier field; the entity identifier field is used for indicating that the content entity is one entity in a replaceable group, or is used for indicating that the content entity is contained in at least two replaceable point cloud code streams; the replaceable group is an entity set composed of multiple content entities having a replaceable relationship; and the content entity is encapsulated into a point cloud file according to the information data boxes. The application embodiment can effectively identify replaceable point cloud code streams in a point cloud file.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of audio and video, and particularly relates to a point cloud encapsulation method, a point cloud decapsulation method, a point cloud encapsulation device, a point cloud decapsulation device, a computer readable medium, an electronic device and a computer program product. BACKGROUND

[0002] A point cloud is a set of discrete points irregularly distributed in space, which expresses the spatial structure and surface attributes of a three-dimensional object or scene. Point cloud media can be divided into 3 degrees of freedom (DoF) media, 3DoF+ media and 6DoF media according to the degree of freedom of users when consuming media content.

[0003] For point cloud media of the same content, replaceable point cloud bitstreams with different qualities can be formed through encoding for users to select and consume. When point cloud bitstreams with different qualities are encapsulated in the same point cloud file, the replacement relationship between the point cloud bitstreams is difficult to identify. SUMMARY

[0004] The present application provides a point cloud encapsulation method, a point cloud decapsulation method, a point cloud encapsulation device, a point cloud decapsulation device, a computer readable medium, an electronic device and a computer program product, which aims to effectively identify replaceable point cloud bitstreams in a point cloud file.

[0005] Other characteristics and advantages of the present application will become apparent from the following detailed description, or will be learned by practice of the present application.

[0006] According to an aspect of an embodiment of the present application, a point cloud encapsulation method is provided, which comprises:

[0007] Obtaining an information data box of a content entity in a point cloud bitstream, the information data box comprising an entity identifier field; the entity identifier field is used to indicate that the content entity is one entity in a replaceable group, or is used to indicate that the content entity is contained in at least two replaceable point cloud bitstreams; the replaceable group is an entity set composed of a plurality of content entities having a replaceable relationship;

[0008] Encapsulating the content entity into a point cloud file according to the information data box.

[0009] According to an aspect of an embodiment of the present application, a point cloud decapsulation method is provided, which comprises:

[0010] An information data box of a content entity in a point cloud file is acquired, the information data box comprising an entity identification field; the entity identification field is used to indicate that the content entity is one entity in a replaceable group, or is used to indicate that the content entity is contained in at least two replaceable point cloud bitstreams; the replaceable group is an entity set composed of a plurality of content entities having a replaceable relationship;

[0011] The point cloud file is unpackaged according to the information data box, to obtain a point cloud bitstream composed of the content entities.

[0012] According to an aspect of an embodiment of the present application, a point cloud packaging device is provided, the device comprising:

[0013] A first acquisition module is configured to acquire an information data box of a content entity in a point cloud bitstream, the information data box comprising an entity identification field; the entity identification field is used to indicate that the content entity is one entity in a replaceable group, or is used to indicate that the content entity is contained in at least two replaceable point cloud bitstreams; the replaceable group is an entity set composed of a plurality of content entities having a replaceable relationship;

[0014] A packaging module is configured to package the content entity into a point cloud file according to the information data box.

[0015] According to an aspect of an embodiment of the present application, a point cloud unpackaging device is provided, the device comprising:

[0016] A second acquisition module is configured to acquire an information data box of a content entity in a point cloud file, the information data box comprising an entity identification field; the entity identification field is used to indicate that the content entity is one entity in a replaceable group, or is used to indicate that the content entity is contained in at least two replaceable point cloud bitstreams; the replaceable group is an entity set composed of a plurality of content entities having a replaceable relationship;

[0017] An unpackaging module is configured to unpack the point cloud file according to the information data box, to obtain a point cloud bitstream composed of the content entities.

[0018] According to an aspect of an embodiment of the present application, a computer readable medium is provided, the computer readable medium having stored thereon a computer program, the computer program being executed by a processor to implement the method in the above technical solutions.

[0019] According to an aspect of an embodiment of the present application, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the method in the above technical solutions via execution of the executable instructions.

[0020] According to an aspect of an embodiment of the present application, a computer program product is provided, comprising a computer program, wherein the computer program is executed by a processor to implement the method in the above technical solutions.

[0021] In the technical solution provided in the embodiments of the present application, by configuring the information data box of the content entity in the point cloud code stream and providing the entity identification field in the information data box, it can be indicated that the content entity is one entity in the replaceable group, or it can be indicated that the content entity is contained in at least two replaceable point cloud code streams, so that the identification degree can be improved at the point cloud code stream level, and the point cloud code streams with the replacement relationship can be effectively identified.

[0022] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0024] Figure 1 A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of the present application can be applied is shown.

[0025] Figure 2 A point cloud media coding flow diagram in an application scenario of an embodiment of the present application is shown.

[0026] Figure 3 A replaceable group encapsulated by multiple tracks in the related technology of the present application is shown.

[0027] Figure 4 A file encapsulation method that can be used for two replaceable point cloud code streams in an embodiment of the present application is shown.

[0028] Figure 5 A file encapsulation method that can be used for three replaceable point cloud code streams in an embodiment of the present application is shown.

[0029] Figure 6 A point cloud encapsulation method based on the indication of the replaceable relationship of the point cloud code stream in an embodiment of the present application is shown.

[0030] Figure 7 A data structure of the newly added replaceable information data box in an embodiment of the present application is shown.

[0031] Figure 8 A point cloud file encapsulation structure based on single track encapsulation in an embodiment of the present application is shown.

[0032] Figure 9 A point cloud file encapsulation structure based on component-based multi-track encapsulation in an embodiment of the present application is shown.

[0033] Figure 10 A point cloud file encapsulation structure based on slice-based multi-track encapsulation (geometric data and attribute data are encapsulated together) in an embodiment of the present application is shown.

[0034] Figure 11 A point cloud file encapsulation structure based on slice-based multi-track encapsulation (geometric data and attribute data are encapsulated separately) in an embodiment of the present application is shown.

[0035] Figure 12 A point cloud replaceable information structure based on the AVS standard in an embodiment of the present application is shown.

[0036] Figure 13 A data box structure of a point cloud replaceable track group based on the AVS standard in an embodiment of the present application is shown.

[0037] Figure 14 A data box structure of a non-timed point cloud media replaceable group based on the AVS standard in an embodiment of the present application is shown.

[0038] Figure 15 A point cloud decapsulation method based on indicating replaceable relationship of point cloud code stream in an embodiment of the present application is shown.

[0039] Figure 16 A structure block diagram of a point cloud encapsulation apparatus provided by an embodiment of the present application is shown schematically.

[0040] Figure 17 A structure block diagram of a point cloud decapsulation apparatus provided by an embodiment of the present application is shown schematically.

[0041] Figure 18 A computer system structure block diagram of an electronic device suitable for implementing an embodiment of the present application is shown schematically. DETAILED DESCRIPTION

[0042] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of example implementations to those skilled in the art.

[0043] Moreover, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the application can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In other instances, well-known structures, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the application.

[0044] The block diagrams in the drawings show only the functionality of the features and can not imply that the functionality must be implemented in the particular correlated order indicated. For example, a step or functionality can be performed in reverse or concurrently. The functionality can also be implemented on different subsystems or devices.

[0045] The flow diagrams shown in the various figures, which can also be referred to as flow charts, are examples of sequences of operations that can be performed by, e.g., computer programs, software, firmware, hardware, or a combination thereof. In the context of software, the operations can take the form of program modules, individual program segments, program instructions, computer-readable code, or the like, and can be stored in one or more computer-readable media or memories at one or more locations. The various operations can be arranged in various orders or omitted, and additional operations can be performed, depending on the implementation. For example, a step or functionality can be performed in reverse or concurrently. The functionality can also be implemented on different subsystems or devices.

[0046] Embodiments of the present application relate to the coding technology of point cloud media, and the related technical terms are explained as follows.

[0047] Point cloud: Point cloud is a set of discrete points that are irregularly distributed in space, expressing the spatial structure and surface attributes of a three-dimensional object or scene. Each point in the point cloud has at least three-dimensional position information, and may, depending on the application scenario, have color, material, or other information. Generally, each point in the point cloud has the same number of additional attributes.

[0048] Slice: Point cloud slice / point cloud strip, representing a set of syntax elements (such as geometry slice, attribute slice) of part or all of the encoded point cloud data.

[0049] Track: Track, track is a set of media data in the media file packaging process, composed of multiple samples with timing. A media file can be composed of one or more tracks, for example, commonly: a media file can contain a video media track, an audio media track, and a subtitle media track. In particular, metadata information can also be included in the file in the form of a metadata media track as a media type.

[0050] Sample: Sample is the encapsulation unit in the media file encapsulation process, a track is composed of many samples, each sample corresponds to specific timestamp information, for example: a video media track can be composed of many samples, a sample is usually a video frame. In the embodiments of the present application, a sample in the point cloud media track can be a point cloud frame.

[0051] Sample Entry: Sample entry is used to indicate the metadata information related to all samples in the track. For example, in the sample entry of a video track, the decoder initialization related metadata information is usually included.

[0052] Tile: Hexahedral spatial sub-block region within the point cloud frame boundary spatial region. A point cloud spatial sub-block is composed of one or more point cloud slices, and there is no coding dependency between point cloud spatial sub-blocks.

[0053] DASH: Dynamic adaptive streaming over HTTP, dynamic adaptive streaming over HTTP, is an adaptive bit rate streaming technology that enables high-quality streaming media to be delivered over the Internet through traditional HTTP web servers.

[0054] MPD: Media presentation description, media presentation description signaling in DASH, used to describe media segment information.

[0055] Representation: In DASH, a combination of one or more media components, such as a video file of a certain resolution can be regarded as a Representation.

[0056] Adaptation Sets: In DASH, a set of one or more video streams, an Adaptation Sets can contain multiple Representations.

[0057] Media Segment: Media segment. A segment that conforms to a certain media format and can be played. When playing, it may need to cooperate with 0 or more previous segments and initialization segments.

[0058] DoF: Degree of freedom, degree of freedom, refers to the freedom of motion supported by the user when watching immersive media and producing content interaction.

[0059] 3DoF: Three degrees of freedom, that is, three degrees of freedom of the user's head rotating around the x, y, and z axes.

[0060] 3DoF+: i.e. on the basis of three degrees of freedom, the user also has the freedom to move along the x, y, z axis.

[0061] 6DoF: i.e. on the basis of three degrees of freedom, the user also has the freedom to move along the x, y, z axis.

[0062] AVS: Audio Video Coding Standard, Chinese national video coding standard.

[0063] MPEG: Moving Picture Experts Group, dynamic image expert group, is an organization established by ISO (International Standardization Organization) and IEC (International Electrotechnical Commission) to formulate international standards for moving image and voice compression.

[0064] ISOBMFF: ISO Based Media File Format, ISO standard based media file format. ISOBMFF is the encapsulation standard of media file, and the most typical ISOBMFF file is MP4 file.

[0065] Point cloud media can be divided into point cloud media compressed based on traditional video coding method (Video-based Point Cloud Compression, VPCC) and point cloud media compressed based on geometric features (Geometry-based Point Cloud Compression, GPCC) according to the coding method. In the file encapsulation of point cloud media, the three-dimensional position information is usually called the geometry component of the point cloud file, and the attribute information is called the attribute component of the point cloud file. A point cloud file has only one geometry component, but can have one or more attribute components.

[0066] Point cloud can flexibly and conveniently express the spatial structure and surface attribute of three-dimensional object or scene, and thus is widely applied, including virtual reality (VR) game, computer aided design (CAD), geography information system (GIS), autonomous navigation system (ANS), digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive telepresence, three-dimensional reconstruction of biological tissue and organ, etc.

[0067] The point cloud can be obtained in the following ways: computer generation, 3D laser scanning, 3D photogrammetry, etc. The computer can generate the point cloud of virtual three-dimensional object and scene. The 3D scanning can obtain the point cloud of static real-world three-dimensional object or scene, and can obtain million-level point cloud per second. The 3D camera can obtain the point cloud of dynamic real-world three-dimensional object or scene, and can obtain ten-million-level point cloud per second. In addition, in the medical field, the point cloud of biological tissue and organ can be obtained by MRI, CT and electromagnetic positioning information. These technologies reduce the cost and time period of point cloud data acquisition, and improve the accuracy of data. The change of point cloud data acquisition mode makes it possible to obtain a large amount of point cloud data. With the continuous accumulation of large-scale point cloud data, efficient storage, transmission, release, sharing and standardization of point cloud data have become the key to point cloud application.

[0068] After the point cloud media is encoded, the encoded data stream needs to be encapsulated and transmitted to the user. Correspondingly, at the point cloud media player end, the point cloud file needs to be decapsulated first, and then decoded, and finally the decoded data stream is presented.

[0069] Figure 1 A schematic diagram of an exemplary system architecture to which the techniques of this application can be applied is shown.

[0070] As Figure 1 shown, system architecture 100 includes a number of terminal devices that can communicate with one another by, for example, network 150. For example, system architecture 100 can include a first terminal device 110 and a second terminal device 120 interconnected via network 150. In Figure 1 an embodiment, first terminal device 110 and second terminal device 120 perform unidirectional transmission of data.

[0071] For example, the first terminal device 110 can encode point cloud data (e.g., point cloud data captured by the terminal device 110) for transmission to the second terminal device 120 over the network 150, the encoded point cloud data being transmitted in the form of one or more encoded point cloud bitstreams, the second terminal device 120 can receive the encoded point cloud data from the network 150, decode the encoded point cloud data to recover the point cloud data, and display point cloud content in accordance with the recovered point cloud data.

[0072] In an embodiment of the present application, the system architecture 100 can include a third terminal device 130 and a fourth terminal device 140 that perform bidirectional transmission of encoded point cloud data, such as can occur during a video conference. For bidirectional data transmission, each of the third terminal device 130 and the fourth terminal device 140 can encode point cloud data (e.g., point cloud data captured by the terminal device) for transmission to the other one of the third terminal device 130 and the fourth terminal device 140 over the network 150. Each of the third terminal device 130 and the fourth terminal device 140 can also receive encoded point cloud data transmitted by the other one of the third terminal device 130 and the fourth terminal device 140, and can decode the encoded point cloud data to recover the point cloud data, and can display point cloud content on an accessible display device in accordance with the recovered point cloud data.

[0073] In Figure 1 In an embodiment of the present application, the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140 can be servers, personal computers, and smart phones, although the principles disclosed herein can not be limited thereto. Embodiments disclosed herein are applicable to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network 150 represents any number of networks that convey encoded point cloud data between the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140, including, for example, wired and / or wireless communication networks. The communication network 150 can exchange data in circuit- switched and / or packet-switched channels. The network can include telecommunication networks, local area networks, wide area networks, and / or the Internet. For purposes of this application, the architecture and topology of the network 150 can be immaterial to the operation of the embodiments disclosed herein, unless otherwise explained herein below.

[0074] The server in the embodiments of the present application can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a vehicle-mounted terminal, a smart television, etc., but is not limited thereto. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited in the present application.

[0075] After the point cloud media is encoded, the encoded data stream needs to be encapsulated and transmitted to the user. Correspondingly, at the point cloud media player end, the point cloud file needs to be decapsulated first, then decoded, and finally the decoded data stream is presented.

[0076] Figure 2 A point cloud media encoding and decoding process diagram in an application scenario of the embodiments of the present application is shown.

[0077] The point cloud data acquisition by the acquisition device 210 can capture the visual scene A of the real world. The acquisition device 210 can be a set of cameras or a camera device with multiple lenses and sensors, for example. The acquisition result is the point cloud source data B, which is a frame sequence composed of a large number of point cloud frames. One or more point cloud frames can be encoded by the encoder 220 to obtain the encoded G-PCC bitstream, which can specifically include the encoded geometry bitstream and attribute bitstream E. The file encapsulator 230 can encapsulate one or more encoded bitstreams according to a specific media container file format to obtain the media file F for file playback or a series of initialization segments and media segments Fs for streaming. In some embodiments of the present application, the media container file format can be the ISO base media file format specified in ISO / IEC 14496-12 [ISOBMFF], for example. The file encapsulator 230 can also encapsulate metadata in the media file F or the media segment Fs.

[0078] The media file F output by the file packager 230 is the same as the media file F' input by the file unpackager 240. The file unpackager can extract the encoded bitstream E' and parse the metadata by processing the media file F' or processing the received media segments F's. The decoder 250 can decode the G-PCC bitstream into a decoded signal D' and generate point cloud data from the decoded signal D'. If applicable, the point cloud data can be rendered and displayed onto a screen of a head-mounted display or any other display device by the Tenderer 260 based on the current viewing position, viewing direction or viewport determined by various types of sensors (e.g. head). The current viewing position or viewing direction can also be used for decoding optimization in addition to being used by the player to access the appropriate portion of the decoded point cloud data. The current viewing position and viewing direction are also passed to a policy module in the viewport-dependent content distributor 270, which can be used to determine the tracks to be received.

[0079] In the transmission technology of point cloud media, the streaming transmission technology is usually used to process the transmission of media resources between the server and the client. Common media streaming transmission technologies include DASH (Dynamic Adaptive Streaming over HTTP), HLS (HTTP Live Streaming), SMT (Smart Media Transport) and the like.

[0080] Taking DASH as an example, DASH is an adaptive bit rate streaming technology that enables high-quality streaming media to be delivered over the Internet via a traditional HTTP web server. DASH will break down the content into a series of small HTTP-based file segments, each containing a short length of playable content, while the total length of the content can be as long as several hours (such as a movie or a live sports event). The content will be made into alternative segments of multiple bit rates to provide multiple bit rate versions for selection. When the media content is played by the DASH client, the client will automatically select which alternative to download and play according to the current network conditions. The client will select the highest bit rate segment that can be downloaded in time for playback, thereby avoiding playback stalls or rebuffering events. As a result, the DASH client can seamlessly adapt to changing network conditions and provide a high-quality playback experience with fewer stalls and rebuffering occurrences. DASH uses existing HTTP web server infrastructure. It allows devices such as Internet TVs, TV set-top boxes, desktop computers, smartphones, tablets, etc. to consume multimedia content (such as video, TV, radio, etc.) delivered over the Internet and can cope with changing Internet reception conditions.

[0081] In the point cloud file, there are alternative groups corresponding to different point cloud tracks. Point cloud tracks with the same point cloud content and different point cloud qualities can be divided into the same alternative group. When GPCC point cloud data is encapsulated in a single track, point cloud tracks with different qualities belong to the same alternative group. When GPCC point cloud data is encapsulated in multiple tracks, geometry component tracks of point cloud content with different qualities belong to the same alternative group, and attribute component tracks can be associated with the geometry component tracks.

[0082] Figure 3 An alternative group encapsulated in multiple tracks in the related art is shown. As shown in Figure 3 , the alternative group 300 includes first point cloud data 310 and second point cloud data 320 with the same point cloud content. Among them, the first point cloud data 310 is lossless compressed point cloud data (Lossless coded GPCC) with relatively high point cloud quality, and the second point cloud data 320 is lossy compressed point cloud data (Lossy coded GPCC) with relatively low point cloud quality.

[0083] The first point cloud data 310 includes a first geometry component track 311 and a first attribute component track 312 associated with the first geometry component track 311, and the second point cloud data 320 includes a second geometry component track 321 and a second attribute component track 322 associated with the second geometry component track 321.

[0084] Figure 4 An alternative group encapsulated in multiple tracks in the related art is shown. As shown in Figure 4 , the alternative group 300 includes first point cloud data 310 and second point cloud data 320 with the same point cloud content. Among them, the first point cloud data 310 is lossless compressed point cloud data (Lossless coded GPCC) with relatively high point cloud quality, and the second point cloud data 320 is lossy compressed point cloud data (Lossy coded GPCC) with relatively low point cloud quality.

[0085] (1) They are respectively encapsulated as two different files, such as file 1 and file 2 shown in the figure. Among them, file 1 contains geometry component track track1 and attribute component track track2 corresponding to point cloud bitstream bitstream1, and file 2 contains geometry component track track1 and attribute component track track2 corresponding to point cloud bitstream bitstream2.

[0086] (2) They are encapsulated in the same file and are mutually replaceable, such as file 3 shown in the figure. At this time, geometry component track track1 and attribute component track track2 correspond to point cloud bitstream bitstream1, and geometry component track track3 and attribute component track track4 correspond to point cloud bitstream bitstream2.

[0087] (3) If the geometry information in the point cloud bitstream bitstream1 and the point cloud bitstream bitstream2 is completely identical, that is, the geometry information adopts the same encoding mode, at this time, one repeated geometry track can be omitted, and the point cloud bitstream bitstream1 and the point cloud bitstream bitstream2 are encapsulated into a file 4. At this time, only the attribute component track track2 and the attribute component track track3 are in a replaceable relationship with each other, the geometry component track track1 corresponds to the point cloud bitstream bitstream1 with the attribute component track track2, and the geometry component track track1 corresponds to the point cloud bitstream bitstream2 with the attribute component track track3.

[0088] Figure 5 An embodiment of the application is shown in the file encapsulation method that can be used for three replaceable point cloud bitstreams. As shown in Figure 5 Based on the third file encapsulation method shown in Figure 4 , the geometry component track track1 corresponds to the point cloud bitstream bitstream1 with the attribute component track track2, the geometry component track track1 corresponds to the point cloud bitstream bitstream2 with the attribute component track track3, and the geometry component track track4 corresponds to the point cloud bitstream bitstream3 with the attribute component track track5. Since there is no repeated geometry component track between the point cloud bitstream bitstream3 and the point cloud bitstream bitstream1 or the point cloud bitstream bitstream2, it is not enough to only indicate the replaceable relationship of track2 and track3, and the replaceable relationship between track1+track2, track1+track3 and track4+track5 also needs to be indicated additionally, otherwise, the replaceable relationship between the point cloud bitstreams cannot be clearly indicated.

[0089] In order to solve the problem that the replaceable relationship between the point cloud bitstreams cannot be clearly indicated in the related art, an embodiment of the application provides a point cloud replaceable content file encapsulation and signaling indication method, which can indicate the replaceable relationship at the point cloud bitstream level, and can indicate the shared component composition in the replaceable bitstream, including the shared geometry component and the possibly shared attribute component.

[0090] Figure 6 An embodiment of the application is shown in the point cloud encapsulation method based on indicating the replaceable relationship of the point cloud bitstreams, which can be applied to the server end, the player end or the intermediate node of the point cloud media system and the like.

[0091] As shown in Figure 6 , the point cloud encapsulation method based on indicating the replaceable relationship of the point cloud bitstreams can include the following steps S610 to S620.

[0092] S610: Obtain an information data box of a content entity in the point cloud bitstream, the information data box comprising an entity identifier field; the entity identifier field is used to indicate that the content entity is one entity in a replaceable group, or is used to indicate that the content entity is contained in at least two replaceable point cloud bitstreams; the replaceable group is an entity set composed of multiple content entities having a replaceable relationship;

[0093] S620: Encapsulate the content entity into a point cloud file according to the information data box.

[0094] The content entity in the embodiment of the application can be a point cloud track or a point cloud item. The point cloud track comprises multiple point cloud samples having a time sequence, for example, one point cloud sample in the point cloud track can be a point cloud frame having a corresponding time stamp, which is equivalent to a video frame in a dynamic video. The point cloud item comprises multiple point cloud samples having a non-time sequence, for example, one point cloud sample in the point cloud item can be a point cloud image having no time sequence relationship, which is equivalent to one image in a static atlas.

[0095] The embodiment of the application utilizes different values of the entity identifier field configured in the information data box to indicate the replaceable relationship between the point cloud bitstreams. For example, the entity identifier field corresponding to the point cloud track track1 takes a value of 1, indicating that the track is contained in two replaceable point cloud bitstreams bitstream1 and bitstream2. The entity identifier fields corresponding to the point cloud tracks track2 and track3 both take a value of 0, indicating that the point cloud track track2 and the point cloud track track3 are respectively one replaceable track in the replaceable group. The point cloud track track1 and the point cloud track track2 can constitute the point cloud bitstream bitstream1, and the point cloud track track1 and the point cloud track track3 can constitute another point cloud bitstream bitstream2. According to the indication function of the entity identifier field, the replaceable relationship between the point cloud bitstreams bitstream1 and bitstream2 can be clearly identified.

[0096] In one embodiment of the application, the information data box further comprises a replacement identifier field, the replacement identifier field being used to indicate an identifier of the replaceable group to which the content entity belongs. For example, in the information data box of the point cloud track track2, the replacement identifier field takes a value of 2; in the information data box of the point cloud track track3, the replacement identifier field also takes a value of 2. Thus, it can be indicated that the point cloud track track2 and the point cloud track track3 belong to the same replaceable group with the identifier 2.

[0097] In an embodiment of the present application, when the entity identification field indicates that the content entity is contained in at least two replaceable point cloud bitstreams, the information data box further comprises a set number field and a set identification field, the set number field is used to indicate the number of content entities contained in the replaceable group, and the set identification field is used to indicate the identifier of the replaceable group.

[0098] For example, in the information data box of the point cloud track track1, the entity identification field takes the value of 1, indicating that the track is contained in two replaceable point cloud bitstreams bitstream1 and bitstream2. The entity identification fields corresponding to the point cloud tracks track2 and track3 both take the value of 0, and the replacement identification fields both take the value of 2, indicating that the point cloud track track2 and the point cloud track track3 belong to the same replaceable group with the identifier 2. The point cloud track track1 and the point cloud track track2 can constitute the point cloud bitstream bitstream1, and the point cloud track track1 and the point cloud track track3 can constitute another point cloud bitstream bitstream2. In order to identify the composition relationship of the point cloud bitstream bitstream1 and the point cloud bitstream bitstream2, the set number field and the set identification field can be configured in the information data box of the point cloud track track1; wherein the set number field takes the value of 1, and the set identification field takes the value of 2, indicating that the point cloud track track1 is used only by the replaceable group with the identifier 2 to form the replaceable point cloud bitstreams bitstream1 and bitstream2.

[0099] For another example, the point cloud track track4 shared in the point cloud bitstream bitstream1 and the point cloud bitstream bitstream2 also comprises a shared point cloud track track4, and the point cloud track track4 belongs to another replaceable group with the identifier 3. Then in the information data box of the point cloud track track1, the set number field can be configured to take the value of 2, and the set identification field takes the values of 2 and 3, indicating that the point cloud track track1 is used by two replaceable groups with the identifiers 2 and 3 respectively to form the replaceable point cloud bitstreams bitstream1 and bitstream2.

[0100] In an embodiment of the present application, when the content entity is a point cloud track, the information data box is contained in the sample entry of the point cloud track.

[0101] The embodiments of the present application can make syntax extension to the ISOBMFF data box of the point cloud track, so as to realize the replaceability identification of the point cloud bitstream as described above.

[0102] Figure 7 The data structure of the newly added replaceable information data box in an embodiment of the present application is shown.

[0103] The `shared_alternative_track_flag` field is the entity identifier field. A value of 0 indicates that the current point cloud track is one of multiple alternative tracks in a set of alternative tracks. A value of 1 indicates that the current point cloud track is shared by multiple tracks in the set of alternative tracks.

[0104] The alternative_id field is the replacement identifier field, which indicates the identifier of the alternative group to which the current track belongs.

[0105] num_share_alternative_group is a set quantity field that indicates the number of alternative group track sets that share the current point cloud track.

[0106] The share_alternative_group_id field is a set identifier field that indicates the identifier of an alternative group that shares the current point cloud orbit.

[0107] In one embodiment of this application, when encapsulating content entities into point cloud files based on information data boxes, various encapsulation modes can be used. For example, single-track encapsulation, component-based multi-track encapsulation, and segmented multi-track encapsulation. Segmented multi-track encapsulation can further include a mode that encapsulates both geometric data and attribute data, or a mode that encapsulates geometric data and attribute data separately.

[0108] Figure 8 This illustration shows a point cloud file encapsulation structure based on single-track encapsulation in one embodiment of this application. For example... Figure 8 As shown, a point cloud track is encapsulated as a single point cloud file. The point cloud track includes a sample entry point and multiple point cloud samples. Each point cloud sample may contain parameter information, geometric data, and attribute data.

[0109] Figure 9 This illustrates a point cloud file encapsulation structure based on component-based multi-track encapsulation in one embodiment of this application. For example... Figure 9 As shown, a geometric component track1 and two attribute components track2 and track3 are encapsulated together into a single point cloud file. Geometric component track1 contains a sample entry point and multiple samples, each containing parameter information and geometric data. Attribute component track2 contains a sample entry point and multiple samples, each containing parameter information and attribute 1 data. Attribute component track3 contains a sample entry point and multiple samples, each containing parameter information and attribute 2 data. Attribute 1 and attribute 2 are two different point cloud attributes, such as color, material, or other attribute information.

[0110] In an embodiment of the present application, when the point cloud bitstream uses the component-based multi-track encapsulation mode, the point cloud bitstream includes a geometry component track for encapsulating point cloud geometry data and an attribute component track for encapsulating point cloud attribute data.

[0111] When the information data box is contained in the sample entry of the geometry component track, the entity identification field indicates that the geometry component track is combined with at least two attribute component tracks in the replaceable group respectively to form a replaceable point cloud bitstream.

[0112] When the information data box is contained in the sample entry of the attribute component track, the entity identification field is used to indicate that the attribute component track is one track in the replaceable group, or is used to indicate that the attribute component track is combined with at least two attribute component tracks in the replaceable group respectively to form a replaceable point cloud bitstream.

[0113] For example, in an application scenario of an embodiment of the present application, when the data box is contained in the sample entry of the point cloud geometry component track, the shared_alternative_track_flag field in the data box must take the value 1, at this time indicating that the point cloud geometry component track is shared by a replaceable plurality of attribute component tracks.

[0114] When the data box is contained in the sample entry of the point cloud attribute component track, if the shared_alternative_track_flag field in the data box takes the value 1, the point cloud geometry component track corresponding to the point cloud attribute component track must contain the GPCCAlternativeInfoBox and the shared_alternative_track_flag field must take the value 1, at this time indicating that the point cloud attribute component and the corresponding point cloud geometry track are shared by other replaceable attribute component tracks.

[0115] Figure 10 A point cloud file encapsulation structure based on slice-based multi-track encapsulation (geometry data and attribute data are jointly encapsulated) in an embodiment of the present application is shown. As shown in Figure 10 The point cloud file contains three point cloud slice tracks, a slicebase track, a slice track 1 and a slice track 2.

[0116] The point cloud slice track slicebase track includes a sample entry and a plurality of samples, wherein each sample contains a geometry header and an attribute header for saving metadata.

[0117] The point cloud slice track slice track 1 includes a sample entry and a plurality of samples, each of which can include a geometry slice header, geometry data, an attribute slice header, attribute data of a point cloud slice slice 1, and a geometry slice header, geometry data, an attribute slice header, attribute data of a point cloud slice slice 2.

[0118] The point cloud slice track slice track 2 includes a sample entry and a plurality of samples, each of which can include a geometry slice header, geometry data, an attribute slice header, attribute data of a point cloud slice slice 3.

[0119] Figure 11 A point cloud file encapsulation structure based on a slice-based multi-track encapsulation (geometric data and attribute data are encapsulated respectively) in an embodiment of the present application is shown. As shown in FIG. 1, the point cloud file includes five point cloud slice tracks slicebase track, slice track 1, slice track 2, slice track 3, and slice track 4. Figure 11 As shown in FIG. 1, the point cloud file includes five point cloud slice tracks slicebase track, slice track 1, slice track 2, slice track 3, and slice track 4.

[0120] The point cloud slice track slicebase track includes a sample entry and a plurality of samples, each of which includes a geometry header and an attribute header for saving metadata.

[0121] The point cloud slice track slice track 1 is a geometry component track, including a sample entry and a plurality of samples, each of which can include a geometry slice header, geometry data of a point cloud slice slice 1, and a geometry slice header, geometry data of a point cloud slice slice 2.

[0122] The point cloud slice track slice track 2 is an attribute component track, including a sample entry and a plurality of samples, each of which can include an attribute slice header, attribute data of a point cloud slice slice 1, and an attribute slice header, attribute data of a point cloud slice slice 2.

[0123] The point cloud slice track slice track 3 is a geometry component track, including a sample entry and a plurality of samples, each of which can include a geometry slice header, geometry data of a point cloud slice slice 3.

[0124] The point cloud slice track slice track 4 is an attribute component track, including a sample entry and a plurality of samples, each of which can include an attribute slice header, attribute data of a point cloud slice slice 3.

[0125] In an embodiment of the present application, when the point cloud code stream uses a slice-based multi-track encapsulation mode, the point cloud code stream includes a point cloud slice track encapsulating samples in the form of point cloud slices.

[0126] When the information data box is contained in the sample entry of the point cloud slice track, and the point cloud slice track contains point cloud geometry data, the entity identification field indicates that the point cloud slice track is combined with at least two tracks in the replaceable group containing point cloud attribute data respectively to form a replaceable point cloud code stream.

[0127] When the information data box is contained in the sample entry of the point cloud slice track, and the point cloud slice track contains point cloud attribute data, the entity identification field is used to indicate that the point cloud slice track is one track in the replaceable group, or is used to indicate that the point cloud slice track is combined with at least two tracks in the replaceable group containing point cloud attribute data respectively to form a replaceable point cloud code stream.

[0128] For example, in one application scenario of the embodiments of the present application, when the data box is contained in the sample entry of the point cloud slice track, and the point cloud slice track contains geometry component data, the shared_alternative_track_flag field in the data box must be valued as 1, which indicates that the point cloud geometry component track is shared by a replaceable plurality of point cloud slice tracks containing attribute component data.

[0129] When the data box is contained in the sample entry of the point cloud slice track, and the point cloud slice track contains attribute component data, if the shared_alternative_track_flag field in the data box is valued as 1, the point cloud slice track corresponding to the point cloud slice track containing geometry component data must contain the GPCCAlternativeInfoBox and the shared_alternative_track_flag field must be valued as 1, which indicates that the point cloud slice track containing attribute component data and the corresponding point cloud slice track containing geometry component data are shared by other replaceable point cloud slice tracks containing attribute component data.

[0130] In one embodiment of the present application, based on the AVS standard, the replaceable relationship between point cloud code streams can also be indicated by extending the replaceable information structure and related track groups and entity groups.

[0131] Figure 12 The point cloud replaceable information structure based on the AVS standard in one embodiment of the present application is shown. As shown in Figure 12 When the point cloud track has replaceable tracks, the point cloud replaceable information structure AvsPCCAlternativeInfoStruct is used to indicate the difference between a plurality of tracks in the replaceable group.

[0132] quality_ranking_flag is equal to 1, it indicates that there is quality alternative relationship between the tracks in the alternative group; and when quality_ranking_flag is equal to 0, it indicates that there is no quality alternative relationship between the tracks in the alternative group.

[0133] quality_ranking indicates quality ranking information, and the smaller the value of the field is, the higher the quality of the corresponding track is.

[0134] shared_alternative_track_flag is equal to 0, it indicates that the current point cloud track is one of the alternative track set; and when shared_alternative_track_flag is equal to 1, it indicates that the current point cloud track is shared by multiple tracks in the alternative track set.

[0135] alternative_id indicates the identifier of the alternative group to which the current track belongs.

[0136] num_share_alternative_group indicates the number of the alternative track set sharing the current point cloud track.

[0137] share_alternative_group_id indicates the identifier of the alternative group sharing the current point cloud track.

[0138] Figure 13 The data box structure of the point cloud alternative track group based on the AVS standard in one embodiment of the present application is shown. The point cloud tracks can have alternative relationship, and the alternative tracks belong to one point cloud alternative track group.

[0139] Figure 14 The data box structure of the non-timed point cloud media alternative group based on the AVS standard in one embodiment of the present application is shown. The point cloud items can have alternative relationship, and the alternative items belong to one point cloud alternative group.

[0140] group_id indicates the identifier of the current entity group.

[0141] num_entities_in_group indicates the number of the entities (tracks or items) in the current entity group.

[0142] entity_id indicates the identifier of the entity.

[0143] AvsPCCAlternativeInfoStruct indicates the alternative information of the current entity.

[0144] In an embodiment of the present application, after encapsulating the content entity as a point cloud file according to the information data box, a signaling file for describing the point cloud file can also be generated, and the signaling file includes an entity identification element, which is used to indicate that the content entity is one entity in the replaceable group or to indicate that the content entity is contained in at least two replaceable point cloud code streams.

[0145] In an embodiment of the present application, the signaling file further includes a replacement identification element, which is used to indicate an identifier of the replaceable group to which the content entity belongs.

[0146] In an embodiment of the present application, when the entity identification element indicates that the content entity is contained in at least two replaceable point cloud code streams, the signaling file further includes a set identification element, which is used to indicate an identifier of the replaceable group.

[0147] Taking DASH signaling as an example, in an application scenario of an embodiment of the present application, a SupplementalProperty element with an @schemeIdUri attribute value of "urn:mpeg:mpegI:gpcc:2020:alternativeInfo" can be configured in the DASH signaling to represent an alternative information descriptor. The descriptor is used to define specific information of representations corresponding to tracks in a replaceable group.

[0148] The descriptor can describe media resources at the Representation, Adaptation Set or Preselection level. When the descriptor describes the Adaptation Set or the Preselection level, it means that all representations in the Adaptation Set or the Preselection level correspond to the same alternative information.

[0149] One or more alternative information descriptors can exist in an MPD signaling.

[0150] Table 1 shows the syntax and semantics of the alternative information descriptor in the MPD signaling.

[0151] Table 1

[0152]

[0153]

[0154] In one embodiment of this application, the existing Preselection structure in DASH signaling can also be used to indicate the interchangeability relationship between point cloud code streams. For example, in this embodiment, each interchangeable track can be organized into an Adaptation Set, and a series of Adaptation Sets can be identified using the @preselectionComponents element in Preselection, where the @id element in Preselection is equal to the corresponding alternative group id.

[0155] Alternatively, in embodiments of this application, each replaceable track can be organized into a Representation, and a series of Adaptation Sets can be identified using the @preselectionComponents element in Preselection, where the @id element in Preselection is equal to the corresponding alternative group id.

[0156] Figure 15 This application illustrates a point cloud decapsulation method based on the replaceability relationship of the point cloud bitstream in one embodiment of the present application. This method can be applied to the server side, player side, or intermediate node of a point cloud media system.

[0157] like Figure 15 As shown, the point cloud decapsulation method based on the replaceable relationship of the point cloud code stream may include the following steps S1510 to S1520.

[0158] S1510: Obtain the information data box of the content entity in the point cloud file. The information data box includes an entity identifier field. The entity identifier field is used to indicate that the content entity is an entity in a replaceable group, or to indicate that the content entity is contained in at least two replaceable point cloud streams. A replaceable group is a set of entities consisting of multiple content entities that have a replaceable relationship.

[0159] S1520: Decapsulate the point cloud file based on the information data box to obtain a point cloud code stream composed of content entities.

[0160] In one embodiment of this application, before obtaining the information data box of the content entity in the point cloud file, the method further includes: obtaining a signaling file for describing the point cloud file, the signaling file including an entity identification element, the entity identification element being used to indicate that the content entity is an entity in a replaceable group, or to indicate that the content entity is included in at least two replaceable point cloud streams; and requesting the transmission of the point cloud file according to the signaling file.

[0161] The implementation details of the point cloud unpackaging method in the embodiments of the present application can refer to the above-described embodiments of the point cloud packaging method, which will not be described here again.

[0162] The point cloud packaging and unpackaging methods provided in the embodiments of the present application are described below in an application scenario as an example.

[0163] The point cloud transmission scheme in the application scenario includes the following processes.

[0164] (1) The server packages point cloud bitstreams into a point cloud file.

[0165] When the N point cloud bitstreams are alternative versions of the same content with different qualities, if the geometry data of M (greater than 1) bitstreams are obtained in the same encoding manner, and the M point cloud bitstreams are packaged into a file in a component-based multi-track packaging manner: only one geometry component track is retained, and the relationship between the shared geometry component track and other alternative attribute component tracks is indicated.

[0166] In the M bitstreams, if the point cloud attribute data of K bitstreams in one or more groups are obtained in the same encoding manner, for one group of point cloud attribute data of the same type, only one attribute component track is retained; and the relationship between the shared point cloud attribute track and the geometry component track is indicated.

[0167] Figure 15 The point cloud file structure obtained by the point cloud packaging in the application scenario is shown.

[0168] As shown in Figure 15 , the server packages point cloud bitstreams into a point cloud file, in which the three point cloud bitstreams bitstream1, bitstream2 and bitstream3 are alternative versions of the same content with different qualities, and the geometry data of bitstream1 and bitstream2 are obtained in the same encoding manner, and bitstream1 and bitstream2 are packaged into a file in a component-based multi-track packaging manner. A certain attribute (such as reflectivity) in bitstream1 and bitstream2 is also obtained in the same encoding manner. Bitstream3 is packaged in a single-track manner.

[0169] In the point cloud file: for the geometry track and the reflectivity attribute track in bitstream1 and bitstream2, only one track is retained; the relationship between the shared geometry track, the reflectivity attribute track and the two color attribute tracks is indicated.

[0170] Track 1: {shared_alternative_track_flag = 1; alternative_id = 1;

[0171] num_share_alternative_group = 2; share_alternative_group_id = {2, 3}}

[0172] Track 2: {shared_alternative_track_flag = 0; alternative_id = 2}

[0173] Track 3: {shared_alternative_track_flag = 0; alternative_id = 2}

[0174] Track 4: {shared_alternative_track_flag = 1; alternative_id = 3;

[0175] num_share_alternative_group = 1; share_alternative_group_id = {2}}

[0176] Track 5: {shared_alternative_track_flag = 0; alternative_id = 1}

[0177] (2) The server transmits the point cloud file to the client in whole.

[0178] (3) The client receives the complete point cloud file F1, which contains all the alternative point cloud tracks. When the client performs unpacking, decoding and rendering on the point cloud file, the client selects a track in F1 to decode and render according to the performance of the user device and the requirement of the rendering effect, in combination with the corresponding metadata information in the file.

[0179] According to the data box information in the point cloud file, it can be known that the client has three choices:

[0180] Track 1 + Track 2 + Track 4;

[0181] Track 1 + Track 3 + Track 4;

[0182] Track 5.

[0183] The following takes another application scenario as an example to illustrate the scheme of point cloud transmission between the server and the client according to the point cloud encapsulation and encapsulation method provided in the embodiments of the present application.

[0184] The scheme of point cloud transmission in this application scenario includes the following processes.

[0185] (1) The server encapsulates the point cloud code stream into a point cloud file.

[0186] The encapsulation structure of the point cloud file is the same as the above-mentioned application scenario, which will not be described here.

[0187] (2) The server generates a corresponding signaling description file according to the sharing and replaceable relationship between the geometry component track and the attribute component track, and transmits the signaling file to the client.

[0188] Based on the point cloud file encapsulated in the above-mentioned application scenario, the generated signaling file includes the following information.

[0189] Representation1 (Track1):

[0190] PCCAlternativeInfo@shared_alternative_track_flag = 1;

[0191] PCCAlternativeInfo@alternative_id = 1;

[0192] PCCAlternativeInfo@share_alternative_group_id = {2, 3}.

[0193] Representation2 (Track2):

[0194] PCCAlternativeInfo@shared_alternative_track_flag = 0;

[0195] PCCAlternativeInfo@alternative_id = 2.

[0196] Representation3 (Track3):

[0197] PCCAlternativeInfo@shared_alternative_track_flag = 0;

[0198] PCCAlternativeInfo@alternative_id = 2.

[0199] Representation4 (Track4):

[0200] PCCAlternativeInfo@shared_alternative_track_flag = 1;

[0201] PCCAlternativeInfo@alternative_id = 3;

[0202] PCCAlternativeInfo@share_alternative_group_id = {2}.

[0203] Representation5 (Track5):

[0204] PCCAlternativeInfo@shared_alternative_track_flag = 0;

[0205] PCCAlternativeInfo@alternative_id = 1.

[0206] (3) The client receives the signaling file, parses the signaling file, and requests the corresponding transport stream (Fs corresponding to one or more tracks in the file) according to its own needs. The client performs decapsulation and decoding on the received file segment, and finally presents the point cloud content.

[0207] According to the signaling information, the client has three choices:

[0208] Representation1 + Representation2 + Representation4;

[0209] Representation1 + Representation3 + Representation4;

[0210] Representation5.

[0211] It should be noted that although the steps of the method in the present application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. In addition or alternatively, some steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of steps, etc.

[0212] The device embodiment of the present application is introduced below, which can be used to execute the method in the above-mentioned embodiments of the present application.

[0213] Figure 16A structural block diagram of a point cloud packaging device provided by an embodiment of the present application is shown schematically.

[0214] As shown in Figure 16 The point cloud packaging device 1600 includes:

[0215] A first obtaining module 1610 is configured to obtain an information data box of a content entity in a point cloud bitstream, the information data box including an entity identifier field; the entity identifier field is used to indicate that the content entity is one entity in a replaceable group, or is used to indicate that the content entity is contained in at least two replaceable point cloud bitstreams; the replaceable group is an entity set composed of multiple content entities having a replaceable relationship;

[0216] A packaging module 1620 is configured to package the content entity into a point cloud file according to the information data box.

[0217] In an embodiment of the present application, based on the point cloud packaging device in the above embodiment, the information data box further includes a replacement identifier field, the replacement identifier field being used to indicate an identifier of a replaceable group to which the content entity belongs.

[0218] In an embodiment of the present application, based on the point cloud packaging device in the above embodiment, when the entity identifier field indicates that the content entity is contained in at least two replaceable point cloud bitstreams, the information data box further includes a set number field and a set identifier field, the set number field being used to indicate a number of the replaceable groups that commonly use the content entity, and the set identifier field being used to indicate an identifier of the replaceable group that commonly uses the content entity.

[0219] In an embodiment of the present application, based on the point cloud packaging device in the above embodiment, the content entity is a point cloud track or a point cloud item, the point cloud track including multiple point cloud samples having a time sequence, and the point cloud item including multiple point cloud samples having a non-time sequence.

[0220] In an embodiment of the present application, based on the point cloud packaging device in the above embodiment, when the content entity is a point cloud track, the information data box is contained in a sample entry of the point cloud track.

[0221] In an embodiment of the present application, based on the point cloud packaging device in the above embodiment, when the point cloud bitstream uses a component-based multi-track packaging mode, the point cloud bitstream includes a geometry component track used to package point cloud geometry data and an attribute component track used to package point cloud attribute data.

[0222] In an embodiment of the present application, based on the point cloud encapsulation apparatus in the above embodiment, when the information data box is contained in the sample entry of the geometry component track, the entity identification field indicates that the geometry component track is combined with at least two attribute component tracks in the replaceable group respectively to form a replaceable point cloud bitstream.

[0223] When the information data box is contained in the sample entry of the attribute component track, the entity identification field is used to indicate that the attribute component track is one track in the replaceable group, or is used to indicate that the attribute component track is combined with at least two attribute component tracks in the replaceable group respectively to form a replaceable point cloud bitstream.

[0224] In an embodiment of the present application, based on the point cloud encapsulation apparatus in the above embodiment, when the point cloud bitstream uses a multi-track encapsulation mode based on a slice, the point cloud bitstream includes a point cloud slice track taking a point cloud slice as an encapsulation sample.

[0225] In an embodiment of the present application, based on the point cloud encapsulation apparatus in the above embodiment, when the information data box is contained in the sample entry of the point cloud slice track, and the point cloud slice track contains point cloud geometry data, the entity identification field indicates that the point cloud slice track is combined with at least two tracks containing point cloud attribute data in the replaceable group respectively to form a replaceable point cloud bitstream.

[0226] When the information data box is contained in the sample entry of the point cloud slice track, and the point cloud slice track contains point cloud attribute data, the entity identification field is used to indicate that the point cloud slice track is one track in the replaceable group, or is used to indicate that the point cloud slice track is combined with at least two tracks containing point cloud attribute data in the replaceable group respectively to form a replaceable point cloud bitstream.

[0227] In an embodiment of the present application, based on the point cloud encapsulation apparatus in the above embodiment, the point cloud encapsulation apparatus 1600 further includes:

[0228] A generation module configured to generate a signaling file for describing the point cloud file, the signaling file including an entity identification element, the entity identification element being used to indicate that the content entity is one entity in a replaceable group, or is used to indicate that the content entity is contained in at least two replaceable point cloud bitstreams.

[0229] In an embodiment of the present application, based on the point cloud encapsulation apparatus in the above embodiment, the signaling file further includes a replacement identification element, the replacement identification element being used to indicate an identifier of the replaceable group to which the content entity belongs.

[0230] In an embodiment of the present application, based on the point cloud encapsulation apparatus in the above embodiment, when the entity identification element indicates that the content entity is contained in at least two replaceable point cloud bitstreams, the signaling file further comprises a set identification element, and the set identification element is used to indicate an identifier of the replaceable group.

[0231] Figure 17 The structure block diagram of the point cloud decapsulation apparatus provided by the embodiments of the present application is schematically shown. As shown in the figure, Figure 17 The point cloud decapsulation apparatus 1700 comprises:

[0232] The second acquisition module 1710 is configured to acquire an information data box of a content entity in a point cloud file, and the information data box comprises an entity identification field; the entity identification field is used to indicate that the content entity is one entity in a replaceable group, or is used to indicate that the content entity is contained in at least two replaceable point cloud bitstreams; and the replaceable group is an entity set composed of multiple content entities having a replaceable relationship.

[0233] The decapsulation module 1720 is configured to decapsulate the point cloud file according to the information data box, to obtain a point cloud bitstream composed of the content entity.

[0234] In an embodiment of the present application, based on the point cloud decapsulation apparatus in the above embodiment, the point cloud decapsulation apparatus further comprises:

[0235] The third acquisition module is configured to acquire a signaling file used to describe the point cloud file, and the signaling file comprises an entity identification element, and the entity identification element is used to indicate that the content entity is one entity in a replaceable group, or is used to indicate that the content entity is contained in at least two replaceable point cloud bitstreams.

[0236] The transmission module is configured to request transmission of the point cloud file according to the signaling file.

[0237] The specific details of the point cloud encapsulation apparatus and the point cloud decapsulation provided in the embodiments of the present application have been described in detail in the corresponding method embodiments, and will not be repeated here.

[0238] Figure 18 The structure block diagram of a computer system of an electronic device for implementing the embodiments of the present application is schematically shown.

[0239] It should be noted that, Figure 18 The computer system 1800 of the electronic device shown is only one example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0240] As Figure 18As shown, the computer system 1800 includes a central processing unit 1801 (CPU), which can execute various appropriate actions and processes according to programs stored in a read-only memory 1802 (ROM) or loaded into a random access memory 1803 (RAM) from a storage section 1808. In the random access memory 1803, various programs and data required for the operation of the system are also stored. The central processing unit 1801, the read-only memory 1802, and the random access memory 1803 are connected to each other through a bus 1804. An input / output interface 1805 (I / O interface) is also connected to the bus 1804.

[0241] Connected to the input / output interface 1805 are an input section 1806 including a keyboard, a mouse, etc.; an output section 1807 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1808 including a hard disk, etc.; and a communication section 1809 including a network interface card such as a local area network card, a modem, etc. The communication section 1809 performs communication processing via a network such as the Internet. A drive 1810 is also connected to the input / output interface 1805 as necessary. A removable recording medium 1811 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 1810 as necessary, so that a computer program read therefrom is installed into the storage section 1808 as necessary.

[0242] In particular, according to embodiments of the present application, the processes described in the various method flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 1809, and / or installed from the removable recording medium 1811. When the computer program is executed by the central processing unit 1801, various functions defined in the system of the present application are executed.

[0243] It should be noted that the computer-readable medium in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, device or apparatus. In this application, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium that can send, propagate or transmit the program for use by or in connection with an instruction execution system, device or apparatus. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, or the like, or any suitable combination thereof.

[0244] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders than that shown in the figures. For example, two blocks noted in succession can actually be executed substantially concurrently, or they can sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the flowcharts or block diagrams, and combinations of blocks in the flowcharts or block diagrams, can be implemented by special-purpose hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0245] It should be noted that, although several modules or units for a device for action execution are mentioned in the above detailed description, such a division is not mandatory. Indeed, according to an embodiment of the application, the features and functionalities of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functionalities of one module or unit described above can be further divided into several modules or units embodied.

[0246] Those skilled in the art can easily understand, through the above description of the embodiments, that the example embodiments described herein can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) or a network, and includes several instructions to make a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) execute the methods according to the embodiments of the present application.

[0247] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the present application cover any and all variations of the application that come within the scope of the present application, along with all of the equivalents thereof. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0248] It should be understood that the present application is not limited to the precise construction that has been described above and illustrated in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the present application. The scope of the present application is limited only by the appended claims.

Claims

1. A point cloud packaging method, characterized by, The method comprises: acquiring an information data box of a content entity in a point cloud code stream, the information data box comprising an entity identification field; the entity identification field is used to indicate that the content entity is one entity in a replaceable group, or is used to indicate that the content entity is simultaneously contained in at least two replaceable point cloud code streams; the replaceable group is an entity set composed of multiple content entities having a replaceable relationship; encapsulating the content entity into a point cloud file according to the information data box.

2. The point cloud packing method of claim 1, wherein, The information data box further comprises a replacement identification field, which is used to indicate an identifier of the replaceable group to which the content entity belongs.

3. The point cloud packing method of claim 1, wherein, When the entity identification field indicates that the content entity is contained in at least two replaceable point cloud code streams, the information data box further comprises a set number field and a set identification field, the set number field is used to indicate the number of the replaceable group that uses the content entity in common, and the set identification field is used to indicate the identifier of the replaceable group that uses the content entity in common.

4. The point cloud packing method of claim 1, wherein, The content entity is a point cloud track or a point cloud item, the point cloud track comprises multiple point cloud samples having a time sequence, and the point cloud item comprises multiple point cloud samples having a non-time sequence.

5. The point cloud packing method according to any one of claims 1 to 4, characterized in that, When the content entity is a point cloud track, the information data box is contained in a sample entry of the point cloud track.

6. The point cloud packing method of claim 5, wherein, When the point cloud code stream uses a component-based multi-track encapsulation mode, the point cloud code stream comprises a geometry component track used for encapsulating point cloud geometry data and an attribute component track used for encapsulating point cloud attribute data.

7. The point cloud packing method of claim 6, wherein, When the information data box is contained in a sample entry of the geometry component track, the entity identification field indicates that the geometry component track is respectively combined with at least two attribute component tracks in a replaceable group to form replaceable point cloud code streams; When the information data box is contained in a sample entry of the attribute component track, the entity identification field is used to indicate that the attribute component track is one track in the replaceable group, or is used to indicate that the attribute component track and a corresponding geometry component track are shared by other replaceable attribute component tracks.

8. The point cloud packing method of claim 5, wherein, When the point cloud code stream uses a component-based multi-track encapsulation mode, the point cloud code stream comprises a geometry component track used for encapsulating point cloud geometry data and an attribute component track used for encapsulating point cloud attribute data.

9. The point cloud packing method of claim 8, wherein, When the information data box is contained in a sample entry of the point cloud slice track, and the point cloud slice track contains point cloud geometry data, the entity identification field indicates that the point cloud slice track is respectively combined with at least two tracks in the replaceable group containing point cloud attribute data to form replaceable point cloud code streams; When the information data box is contained in a sample entry of the point cloud slice track, and the point cloud slice track contains point cloud attribute data, the entity identification field is used to indicate that the point cloud slice track is one track in the replaceable group, or is used to indicate that the point cloud slice track and a corresponding point cloud slice track containing point cloud geometry data are shared by other replaceable point cloud slice tracks containing point cloud attribute data.

10. The point cloud packing method according to any one of claims 1 to 4, characterized in that, After the content entity is encapsulated into a point cloud file according to the information data box, the method further comprises: generating a signaling file for describing the point cloud file, the signaling file comprising an entity identification element, the entity identification element being used for indicating that the content entity is one entity in a replaceable group, or being used for indicating that the content entity is contained in at least two replaceable point cloud bitstreams.

11. The point cloud packing method of claim 10, wherein, The signaling file further comprises a replacement identification element, the replacement identification element being used for indicating an identifier of the replaceable group to which the content entity belongs.

12. The point cloud packing method of claim 10, wherein, When the entity identification element indicates that the content entity is contained in at least two replaceable point cloud bitstreams, the signaling file further comprises a set identification element, the set identification element being used for indicating an identifier of the replaceable group.

13. A point cloud decapsulation method, comprising: Comprising: obtaining an information data box of a content entity in a point cloud file, the information data box comprising an entity identification field; the entity identification field being used for indicating that the content entity is one entity in a replaceable group, or being used for indicating that the content entity is contained in at least two replaceable point cloud bitstreams simultaneously; the replaceable group being an entity set composed of a plurality of content entities having a replaceable relationship; decapsulating the point cloud file according to the information data box to obtain a point cloud bitstream composed of the content entity.

14. The point cloud decapsulation method of claim 13, wherein, Before obtaining the information data box of the content entity in the point cloud file, the method further comprises: obtaining a signaling file for describing the point cloud file, the signaling file comprising an entity identification element, the entity identification element being used for indicating that the content entity is one entity in a replaceable group, or being used for indicating that the content entity is contained in at least two replaceable point cloud bitstreams simultaneously; requesting transmission of the point cloud file according to the signaling file.

15. A point cloud packaging apparatus, comprising: Comprising: a first obtaining module configured to obtain an information data box of a content entity in a point cloud bitstream, the information data box comprising an entity identification field; the entity identification field being used for indicating that the content entity is one entity in a replaceable group, or being used for indicating that the content entity is contained in at least two replaceable point cloud bitstreams simultaneously; the replaceable group being an entity set composed of a plurality of content entities having a replaceable relationship; an encapsulating module configured to encapsulate the content entity into a point cloud file according to the information data box.

16. A point cloud decapsulation apparatus, comprising: Comprising: a second obtaining module configured to obtain an information data box of a content entity in a point cloud file, the information data box comprising an entity identification field; the entity identification field being used for indicating that the content entity is one entity in a replaceable group, or being used for indicating that the content entity is contained in at least two replaceable point cloud bitstreams simultaneously; the replaceable group being an entity set composed of a plurality of content entities having a replaceable relationship; a decapsulating module configured to decapsulate the point cloud file according to the information data box to obtain a point cloud bitstream composed of the content entity.

17. A computer readable medium characterized by The computer readable medium has stored thereon a computer program, which, when executed by a processor, implements the method of any one of claims 1 to 14.

18. An electronic device, comprising: Comprising: a processor; and a memory for storing executable instructions of the processor; The processor is configured to, via execution of the executable instructions, cause the electronic device to perform the method of any one of claims 1-14.

19. A computer program product comprising a computer program, characterized in that, The computer program, when executed by the processor, implements the method of any one of claims 1-14.

Images