Point group encoding / decoding method, device, and electronic device

Abstract

The present application belongs to the technical field of computers and communications, and specifically relates to a point cloud encoding / decoding method, a point cloud encoding / decoding device, a computer-readable medium, and an electronic device. In an embodiment of the present application, a point cloud decoding method includes the steps of: receiving a point cloud file transmitted by a data source, the point cloud file includes one or more point cloud media tracks having the same point cloud content, and the point cloud file includes some point cloud media tracks with different frame rates; analyzing file encapsulation information of the one or more point cloud media tracks to obtain frame rate indication information carried in the file encapsulation information, the frame rate indication information being used to indicate the frame rate of the one or more point cloud media tracks; and selecting and decoding a point cloud media track having a designated frame rate from the point cloud file based on the frame rate indication information carried in the file encapsulation information.

Description

[Technical field]

[0001] The present application relates to the technical fields of computers and communications, and more particularly to a point cloud encoding / decoding method, a point cloud encoding / decoding device, a computer-readable medium, and an electronic device.

[0002] This application claims priority to a Chinese patent application filed on April 22, 2021, bearing application number 202110437255.4 and entitled "Point cloud encoding and decoding method and related equipment." [Background technology]

[0003] A point cloud is a set of discrete points irregularly distributed in space, which represents the spatial structure and surface properties of a three-dimensional object or scene. After a large-scale point cloud data is acquired by a point cloud collection device, the point cloud data can be encoded and encapsulated for transmission and presentation to users. Summary of the Invention [Problem to be solved by the invention]

[0004] The embodiments of the present application provide a point cloud encoding / decoding method, a point cloud encoding / decoding device, a computer-readable medium, and an electronic device.

[0005] According to one aspect of an embodiment of the present application, a point cloud decoding method is provided, the method including: a step of receiving a point cloud file transmitted by a data source, the point cloud file including one or more point cloud media tracks having the same point cloud content, and the point cloud file including some point cloud media tracks with different frame rates; a step of analyzing file encapsulation information of the one or more point cloud media tracks to obtain frame rate indication information contained in the file encapsulation information, the frame rate indication information being used to indicate a frame rate of the one or more point cloud media tracks; and a step of selecting and decoding a point cloud media track having a specified frame rate from the point cloud file based on the frame rate indication information contained in the file encapsulation information.

[0006] According to one aspect of an embodiment of the present application, a point cloud decoding device is provided, the device including: a receiving module configured to receive a point cloud file transmitted by a data source, the point cloud file including one or more point cloud media tracks having the same point cloud content, and the point cloud file including some point cloud media tracks with different frame rates; an analysis module configured to analyze file encapsulation information of the one or more point cloud media tracks to obtain frame rate indication information carried in the file encapsulation information, the frame rate indication information being used to indicate a frame rate of the one or more point cloud media tracks; and a decoding module configured to select and decode a point cloud media track having a specified frame rate from the point cloud file based on the frame rate indication information carried in the file encapsulation information.

[0007] In some embodiments of the present application, based on the above technical means, the receiving module includes: a signaling receiving unit configured to receive a stream media signaling sent by a data source and used for transmitting point cloud data; a signaling analysis unit configured to analyze the stream media signaling to obtain a time domain hierarchical group identifier contained in the stream media signaling and used for identifying a track group, where the track group includes one or more point cloud media tracks having the same point cloud content, and some point cloud media tracks in the track group have different frame rates; a request sending unit configured to send a first data transmission request to the data source based on the time domain hierarchical group identifier; and a file receiving unit configured to receive a point cloud file corresponding to the first data transmission request sent by the data source.

[0008] In some embodiments of the present application, based on the above technical means, the request sending unit includes a bandwidth acquisition subunit configured to acquire a network bandwidth for data transmission with the data source, a track selection subunit configured to select one or more target point cloud media tracks having a target frame rate matching the network bandwidth from the track group based on the time domain hierarchical group identifier, and a request sending subunit configured to send a first data transmission request to the data source, the first data transmission request being used to request transmission of the one or more target point cloud media tracks.

[0009] In some embodiments of the present application, based on the above technical means, the analysis module includes an information analysis unit configured to determine a frame rate indication field corresponding to a frame rate of the one or more point cloud media tracks by analyzing file encapsulation information of the one or more point cloud media tracks, and an information determination unit configured to determine frame rate indication information of the one or more point cloud media tracks based on the value of the frame rate indication field.

[0010] In some embodiments of the present application, based on the above technical means, the point cloud decoding device further includes a frame rate acquisition module configured to acquire a frame rate of a point cloud media track to be displayed, a track selection module configured to select another point cloud media track having the same frame rate as the point cloud media track to be displayed from the point cloud file based on frame rate indication information contained in the file encapsulation information, and a first replacement module configured to decode and display the other point cloud media track by replacing the point cloud media track to be displayed with the other point cloud media track.

[0011] In some embodiments of the present application, based on the above technical means, the point cloud decoding device further includes a frame rate acquisition module configured to acquire a frame rate of a point cloud media track to be displayed, a track selection module configured to select one or more other point cloud media tracks having the same frame rate as the point cloud media track to be displayed from the point cloud file based on frame rate indication information contained in the file encapsulation information, and a first merging module configured to decode the one or more other point cloud media tracks and merge and display the one or more other point cloud media tracks with the point cloud media track to be displayed.

[0012] In some embodiments of the present application, based on the above technical means, the point cloud decoding device further includes: a frame rate acquisition module configured to acquire a frame rate of a point cloud media track to be displayed; a file acquisition module configured to receive a supplemental point cloud file transmitted by the data source by sending a second data transmission request to the data source, where the supplemental point cloud file includes another point cloud media track having the same point cloud content and the same frame rate as the point cloud media track to be displayed; and a second replacement module configured to decode and display the other point cloud media track by replacing the point cloud media track to be displayed with the other point cloud media track.

[0013] In some embodiments of the present application, based on the above technical means, the point cloud decoding device further includes: a frame rate acquisition module configured to acquire a frame rate of a point cloud media track to be displayed; a file acquisition module configured to receive a supplementary point cloud file transmitted by the data source by sending a third data transmission request to the data source, where the supplementary point cloud file includes one or more other point cloud media tracks having the same point cloud content and the same frame rate as the point cloud media track to be displayed; and a second merging module configured to decode the one or more other point cloud media tracks and merge and display the one or more other point cloud media tracks with the point cloud media track to be displayed.

[0014] According to one aspect of an embodiment of the present application, a point cloud encoding method is provided, the method including: a step of performing encoding processes on point cloud data to be transmitted according to different encoding standards to obtain multiple point cloud code streams having the same point cloud content, wherein the multiple point cloud code streams include some point cloud code streams with different frame rates; and a step of encapsulating the multiple point cloud code streams into multiple point cloud media tracks, wherein frame rate indication information corresponding to the multiple point cloud code streams is embedded in the multiple point cloud media tracks, and the frame rate indication information is used to indicate frame rates of the multiple point cloud media tracks.

[0015] According to one aspect of an embodiment of the present application, a point cloud encoding device is provided, the device including: an encoding module configured to perform encoding processes on point cloud data to be transmitted according to different encoding standards to obtain multiple point cloud code streams having the same point cloud content, where the multiple point cloud code streams include some point cloud code streams with different frame rates; and an encapsulation module configured to encapsulate the multiple point cloud code streams into multiple point cloud media tracks and embed frame rate indication information corresponding to the multiple point cloud code streams in the multiple point cloud media tracks, where the frame rate indication information is used to indicate a frame rate of the multiple point cloud media tracks.

[0016] In some embodiments of the present application, based on the above technical means, the point cloud encoding device further includes: a signaling generation module configured to generate a stream media signaling used for transmitting point cloud data; a signaling embedding module configured to embed a time domain hierarchical group identifier used for identifying a track group into the stream media signaling, where the track group includes one or more point cloud media tracks having the same point cloud content, and some point cloud media tracks in the track group have different frame rates; and a signaling transmission module configured to transmit the stream media signaling to a data receiving side.

[0017] In some embodiments of the present application, based on the above technical means, the point cloud encoding device further includes: a request receiving module configured to receive a data transmission request generated based on the stream media signaling sent by the data receiving side; and a file transmission module configured to transmit a point cloud file to the data receiving side based on the data transmission request, wherein the point cloud file includes one or more point cloud media tracks having the same point cloud content, and the point cloud file includes some point cloud media tracks with different frame rates.

[0018] In some embodiments of the present application, based on the above technical means, the encapsulation module includes an information determination unit configured to determine a frame rate indication field corresponding to the frame rate indication information in file encapsulation information of the multiple point cloud media tracks, and an information embedding unit configured to embed the frame rate indication information corresponding to the multiple point cloud code streams in the frame rate indication field in the file encapsulation information.

[0019] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored, which, when executed by a processor, realizes the method in the technical means.

[0020] According to one aspect of an embodiment of the present application, an electronic device is provided, the electronic device including a processor and a memory used for storing executable instructions of the processor, wherein the processor is configured to perform a method in the technical means by executing the executable instructions.

[0021] According to one aspect of the present invention, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium, a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method in the technical means.

[0022] In the technical means provided by the embodiments of the present application, a time-progressive relationship of point cloud media is established by associating point cloud media resources with the same frame rate and point cloud media resources with different frame rates. Based on such a time-progressive relationship, a user can request corresponding point cloud media files based on his / her needs when consuming point cloud media, thereby saving the bandwidth of the transmission network and improving the flexibility of encoding and decoding of point cloud data. [Brief description of the drawings]

[0023] [Figure 1] 1 shows a schematic diagram of an exemplary system architecture to which the technical means of the embodiments of the present application can be applied. [Diagram 2] 1 shows a method for placing a point cloud encoding device and a point cloud decoding device in a streaming transmission environment. [Diagram 3] 1 shows a step flowchart of a point cloud decoding method in one embodiment of the present application. [Figure 4] FIG. 2 illustrates a schematic diagram of a multi-track encapsulated replaceable group in one embodiment of the present application. [Diagram 5] 1 illustrates a flowchart of steps for receiving a point cloud file from a data source in one embodiment of the present application. [Figure 6] 1 shows a step flowchart of a point cloud encoding method in one embodiment of the present application. [Figure 7] 1 shows a step flowchart for encoding and decoding point cloud data in one application scene of an embodiment of the present application. [Figure 8] FIG. 2 is a schematic diagram showing a structural block diagram of a point cloud decoding device provided by an embodiment of the present application; [Figure 9] FIG. 2 is a schematic diagram showing a structural block diagram of a point cloud encoding device provided by an embodiment of the present application; [Figure 10] FIG. 2 shows a schematic block diagram of a computer system suitable for use in implementing electronic equipment according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Now, exemplary embodiments will be described more completely with reference to the drawings. However, the exemplary embodiments may be implemented in various forms and should not be understood to be limited to the examples discussed herein. On the contrary, these embodiments are provided to make the present application more complete and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0025] In addition, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a thorough understanding of the embodiments of the present application. However, as one skilled in the art will recognize, the technical means of the present application may be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be employed. In other circumstances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present application.

[0026] The block diagrams shown in the drawings are merely functional entities that do not necessarily correspond to physically separate entities, i.e. they may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0027] The flowcharts shown in the drawings are merely illustrative and do not necessarily include all the following contents and operations / steps, and are not required to be performed in the order described. For example, some operations / steps may be decomposed, some operations / steps may be merged or partially merged, and therefore the order of actual execution may be changed based on the actual situation.

[0028] A point cloud is a set of discrete points randomly distributed in space, which represents the spatial structure and surface attributes of a three-dimensional object or scene. Each point in a point cloud has at least three-dimensional position information, and may have color, material, or other information based on the difference in the application scene. Usually, each point in a point cloud has the same number of additional attributes. In terms of encoding, the point cloud media may be further divided into point cloud media compressed based on a conventional video encoding method (Video-based Point Cloud Compression, VPCC) and point cloud media compressed based on geometric features (Geometry-based Point Cloud Compression, GPCC). In the file encapsulation of the point cloud media, the three-dimensional position information is usually called the Geometry Component of the point cloud file, and the attribute information is called the Attribute Component of the point cloud file. A point cloud file may have only one geometry component, but there may be one or more attribute components.

[0029] Point clouds are widely used because they can flexibly and easily represent the spatial structure and surface attributes of 3D objects or scenes. Their main application scenes can be divided into two categories: 1) Machine-sensing point clouds, such as autonomous navigation systems, real-time patrol systems, geographic information systems, visual sorting robots, and disaster relief robots; 2) Eye-sensing point clouds, such as digital cultural heritage, free viewpoint broadcasting, 3D immersive communication, and 3D immersive interaction.

[0030] The acquisition of point cloud mainly includes computer generation, 3D laser scanning, 3D photography measurement and other methods. Computers can generate point clouds of virtual 3D objects and scenes. 3D scanning can acquire point clouds of static real-world 3D objects or scenes, and can acquire millions of point clouds per second. 3D imaging can acquire point clouds of dynamic real-world 3D objects or scenes, and can acquire tens of millions of point clouds per second. In addition, in the medical field, MRI, CT, and electromagnetic positioning information can be used to acquire point clouds of biological tissues and organs. These technologies reduce the acquisition cost and time period of point cloud data and improve the accuracy of data. The transformation of point cloud data acquisition methods enables the acquisition of large amounts of point cloud data. With the continuous accumulation of large-scale point cloud data, efficient storage, transmission, publication, sharing, and standardization of point cloud data are the keys to point cloud applications.

[0031] After the point cloud media is encoded, the encoded data stream needs to be encapsulated and transmitted to the user, and correspondingly, the point cloud media player needs to first decapsulate the point cloud file, then decode it, and finally present the decoded data stream.

[0032] FIG. 1 shows a schematic diagram of an exemplary system architecture to which the technical means of the embodiments of the present application can be applied.

[0033] As shown in Fig. 1, the system architecture 100 includes a plurality of terminal devices, which can communicate with each other, for example, via a network 150. For example, the system architecture 100 may include a first terminal device 110 and a second terminal device 120 interconnected via the network 150. In the embodiment of Fig. 1, the first terminal device 110 and the second terminal device 120 perform unidirectional data transmission.

[0034] For example, the first terminal device 110 may transmit point cloud data (e.g., a point cloud code stream collected by the first terminal device 110) to the second terminal device 120 via the network 150 by encoding the point cloud data. The encoded point cloud data is transmitted in the form of one or more encoded point cloud code streams, and the second terminal device 120 may receive the encoded point cloud data from the network 150, decode the encoded point cloud data, and display the decoded point cloud data.

[0035] In one embodiment of the present application, the system architecture 100 may include a third terminal device 130 and a fourth terminal device 140 performing bidirectional transmission of encoded point cloud data, which may occur, for example, during a video conference. For the bidirectional data transmission, each of the third terminal device 130 and the fourth terminal device 140 may transmit point cloud data (e.g., a point cloud code stream collected by the terminal device) to the other of the third terminal device 130 and the fourth terminal device 140 via the network 150. Each of the third terminal device 130 and the fourth terminal device 140 may receive the encoded point cloud data transmitted by the other of the third terminal device 130 and the fourth terminal device 140, may decode the encoded point cloud data to restore the point cloud data, and may display the point cloud data on an accessible display device based on the restored point cloud data.

[0036] In the embodiment of FIG. 1, the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140 may be a server, a personal computer, and a smartphone, but the principles disclosed herein are not limited thereto. The embodiments disclosed herein apply to laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network 150 represents any number of networks that transfer encoded point cloud data between the first terminal device 110, the second terminal device 120, the third terminal device 130, and the fourth terminal device 140, including, for example, wired and / or wireless communication networks. The network 150 may exchange data in circuit-switched and / or packet-switched channels. The network may include a telecommunications network, a local area network, a wide area network, and / or the Internet. For purposes of this application, unless otherwise construed below, the architecture and topology of the network 150 may not be important to the operation disclosed herein.

[0037] In one embodiment of the present application, Figure 2 illustrates an installation scheme of a point cloud encoding device and a point cloud decoding device in a streaming transmission environment. The subject matter disclosed in this application can be equally applied to other applications used to support point clouds, including video conferencing, digital television, and storing compressed point cloud data on digital media including CDs, DVDs, memory sticks, etc.

[0038] The streaming transmission system may include a collection subsystem 213, which may include a point cloud data source 201, such as a digital camera, which may generate, for example, uncompressed point cloud data 202. In an embodiment, the point cloud data 202 includes samples captured by a digital camera. Compared to the encoded point cloud data 204 (or the encoded point cloud code stream 204), the point cloud data 202 is rendered as a thick line to emphasize the high data content of the point cloud data. The point cloud data 202 may be processed by an electronic device 220, which includes a video encoder 203 coupled to a video source 201. The video encoder 203 may include hardware, software, or a combination of software and hardware to realize or perform aspects of the disclosed subject matter described in more detail below. Compared to the point cloud data 202, the encoded point cloud data 204 (or the encoded point cloud code stream 204) is rendered as thin lines to emphasize the relatively low data volume of the encoded point cloud data 204 (or the encoded point cloud code stream 204), which may be stored on the streaming transmission server 205 for future use. One or more streaming transmission client terminal subsystems, such as the client terminal subsystem 206 and the client terminal subsystem 208 in FIG. 2, can access the streaming transmission server 205 to retrieve the encoded point cloud data 207 and the point cloud data 209 as copies of the point cloud data 204. The client terminal subsystem 206 may include a point cloud decoder 210, for example in the electronic device 230. The point cloud decoder 210 decodes the incoming copy of the encoded point cloud data 207 and generates output point cloud data 211 that can be presented on a display 212 (e.g., a display screen) or another presentation device. In some streaming transmission systems, the point cloud data 204, point cloud data 207, and point cloud data 209 (e.g., a point cloud codestream) may be encoded based on some point cloud encoding / compression standard.Examples of these standards may include standards developed by MPEG for the GPCC.

[0039] It should be noted that electronic device 220 and electronic device 230 may include other components not shown in the figures. For example, electronic device 220 may include a point cloud decoder, and electronic device 230 may further include a point cloud encoder.

[0040] The following describes in detail the technical means, including a point cloud encoding / decoding method, a point cloud encoding / decoding device, a computer-readable medium, and an electronic device, provided by the present application in conjunction with specific embodiments.

[0041] Figure 3 shows a step flow chart of a point cloud decoding method in one embodiment of the present application, the method may be applied to the stages of a server, a client terminal, an intermediate node, etc. of a point cloud media system, and the embodiment of the present application takes as an example a point cloud decoding method executed by a client terminal device installed with a point cloud decoding device. As shown in Figure 3, the point cloud decoding method may mainly include the following steps S310 to S330.

[0042] Step S310: Receive a point cloud file transmitted by a data source, where the point cloud file includes one or more point cloud media tracks having the same point cloud content, and some point cloud media tracks in the point cloud file include those with different frame rates.

[0043] In one embodiment of the present application, a point cloud file may individually encapsulate one point cloud media track corresponding to a specified frame rate, or may encapsulate multiple point cloud media tracks with the same or different frame rates, where multiple point cloud media tracks can form a track group, and the content of the track group may include a point cloud media track (including a geometric component and an attribute component) in a single-track encapsulation mode, or a geometric component track in a multi-track encapsulation mode (the attribute component track is obtained by indexing the geometric component track). When decoding and displaying point cloud media tracks with the same frame rate, they can be substituted for each other, and a better point cloud presentation effect can be achieved by combining and consuming point cloud media tracks with the same frame rate.

[0044] In one embodiment of the present application, some point cloud media tracks in a point cloud file may have the same frame rate, while other point cloud media tracks may have different frame rates, for example, a point cloud file encapsulates three point cloud media tracks, track1, track2, and track3, where track1 has a frame rate of 60 fps, and track2 and track3 both have a frame rate of 30 fps.

[0045] A point cloud file can encapsulate alternative groups corresponding to different point cloud media tracks, where point cloud media tracks having the same point cloud content but different point cloud quality may be split within the same alternative group, and the point cloud quality may include various quality parameters corresponding to different standards, such as code rate, frame rate, resolution, etc.

[0046] When GPCC point cloud data is encapsulated in a single track, the tracks of point cloud content with different qualities belong to one replaceable group. When GPCC point cloud data is encapsulated in a multi-track, the geometric component tracks of point cloud content with different qualities belong to one replaceable group, but attribute component tracks may be associated with geometric component tracks.

[0047] 4 is a schematic diagram of a replaceable group encapsulated in multiple tracks in one embodiment of the present application. As shown in FIG. 4, a replaceable group 400 includes a first point cloud data 410 and a second point cloud data 420 having the same point cloud content. Here, the first point cloud data 410 is lossless compressed point cloud data (Lossless coded GPCC) having a relatively high point cloud quality, and the second point cloud data 420 is lossy compressed point cloud data (Lossy coded GPCC) having a relatively low point cloud quality.

[0048] The first point cloud data 410 includes a first geometric component track 411 and a first attribute component track 412 associated with the first geometric component track 411, and the second point cloud data 420 includes a second geometric component track 421 and a second attribute component track 422 associated with the second geometric component track 421.

[0049] Step S320: Analyze the file encapsulation information of one or more point cloud media tracks to obtain frame rate indication information contained in the file encapsulation information, and the frame rate indication information is used to indicate the frame rate of the one or more point cloud media tracks.

[0050] In one embodiment of the present application, a method for analyzing file encapsulation information to obtain quality indication information (frame rate indication information) may include a step of determining a frame rate indication field corresponding to a frame rate of one or more point cloud media tracks by analyzing the file encapsulation information of one or more point cloud media tracks, and a step of determining frame rate indication information of the one or more point cloud media tracks based on a value of the frame rate indication field.

[0051] In one embodiment of the present application, the file encapsulation information is an ISOBMFF (ISO Base Media File Format) data box generated when encapsulating the point cloud code stream into the point cloud media track, and the specific content of the ISOBMFF can be referred to the international standard ISO / IEC 14496-12.

[0052] In one embodiment of the present application, the file encapsulation information may be specifically expressed as an expanded track group data box TrackGroupTypeBox, the syntax of which is as follows: aligned(8) class GPCCTemporalScaleBox extends TrackGroupTypeBox('gpts') { / / track_group_id is inherited from TrackGroupTypeBox; unsigned int(8)frame_rate; } Here, frame_rate is the frame rate indication information used to indicate the corresponding frame rate of the point cloud file, and its value is an unsigned integer with a length of 8 bytes.

[0053] By extending the Track Group Data Box, point cloud media tracks of the same content but different frame rates can be associated with each other. Point cloud tracks that belong to a Track Group satisfy the following conditions:

[0054] (1) The track is a point cloud track (containing geometric and attribute components) in single-track encapsulation mode, or a geometric component track in multi-track encapsulation mode (the attribute component track is obtained by indexing with the geometric component track).

[0055] (2) Tracks with the same frame rate can be substituted for each other when decoding and presenting them.

[0056] (3) By combining and consuming tracks with the same frame rate, a higher frame rate presentation effect can be achieved.

[0057] Step S330: According to the frame rate indication information contained in the file encapsulation information, a point cloud media track having a specified frame rate is selected from the point cloud file, and decoded.

[0058] In the file encapsulation information corresponding to each point cloud media track, each point cloud media track has a frame rate indication information, which identifies the frame rate of the point cloud media track in an explicit manner. When the point cloud file is transmitted by a data source to a data sink that is a user, the data sink can decode the point cloud media track with the specified frame rate according to the device capabilities and the needs of the user.

[0059] In one embodiment of the present application, the device performance of the data receiving side is collected, and a matching detection is performed between the device performance and the quality indication information (frame rate indication information) contained in the file encapsulation information, thereby determining a frame rate that matches the device performance of the data receiving side, and further selecting and decrypting a point cloud media track having the specified frame rate from the point cloud file.

[0060] The device performance may include at least one of hardware performance, software performance, and network performance. The hardware performance may include, for example, the device model number, processor model number, memory capacity, and display size of the electronic device, the software performance may include, for example, the program version of the point cloud decoder installed by the data receiving side, and the network performance may include, for example, the network bandwidth and the network transmission state.

[0061] In one embodiment of the present application, a frame rate selection rule set by a data receiving side is obtained, and a matching detection is performed between the frame rate selection rule and the frame rate indication information contained in the file encapsulation information, a frame rate that matches the frame rate selection rule set by the data receiving side is determined, and a point cloud media track having the specified frame rate is selected from the point cloud file and decoded.

[0062] The frame rate selection rule may be a selection rule used to select point cloud data having a specified frame rate arranged based on the user's needs, for example, selecting point cloud data having a frame rate greater (or less) than a specified value based on the user's instructions.

[0063] Figure 5 shows a step flow chart of receiving a point cloud file from a data source in one embodiment of the present application. As shown in Figure 5, based on the above embodiment, the step of receiving a point cloud file transmitted by a data source in step S310 may include the following steps S510 to S540.

[0064] Step S510: Receive stream media signaling sent by a data source, which is used for transmitting point cloud data.

[0065] In one embodiment of the present application, the stream media signaling used to transmit point cloud data may be dynamic adaptive streaming over HTTP (DASH) signaling, where DASH is a type of adaptive bitrate streaming technology that can transmit high-quality stream media over the Internet through a traditional HTTP network server.

[0066] Step S520: Analyze the streaming media signaling to obtain a time-domain hierarchical group identifier contained in the streaming media signaling, which is used to identify a track group, where the track group includes one or more point cloud media tracks having the same point cloud content, and some point cloud media tracks in the track group include those with different frame rates.

[0067] In DASH signaling, existing fields can be used to store frame rate indication information for point cloud media tracks, for example the frameRate field in DASH signaling can be used to indicate the frame rates of multiple different point cloud media tracks.

[0068] In one embodiment of the present application, all frame rate point cloud media tracks belonging to one gpts track group in the file encapsulation can be identified in the DASH signaling via one group ID, for example, GPCC Temporal Scale Group Identifier (GPCCTemporalScaleGroupId). The group ID element is a sub-element of the Adaptation Set element. The GPCCTemporalScaleGroupId element may appear in the adaptation set hierarchy, but must not appear in any other hierarchy. Table 1 shows the semantics and attributes of the GPCC Temporal Scale Group Id in one embodiment of the present application.

[0069] [Table 1]

[0070] Step S530: Send a data transmission request to the data source according to the time domain hierarchical group identifier.

[0071] According to the analyzed time domain hierarchical group identifier carried in the streaming media signaling, one or more frame rate indication information can be selected as a target frame rate, and a data transmission request corresponding to the target frame rate can be further sent to the data source, where the data transmission request is, for example, a first data transmission request.

[0072] In one embodiment of the present application, a method for sending a data transmission request may include the steps of obtaining a network bandwidth for data transmission with a data source, selecting one or more target point cloud media tracks having a target frame rate matching the network bandwidth from the track group based on a time domain hierarchical group identifier, and sending a data transmission request to the data source, used to request transmission of the target point cloud media tracks.

[0073] In one embodiment of the present application, two or more target point cloud media tracks can be selected when the network bandwidth is greater than a set threshold, and one target point cloud media track can be selected when the network bandwidth is less than or equal to a set threshold.

[0074] Step S540: Receive a data transmission request and a corresponding point cloud file transmitted by a data source.

[0075] When sending a data transmission request corresponding to a target point cloud media track according to the time domain hierarchy group identifier to a data source, the data source can transmit a corresponding target point cloud media track to a data sink according to the request.When sending multiple data transmission requests corresponding to a target point cloud media track according to the time domain hierarchy group identifier to a data source, the data source can transmit multiple corresponding target point cloud media tracks with the same frame rate or different frame rates to a data sink according to the request.

[0076] In one embodiment of the present application, by obtaining multiple point cloud media files with the same frame rate, individual point cloud media files can be replaced or merged to improve the display effect of the point cloud media files.

[0077] In one embodiment of the present application, when the point cloud media track to be displayed has problems such as selection failure, decoding failure, or poor point cloud quality obtained after decoding, other point cloud media tracks with the same frame rate can be used to perform track replacement, for example, two point cloud media files with the same point cloud content, both with a frame rate of 30 fps, can be replaced with each other.

[0078] In one embodiment of the present application, when the network environment of the data receiving side is optimized and the network bandwidth is relatively high, the display frame rate of point cloud data can be improved by performing track merging of multiple point cloud media tracks with the same frame rate. For example, two point cloud media tracks with the same point cloud content, both with a frame rate of 30 fps, can form a point cloud media track with a frame rate of 60 fps after track merging, thereby achieving a better point cloud media file display effect in a way of improving the frame rate.

[0079] In one embodiment of the present application, after selecting and decoding a point cloud media track to be displayed from a point cloud file, if another point cloud media track having the same frame rate is further included in the point cloud file, a method for replacing the point cloud media track can be performed, that is, the frame rate of the point cloud media track to be displayed is obtained, and according to the frame rate indication information contained in the file encapsulation information, another point cloud media track having the same frame rate as the point cloud media track to be displayed is selected from the point cloud file, and the point cloud media track to be displayed is replaced with the other point cloud media track, thereby decoding and displaying the other point cloud media track.

[0080] In one embodiment of the present application, after selecting and decoding a point cloud media track to be displayed from a point cloud file, if the point cloud file further contains other point cloud media tracks having the same frame rate, a point cloud media track merging method can be executed, namely, obtaining the frame rate of the point cloud media track to be displayed, selecting one or more other point cloud media tracks having the same frame rate as the point cloud media track to be displayed from the point cloud file according to the frame rate indication information contained in the file encapsulation information, decode the one or more other point cloud media tracks, and merging and displaying the one or more other point cloud media tracks with the point cloud media track to be displayed.

[0081] In one embodiment of the present application, after selecting and decoding the point cloud media track to be displayed from the point cloud file, if only one point cloud media track is encapsulated in the point cloud file or none of the frame rates of the other point cloud media tracks in the point cloud file match the point cloud media track to be displayed, a method for replacing the point cloud media track can be performed, namely, obtaining the frame rate of the point cloud media track to be displayed, sending a data transmission request (e.g., a second data transmission request) to the data source to receive a supplemental point cloud file transmitted by the data source, the supplemental point cloud file including another point cloud media track having the same point cloud content and the same frame rate as the point cloud media track to be displayed, and replacing the point cloud media track to be displayed with the other point cloud media track to decode and display the other point cloud media track.

[0082] In one embodiment of the present application, after selecting and decoding the point cloud media track to be displayed from the point cloud file, if only one point cloud media track is encapsulated in the point cloud file or the frame rates of other point cloud media tracks in the point cloud file are not consistent with the point cloud media track to be displayed, a method for merging point cloud media tracks can be performed, namely, obtaining the frame rate of the point cloud media track to be displayed, and sending a data transmission request (e.g., a third data transmission request) to the data source to transmit a supplemental point cloud file transmitted by the data source. The supplemental point cloud file includes one or more other point cloud media tracks having the same point cloud content and the same frame rate as the point cloud media track to be displayed, and decoding the one or more other point cloud media tracks, and merging and displaying the one or more other point cloud media tracks with the point cloud media track to be displayed.

[0083] Figure 6 shows a step flow chart of a point cloud encoding method in one embodiment of the present application. The point cloud encoding method may be applied to the stages of a server, a client terminal, an intermediate node, etc. of a point cloud media system, and the embodiment of the present application takes as an example a point cloud encoding method executed by a server device in which a point cloud encoding device is installed. As shown in Figure 6, the point cloud encoding method may mainly include the following steps S610 to S620.

[0084] Step S610: Perform encoding processing on the point cloud data to be transmitted according to different encoding standards to obtain multiple point cloud code streams having the same point cloud content, and some point cloud code streams having different frame rates are included in the multiple point cloud code streams.

[0085] In order to meet the frame rate needs of different data receivers for point cloud data, a point cloud content can have multiple point cloud code streams with the same frame rate or different frame rates after being encoded according to multiple different encoding standards. The encoding standard may include parameter values ​​of one or multiple quality parameters, and different parameter values ​​of various quality parameters can be combined to form multiple different encoding standards. For example, the quality parameters include a code rate and a frame rate, the code rate includes two different code rate values ​​A1, A2, and the frame rate also includes two different frame rate values ​​B1, B2, so that four encoding standards A1B1, A1B2, A2B1, and A2B2 corresponding to different quality parameter values ​​can be determined.

[0086] Step S620: encapsulate the multiple point cloud code streams into multiple point cloud media tracks, and embed the multiple point cloud code streams and corresponding frame rate indication information in the multiple point cloud media tracks, where the frame rate indication information is used to indicate the frame rate of the multiple point cloud media tracks.

[0087] Each point cloud media track has a corresponding file encapsulation information, which may be an ISOBMFF data box generated when encapsulating the point cloud code stream into the point cloud media track, or may be an extended track group data box, such as a TrackGroupTypeBox. After determining the frame rate indication information and the corresponding frame rate indication field in the file encapsulation information of the point cloud media track, the frame rate indication information corresponding to the point cloud code stream can be embedded in the frame rate indication field in the file encapsulation information by quoting the frame rate of the point cloud code stream.

[0088] In one embodiment of the present application, after the point cloud code stream is encapsulated into a point cloud media track, a stream media signaling can be generated according to data transmission needs for transmitting point cloud data, and the stream media signaling can be the DASH signaling in the above embodiment. A time domain hierarchical group identifier is embedded in the stream media signaling, which is used to identify a track group, and the track group includes one or more point cloud media tracks with the same point cloud content, and some point cloud media tracks with different frame rates are included in the track group. Then, the stream media signaling is sent to the data sink, thereby transmitting point cloud data between the data source and the data sink.

[0089] In one embodiment of the present application, after the data sink responds to the stream media signaling, the data source can receive a data transmission request generated according to the stream media signaling sent by the data sink, and transmits a point cloud file to the data sink according to the data transmission request. The point cloud file includes one or more point cloud media tracks with the same point cloud content, and some point cloud media tracks in the point cloud file have different frame rates.

[0090] Fig. 7 shows a step flow chart of encoding and decoding point cloud data in one application scenario of an embodiment of the present application. As shown in Fig. 7, a server is a data source that generates point cloud data, and a method for transmitting, encoding and decoding point cloud data between the server and a client terminal of a user may include the following steps:

[0091] Step S701: Encoding process is performed on the point cloud content A on the server to obtain three point cloud code streams S1, S2, and S3 corresponding to two different frame rates. For example, the frame rate of the point cloud code stream S1 is 60 fps, the frame rate of the point cloud code stream S2 is 30 fps, and the frame rate of the point cloud code stream S3 is also 30 fps.

[0092] Step S702: Encapsulate the three point cloud code streams into three mutually associated point cloud media tracks Track1, Track2, and Track3, respectively, and embed the frame rate indication information of each point cloud media track correspondingly. For example, embed the frame rate indication field of the point cloud media track Track1 as frame_rate=60, embed the frame rate indication field of the point cloud media track Track2 as frame_rate=30, and embed the frame rate indication field of the point cloud media track Track3 as frame_rate=30 in the track group data box.

[0093] Step S703: Use the frameRate field in the DASH signaling to indicate the frame rate of each point cloud media track, and use the time domain hierarchical group identifier GPCCTemporalScaleGroupId in the DASH signaling to indicate a track group consisting of multiple point cloud media tracks, and send the DASH signaling to client terminals C1 and C2 where the user is located.

[0094] Step S704: The client terminals C1 and C2 request point cloud files according to the network bandwidth and the information in the DASH signaling, where the point cloud file requested by C1 contains a point cloud media track Track1, and the point cloud file requested by C2 contains a point cloud media track Track2.

[0095] Step S705: The server transmits point cloud files to the client terminals C1 and C2, respectively. The point cloud file transmitted by the server to the client terminal C1 includes a point cloud media track Track1, and the point cloud file transmitted by the server to the client terminal C2 includes a point cloud media track Track2.

[0096] Step S706: The client terminal receives the point cloud file, and decodes and displays the corresponding point cloud media tracks according to the frameRate field information in Track1 and Track2.

[0097] After the network condition of client terminal C2 improves, it can further request point cloud media track Track3, and present point cloud media track Track2 together with point cloud media track Track3 for consumption, thereby achieving a point cloud display effect with a frame rate of 60 fps.

[0098] In the technical means provided by the embodiments of the present application, a time-progressive relationship of point cloud media is established by associating point cloud media resources with the same frame rate and point cloud media resources with different frame rates. Based on such a time-progressive relationship, when consuming point cloud media, a user can request corresponding point cloud media files according to his / her needs, thereby saving transmission bandwidth.

[0099] It should be noted that although the figures depict individual steps of the methods herein in a particular order, this does not require or imply that the steps need to be performed in that particular order to achieve desired results, or that all of the steps shown need to be performed. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps, etc.

[0100] Hereinafter, an apparatus embodiment of the present application that can be used to execute the point cloud encoding / decoding method in the above-mentioned embodiment of the present application will be introduced.

[0101] Figure 8 shows a structural block diagram of a point cloud decoding device in one embodiment of the present application. As shown in Figure 8, the point cloud decoding device 800 may mainly include: a receiving module 810 configured to receive a point cloud file transmitted by a data source, the point cloud file includes one or more point cloud media tracks with the same point cloud content, and the point cloud file includes some point cloud media tracks with different frame rates; an analysis module 820 configured to analyze file encapsulation information of the one or more point cloud media tracks to obtain frame rate indication information carried in the file encapsulation information, the frame rate indication information being used to indicate the frame rate of the one or more point cloud media tracks; and a decoding module 830 configured to select and decode a point cloud media track having a designated frame rate from the point cloud file according to the frame rate indication information carried in the file encapsulation information.

[0102] In some embodiments of the present application, based on each of the above embodiments, the receiving module 810 includes: a signaling receiving unit configured to receive a stream media signaling used for transmitting point cloud data sent by a data source; a signaling analysis unit configured to analyze the stream media signaling to obtain a time domain hierarchical group identifier used for identifying a track group contained in the stream media signaling, where the track group includes one or more point cloud media tracks having the same point cloud content, and where some point cloud media tracks in the track group have different frame rates; a request sending unit configured to send a first data transmission request to the data source based on the time domain hierarchical group identifier; and a file receiving unit configured to receive a point cloud file corresponding to the first data transmission request sent by the data source.

[0103] In some embodiments of the present application, based on each of the above embodiments, the request sending unit includes a bandwidth acquisition subunit configured to acquire a network bandwidth for data transmission with the data source, a track selection subunit configured to select one or more target point cloud media tracks having a target frame rate matching the network bandwidth from the track group based on the time domain hierarchical group identifier, and a request sending subunit configured to send a first data transmission request to the data source, the first data transmission request being used to request transmission of the one or more target point cloud media tracks.

[0104] In some embodiments of the present application, based on each of the above embodiments, the analysis module 820 includes an information analysis unit configured to determine a frame rate indication field corresponding to a frame rate of the one or more point cloud media tracks by analyzing file encapsulation information of the one or more point cloud media tracks, and an information determination unit configured to determine frame rate indication information of the one or more point cloud media tracks based on the value of the frame rate indication field.

[0105] In some embodiments of the present application, based on each of the above embodiments, the point cloud decoding device 800 further includes a frame rate acquisition module configured to acquire a frame rate of the point cloud media track to be displayed, a track selection module configured to select another point cloud media track having the same frame rate as the point cloud media track to be displayed from the point cloud file based on frame rate indication information contained in the file encapsulation information, and a first replacement module configured to decode and display the other point cloud media track by replacing the point cloud media track to be displayed with the other point cloud media track.

[0106] In some embodiments of the present application, based on each of the above embodiments, the point cloud decoding device 800 further includes a frame rate acquisition module configured to acquire a frame rate of the point cloud media track to be displayed, a track selection module configured to select one or more other point cloud media tracks having the same frame rate as the point cloud media track to be displayed from the point cloud file based on frame rate indication information contained in the file encapsulation information, and a first merging module configured to decode the one or more other point cloud media tracks and merge and display the one or more other point cloud media tracks with the point cloud media track to be displayed.

[0107] In some embodiments of the present application, based on each of the above embodiments, the point cloud decoding device 800 further includes a frame rate acquisition module configured to acquire a frame rate of the point cloud media track to be displayed, a file acquisition module configured to receive a supplemental point cloud file transmitted by the data source by sending a second data transmission request to the data source, where the supplemental point cloud file includes another point cloud media track having the same point cloud content and the same frame rate as the point cloud media track to be displayed, and a second replacement module configured to decode and display the other point cloud media track by replacing the point cloud media track to be displayed with the other point cloud media track.

[0108] In some embodiments of the present application, based on each of the above embodiments, the point cloud decoding device 800 further includes a frame rate acquisition module configured to acquire a frame rate of the point cloud media track to be displayed, a file acquisition module configured to receive a supplemental point cloud file transmitted by the data source by sending a third data transmission request to the data source, where the supplemental point cloud file includes one or more other point cloud media tracks having the same point cloud content and the same frame rate as the point cloud media track to be displayed, and a second merging module configured to decode the one or more other point cloud media tracks and merge and display the one or more other point cloud media tracks with the point cloud media track to be displayed.

[0109] Figure 9 shows a structural block diagram of a point cloud encoding device in one embodiment of the present application. As shown in Figure 9, the point cloud encoding device 900 may mainly include: an encoding module 910, configured to perform encoding processes on point cloud data to be transmitted according to different encoding standards to obtain multiple point cloud code streams having the same point cloud content, where the multiple point cloud code streams include some point cloud code streams with different frame rates; and an encapsulation module 920, configured to encapsulate the multiple point cloud code streams into multiple point cloud media tracks and embed frame rate indication information corresponding to the multiple point cloud code streams in the multiple point cloud media tracks, where the frame rate indication information is used to indicate the frame rates of the multiple point cloud media tracks.

[0110] In some embodiments of the present application, based on each of the above embodiments, the point cloud encoding device 900 further includes a signaling generation module configured to generate a stream media signaling used for transmitting point cloud data, a signaling embedding module configured to embed a time domain hierarchical group identifier used for identifying a track group in the stream media signaling, where the track group includes one or more point cloud media tracks having the same point cloud content, and some point cloud media tracks in the track group have different frame rates, and a signaling transmission module configured to transmit the stream media signaling to a data receiving side.

[0111] In some embodiments of the present application, based on each of the above embodiments, the point cloud encoding device 900 further includes a request receiving module configured to receive a data transmission request generated based on the stream media signaling sent by the data receiving side, and a file transmission module configured to transmit a point cloud file to the data receiving side based on the data transmission request, wherein the point cloud file includes one or more point cloud media tracks having the same point cloud content, and some point cloud media tracks in the point cloud file have different frame rates.

[0112] In some embodiments of the present application, based on each of the above embodiments, the encapsulation module 920 includes an information determination unit configured to determine a frame rate indication field corresponding to the frame rate indication information in file encapsulation information of the multiple point cloud media tracks, and an information embedding unit configured to embed frame rate indication information corresponding to the multiple point cloud code streams in the frame rate indication field in the file encapsulation information.

[0113] The specific details of the point cloud encoding / decoding device provided in each embodiment of the present application are described in detail in the corresponding method embodiments, and therefore a detailed description thereof will be omitted here.

[0114] FIG. 10 is a schematic block diagram of a computer system used to implement the electronic device of the present application.

[0115] As should be explained, the electronic device computer system 1000 shown in FIG. 10 is merely an example and does not pose any limitation on the functionality and scope of use of the embodiments of the present application.

[0116] As shown in Fig. 10, the computer system 1000 includes a central processor 1001 (Central Processing Unit, CPU), which can execute various appropriate operations and processes based on programs stored in a read-only memory 1002 (Read-Only Memory, ROM) or programs loaded from a storage portion 1008 in a random access memory 1003 (Random Access Memory, RAM). In the random access memory 1003, various programs and data required for system operation are further stored. The central processor 1001, the read-only memory 1002, and the random access memory 1003 are coupled to each other via a bus 1004. An input / output interface 1005 (Input / Output interface, i.e., I / O interface) is also connected to the bus 1004.

[0117] Components such as an input section 1006 including a keyboard, a mouse, etc., an output section 1007 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, a storage section 1008 including a hard disk, etc., and a communication section 1009 including, for example, a network interface card such as a local area network card and a modem, are connected to the input / output interface 1005. The communication section 1009 executes communication processing via a network such as the Internet. A driver 1010 is also connected to the input / output interface 1005 as necessary. For example, a removable medium 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is installed on the driver 1010 as necessary, and a computer program read from the removable medium 1011 is installed in the storage section 1008 as necessary.

[0118] In particular, in accordance with the embodiments of the present application, the processes described in the flowcharts of the individual methods may be implemented as a computer software program. For example, the embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code used to execute the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication unit 1009 and / or installed from a removable medium 1011. When the computer program is executed by the central processor 1001, it performs various functions defined in the system of the present application.

[0119] As needs to be explained, the computer readable medium shown in the embodiments of the present application may be a computer readable signal medium, or a computer readable storage medium, or any combination of the above. The computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer readable storage medium may include, but are not limited to, an electrical connection having one or more conductors, a portable computer magnetic disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM), a flash memory, an optical fiber, a portable compact magnetic disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium may be any tangible medium that contains or stores a program, which may be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, in which computer-readable program code is carried. Such propagated data signals may take a number of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which may transmit, propagate, or transmit a program used by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wired, or the like, or any suitable combination of the above.

[0120] Those skilled in the art may easily conceive of other implementations of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses, or adaptations of the present application, which modifications, uses, or adaptations follow the general principles of the present application and include common general knowledge or commonly used technical means in the art that are not disclosed herein.

[0121] It should be understood that the present application is not limited to the exact construction described above and illustrated in the drawings, and various modifications and changes can be made without departing from the scope thereof, which is limited only by the appended claims. [Explanation of symbols]

[0122] 110 First terminal device 120 Second Terminal Device 130 Third Terminal Device 140 4th Terminal Device 150 Network 201 Video Sources 202 Point Cloud Data 203 Video Encoding Device 204 Point Cloud Data 205 Streaming Transmission Server 206 Client Terminal Subsystem 208 Client Terminal Subsystem 209 Point Cloud Data 210 Point Cloud Decoding Device 211 Output point cloud data 212 Display 213 Collection Subsystem 800 Point Cloud Decoder 810 Receiver Module 820 Analysis Module 830 Decryption Module 900 point cloud encoder 910 Encoding Module 920 Encapsulation Module 1000 Computer Systems 1001 Central Processor 1002 Dedicated Memory 1003 Random Access Memory 1004 Bus 1005 Output Interface 1006 Input section 1007 Output part 1008 Memory part 1009 Communication section 1010 Driver 1011 Removable media

Claims

1. 1. A computer-implemented method for decoding a point cloud, comprising: receiving a point cloud file transmitted by a data source, the point cloud file including one or more point cloud media tracks having the same point cloud content, and some point cloud media tracks in the point cloud file having different frame rates; analyzing file encapsulation information of the one or more point cloud media tracks to obtain frame rate indication information carried in the file encapsulation information, the frame rate indication information being used to indicate a frame rate of the one or more point cloud media tracks; selecting and decoding a point cloud media track having a specified frame rate from the point cloud file based on frame rate indication information included in the file encapsulation information; The step of analyzing file encapsulation information of the one or more point cloud media tracks to obtain frame rate indication information carried in the file encapsulation information includes: determining a frame rate indication field corresponding to a frame rate of the one or more point cloud media tracks by analyzing file encapsulation information of the one or more point cloud media tracks; determining a frame rate indication for the one or more point cloud media tracks based on a value of the frame rate indication field.

2. The step of receiving a point cloud file transmitted by a data source includes: receiving a stream media signaling sent by a data source, the stream media signaling being used for transmitting point cloud data; Analyzing the stream media signaling to obtain a time domain hierarchical group identifier in the stream media signaling, the time domain hierarchical group identifier being used for identifying a track group, the track group including one or more point cloud media tracks having the same point cloud content, and the track group including some point cloud media tracks having different frame rates; sending a first data transmission request to the data source based on the time domain hierarchical group identifier; The method of claim 1 , further comprising: receiving a point cloud file corresponding to the first data transmission request transmitted by the data source.

3. The step of sending a first data transmission request to the data source based on the time domain hierarchical group identifier includes: obtaining a network bandwidth for data transmission with the data source; selecting one or more target point cloud media tracks from the track group based on the time domain hierarchy group identifier, the target point cloud media tracks having a target frame rate that matches the network bandwidth; and sending a first data transmission request to the data source, the first data transmission request being used to request transmission of the one or more target point cloud media tracks.

4. After the step of selecting and decoding a point cloud media track having a specified frame rate from the point cloud file based on frame rate indication information included in the file encapsulation information, the method further comprises: obtaining a frame rate for a point cloud media track to be displayed; selecting another point cloud media track having the same frame rate as the point cloud media track to be displayed from the point cloud file based on frame rate indication information included in the file encapsulation information; The point cloud decoding method according to any one of claims 1 to 3, further comprising: a step of replacing the point cloud media track to be displayed with the other point cloud media track, thereby decoding and displaying the other point cloud media track.

5. After the step of selecting and decoding a point cloud media track having a specified frame rate from the point cloud file based on frame rate indication information included in the file encapsulation information, the method further comprises: obtaining a frame rate for a point cloud media track to be displayed; selecting, based on frame rate indication information included in the file encapsulation information, one or more other point cloud media tracks from the point cloud file having the same frame rate as the point cloud media track to be displayed; The point cloud decoding method according to any one of claims 1 to 3, further comprising a step of decoding the one or more other point cloud media tracks, and merging and displaying the one or more other point cloud media tracks and the point cloud media track to be displayed.

6. After the step of selecting and decoding a point cloud media track having a specified frame rate from the point cloud file based on frame rate indication information included in the file encapsulation information, the method further comprises: obtaining a frame rate for a point cloud media track to be displayed; receiving a supplemental point cloud file transmitted by the data source by sending a second data transmission request to the data source, the supplemental point cloud file including another point cloud media track having the same point cloud content and the same frame rate as the point cloud media track to be displayed; The point cloud decoding method according to any one of claims 1 to 3, further comprising: a step of replacing the point cloud media track to be displayed with the other point cloud media track, thereby decoding and displaying the other point cloud media track.

7. After the step of selecting and decoding a point cloud media track having a specified frame rate from the point cloud file based on frame rate indication information included in the file encapsulation information, the method further comprises: obtaining a frame rate for a point cloud media track to be displayed; receiving a supplemental point cloud file transmitted by the data source by sending a third data transmission request to the data source, the supplemental point cloud file including one or more other point cloud media tracks having the same point cloud content and the same frame rate as the displayed point cloud media track; The point cloud decoding method according to any one of claims 1 to 3, further comprising a step of decoding the one or more other point cloud media tracks, and merging and displaying the one or more other point cloud media tracks and the point cloud media track to be displayed.

8. 1. A computer-implemented method for encoding a point cloud, comprising: performing encoding processing on the point cloud data to be transmitted according to different encoding standards to obtain a plurality of point cloud code streams having the same point cloud content, in which some point cloud code streams have different frame rates; encapsulating the plurality of point cloud code streams into a plurality of point cloud media tracks, embedding frame rate indication information corresponding to the plurality of point cloud code streams in the plurality of point cloud media tracks, the frame rate indication information being used to indicate a frame rate of the plurality of point cloud media tracks; Embedding the plurality of point cloud codestreams and corresponding frame rate indication information in the plurality of point cloud media tracks includes: determining a frame rate indication field corresponding to the frame rate indication information in File Encapsulation Information of the plurality of point cloud media tracks; and embedding frame rate indication information corresponding to the plurality of point cloud codestreams in a frame rate indication field in the file encapsulation information.

9. After the step of encapsulating the plurality of point cloud code streams into a plurality of point cloud media tracks, the method further comprises: generating a stream media signaling used for transmitting the point cloud data; Embedding a time domain hierarchical group identifier in the stream media signaling, the time domain hierarchical group identifier being used to identify a track group, the track group including one or more point cloud media tracks having the same point cloud content, and including some point cloud media tracks in the track group with different frame rates; The point cloud encoding method of claim 8, further comprising: transmitting the stream media signaling to a data receiving side.

10. After the step of transmitting the stream media signaling to a data recipient, the method further comprises: receiving a data transmission request generated based on the stream media signaling sent by the data sink; 10. The point cloud encoding method of claim 9, further comprising: transmitting a point cloud file to the data receiving side based on the data transmission request, the point cloud file including one or more point cloud media tracks having the same point cloud content, and the point cloud file including some point cloud media tracks having different frame rates.

11. A point cloud decoding device, comprising: a receiving module configured to receive a point cloud file transmitted by a data source, the point cloud file including one or more point cloud media tracks having the same point cloud content, and some point cloud media tracks in the point cloud file having different frame rates; an analysis module configured to analyze file encapsulation information of the one or more point cloud media tracks to obtain frame rate indication information carried in the file encapsulation information, the frame rate indication information being used to indicate a frame rate of the one or more point cloud media tracks; a decoding module configured to select and decode a point cloud media track having a specified frame rate from the point cloud file based on frame rate indication information included in the file encapsulation information; Analysing file encapsulation information of the one or more point cloud media tracks to obtain frame rate indication information carried in the file encapsulation information includes: determining a frame rate indication field corresponding to a frame rate of the one or more point cloud media tracks by analyzing file encapsulation information of the one or more point cloud media tracks; and determining a frame rate indication information for the one or more point cloud media tracks based on a value of the frame rate indication field.

12. A point cloud encoding device, comprising: An encoding module configured to perform encoding processes on point cloud data to be transmitted according to different encoding standards to obtain a plurality of point cloud code streams having the same point cloud content, the plurality of point cloud code streams including some point cloud code streams having different frame rates; an encapsulation module configured to encapsulate the plurality of point cloud code streams into a plurality of point cloud media tracks and embed frame rate indication information corresponding to the plurality of point cloud code streams in the plurality of point cloud media tracks, the frame rate indication information being used to indicate a frame rate of the plurality of point cloud media tracks; Embedding the plurality of point cloud codestreams and corresponding frame rate indication information in the plurality of point cloud media tracks includes: determining a frame rate indication field corresponding to the frame rate indication information in File Encapsulation Information of the plurality of point cloud media tracks; and embedding frame rate indication information corresponding to the plurality of point cloud codestreams in a frame rate indication field in the file encapsulation information.

13. An electronic device, A processor; a memory adapted to store executable instructions for said processor; 8. An electronic device, characterized in that the processor is configured to perform the method according to any one of claims 1 to 7 by executing the executable instructions.

14. An electronic device, comprising: A processor; a memory adapted to store executable instructions for said processor; 11. An electronic device, characterized in that the processor is configured to perform the method according to any one of claims 8 to 10 by executing the executable instructions.

Images