3D point cloud transmission method, reception method, transmission device, reception device, and storage medium

Abstract

The present application provides a method, device and medium for transmitting and receiving a three-dimensional point cloud, the transmission method including the steps of: analyzing point cloud data of an original point cloud and determining a quality level of the spatial domain of the point cloud to be transmitted; selecting point cloud data in the spatial domain of the point cloud to be transmitted based on the quality level of the spatial domain of the point cloud to be transmitted and generating a media code stream; determining quality indication information of the spatial domain of the point cloud to be transmitted based on the selected point cloud data in the spatial domain of the point cloud to be transmitted, writing the quality indication information of the spatial domain of the point cloud to be transmitted to the media code stream; and transmitting the media code stream.

Description

[Technical field]

[0001] This application claims priority to a Chinese patent application filed with the China Patent Office on March 15, 2021 and accepted under application number 202110277373.3, a Chinese patent application filed with the China Patent Office on April 23, 2021 and accepted under application number 202110441806.4, and a Chinese patent application filed with the China Patent Office on June 11, 2021 and accepted under application number 202110653948.7, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the technical field of multimedia, for example, to a transmission method, a reception method, a transmission device, a reception device, and a storage medium for a three-dimensional point cloud. [Background technology]

[0003] With the rapid development of information technology, the consumption model based on text, images and traditional videos is becoming mature, and providing higher reality, higher interactivity and more diversified visual communication has become a hotspot for development. Through technical means such as accurately rendered 3D point clouds, omnidirectional experience with six degrees of freedom, and real-time interaction combining real and virtual reality, visual communication provides users with an immersive experience that is not limited by time, location or real conditions, and also brings infinite space for new applications.

[0004] Visual media data is significantly different from traditional media data in terms of generation, transmission, processing and presentation. Visual media data is more complex and diverse. In order to meet the presentation needs of users, corresponding data description methods have also attracted widespread attention. As 3D scanning technologies and systems are becoming more mature, 3D point cloud data has attracted widespread attention in academia and industry.

[0005] A 3D point cloud is a set of geometry points in space, which records a set of 3D coordinate information for each point on the scanned object surface, as well as multiple attribute information such as texture, material, color, occupancy, normal vector, reflectance intensity, features, etc. 3D point cloud data is a geometric description of real objects and a new 3D model data format. As the main carrier of expression information in visual communication scenes, it can not only effectively express static real objects and scenes in visual media services, but also render accurate three-dimensional models in real time and realistically describe dynamic real object or scene information. Therefore, 3D point cloud data can bring users a real-to-virtual combination, real-time interactive immersive consumption experience.

[0006] At present, a certain amount of research has been conducted on how to store and transmit 3D point cloud data. Currently, when transmitting, the 3D point cloud is usually transmitted with the same quality for multiple spatial regions, and the corresponding data amount varies depending on the quality of the point cloud. If a low-quality point cloud is adopted, it will affect the overall sensing quality. If a high-quality point cloud is adopted, the amount of data that needs to be transmitted will be very large. There are defects in the encapsulation and transmission of 3D point cloud data in related technologies. Summary of the Invention [Means for solving the problem]

[0007] The present application provides a method, a device, a storage medium, and a transmission method for a three-dimensional point cloud.

[0008] According to the method for transmitting a 3D point cloud according to the present application, Analyzing the point cloud data of the source point cloud and determining a quality level of the spatial domain of the point cloud to be transmitted; Selecting point cloud data in the spatial domain of the point cloud to be transmitted based on a quality level of the spatial domain of the point cloud to be transmitted and generating a media code stream; determining quality indication information of the spatial domain of the point cloud to be transmitted based on point cloud data in the spatial domain of the selected point cloud to be transmitted, and writing the quality indication information of the spatial domain of the point cloud to be transmitted into the media codestream; transmitting the media code stream; The source point cloud is composed of point cloud data of different spatial regions and different quality levels, there is at least one spatial region of the point cloud to be transmitted, each spatial region includes point cloud data of at least one quality level, and quality indication information of the spatial region of the point cloud to be transmitted includes a quality level of the spatial region of the point cloud to be transmitted.

[0009] According to the method for receiving a 3D point cloud according to the present application, Analysing from the received media codestream a spatial domain quality indication of the point cloud to be transmitted; decoding the media codestream based on spatial domain quality indication information of the point cloud to be transmitted to obtain point cloud data of the point cloud to be transmitted; rendering point cloud data of the point cloud to be transmitted; The point cloud data is point cloud data of different spatial regions and different quality levels of the point cloud to be transmitted, and the quality indication information of the spatial region includes a quality level of the spatial region.

[0010] According to the 3D point cloud transmission device of the present application, a point cloud analysis module configured to analyze the point cloud data of the source point cloud; a quality level confirmation module configured to determine a quality level of the spatial domain of the point cloud to be transmitted; a point cloud data selection module configured to select point cloud data in the spatial domain of the point cloud to be transmitted based on the quality level of the spatial domain of the point cloud to be transmitted and generate a media code stream; a quality indication information confirmation module configured to determine quality indication information of the spatial domain of the point cloud to be transmitted based on point cloud data in the spatial domain of the selected point cloud to be transmitted; an encapsulation module configured to write a spatial domain quality indication of the point cloud to be transmitted into the media codestream; a transmission module configured to transmit the media code stream; The source point cloud is composed of point cloud data of different spatial regions and different quality levels, there is at least one spatial region of the point cloud to be transmitted, each spatial region includes point cloud data of at least one quality level, and quality indication information of the spatial region of the point cloud to be transmitted includes a quality level of the spatial region of the point cloud to be transmitted.

[0011] According to the 3D point cloud receiving device of the present application, a quality indication information analysis module configured to analyze quality indication information of a spatial domain of a point cloud to be transmitted from the received media codestream; a decoding module configured to decode the media codestream based on a spatial domain quality indication of the point cloud to be transmitted to obtain point cloud data of the point cloud to be transmitted; a rendering module configured to render point cloud data of the point cloud to be transmitted; The point cloud data is point cloud data of different spatial regions and different quality levels of the point cloud to be transmitted, and the quality indication information of the spatial region includes a quality level of the spatial region.

[0012] According to a computer-readable storage medium of the present application, a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the above transmission method and / or the above receiving method are realized. [Brief description of the drawings]

[0013] [Figure 1] 1 is a flowchart of a 3D point cloud transmission method according to an embodiment of the present application; [Diagram 2] 2 is a flowchart of a method for receiving a 3D point cloud according to an embodiment of the present application; [Diagram 3] 1 is a flowchart of a method for encapsulating and consuming a 3D point cloud according to an embodiment of the present application. [Figure 4] 1 is a flowchart of a method for describing a 3D point cloud according to an embodiment of the present application. [Diagram 5] FIG. 2 is a schematic diagram of quality level settings in different spatial regions of 3D point cloud data according to an embodiment of the present application. [Figure 6] 1 is a structural schematic diagram of a 3D point cloud transmission device according to an embodiment of the present application; [Figure 7] 1 is a structural schematic diagram of a 3D point cloud receiving device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] FIG. 1 shows a flow chart of a point cloud transmission method, which can be applied to the server side. Here, the original point cloud is composed of point cloud data of different spatial regions and different quality levels, each spatial region includes point cloud data of at least one quality level, and the quality indication information of the spatial region includes information of the quality level of the spatial region. After analyzing the point cloud data of the original point cloud, it can be determined which quality level point cloud data exists in each spatial region of the original point cloud. The partition of the spatial region is related to the inherent attribute of the point cloud data included in the original point cloud, so that the partition of the spatial region of the original point cloud and the point cloud to be transmitted coincide with each other. According to the user's request information (e.g., user position, viewing angle, etc.), a related spatial region for transmission can be selected from the multiple spatial regions of the original point cloud as the spatial region of the point cloud to be transmitted, the quality level of the spatial region of the point cloud to be transmitted can be confirmed, and the point cloud data of the corresponding quality level can be selected from the spatial region of the point cloud to be transmitted as the point cloud data of the spatial region of the point cloud to be transmitted.

[0015] A method for transmitting a 3D point cloud according to an embodiment of the present application includes the following steps.

[0016] In S110, the point cloud data of the source point cloud is analyzed to determine the quality level of the spatial domain of the point cloud to be transmitted, which is performed by a point cloud analysis module and a quality level confirmation module.

[0017] In step S120, according to a quality level of the spatial domain of the point cloud to be transmitted, point cloud data in the spatial domain of the point cloud to be transmitted is selected to generate a media codestream, this step being performed by a point cloud data selection module.

[0018] In S130, determine quality indication information of the spatial domain of the point cloud to be transmitted according to the point cloud data in the spatial domain of the selected point cloud to be transmitted, and write the quality indication information of the spatial domain of the point cloud to be transmitted into the media codestream, this step being performed by a quality indication information checking module and an encapsulation module.

[0019] In S140, the media code stream is transmitted, this step being performed in a transmission module.

[0020] In one embodiment, each of the spatial regions includes at least one component track group, each component track group includes at least one component track, each of the component tracks includes at least one type of point cloud data, and each type of point cloud data includes at least one quality level, and determining the quality level of the spatial region of the point cloud to be transmitted includes: determining a quality level of a component track of each component track group of each spatial region of the point cloud to be transmitted; determining a joint quality level for each component track group in each spatial region of the point cloud to be transmitted; The joint quality level of each of the component track groups includes a quality level of each component track included in each of the component track groups, and the quality level of each component track is determined based on a quality level of at least one type of point cloud data included in each component track.

[0021] In one embodiment, after determining the spatial domain quality indication of the point cloud to be transmitted, determining an indication attribute identification of the point cloud to be transmitted based on a spatial domain quality indication of the point cloud to be transmitted; and writing an indication attribute identification of the point cloud to be transmitted into the media codestream; The indication attribute identification of the point cloud to be transmitted is used to indicate at least one of whether the point cloud to be transmitted is composed of different spatial regions, point cloud data of different quality levels, and the spatial regions included in the point cloud to be transmitted.

[0022] In one embodiment, the step of selecting point cloud data in the spatial domain of the point cloud to be transmitted based on a quality level of the spatial domain of the point cloud to be transmitted and generating a media code stream includes: The method includes selecting point cloud data of component tracks included in each component track group in the spatial domain of the point cloud to be transmitted based on a joint quality level of each component track group in the spatial domain of the point cloud to be transmitted, forming point cloud data of each component track group in the spatial domain of the point cloud to be transmitted, and generating a media code stream.

[0023] In one embodiment, the spatial domain quality indication information of the point cloud to be transmitted is Further including at least one of: number of spatial regions, identification of the spatial region, position of the spatial region, dimension of the spatial region, size of the spatial region, number of component tracks of the spatial region, identification of the component tracks of the spatial region, type of component track group of the spatial region, number of component track groups of the spatial region, identification of the component track group of the spatial region.

[0024] In one embodiment, the quality indication of the spatial domain of the point cloud to be transmitted is determined based on point cloud data selected from a corresponding spatial domain of a source point cloud.

[0025] 2 is a flow chart of a 3D point cloud receiving method according to an embodiment of the present application, which can be applied to a user side, where the point cloud data are point cloud data of different spatial domains and different quality levels of the point cloud to be transmitted, and the quality indication information of the spatial domain includes a quality level of the spatial domain. The method includes the following steps:

[0026] In step S210, quality indication information of the spatial domain of the point cloud to be transmitted is analyzed from the received media codestream, this step being performed in a quality indication information analysis module.

[0027] In S220, according to the spatial domain quality indication information of the point cloud to be transmitted, the media codestream is decoded to obtain point cloud data of the point cloud to be transmitted, this step being performed in a decoding module.

[0028] In step S230, the point cloud data of the point cloud to be transmitted is rendered, which is performed in a rendering module.

[0029] In one embodiment, before analyzing the spatial domain quality indication of the point cloud to be transmitted from the received media codestream, The method further includes analyzing an instruction attribute identification of the point cloud to be transmitted from the received media code stream, and determining whether the point cloud to be transmitted is composed of point cloud data of different spatial domains based on the instruction attribute identification of the point cloud to be transmitted.

[0030] In one embodiment, according to a determination result that the received transmission point cloud is composed of point cloud data of different spatial domains, analyzing spatial domain quality indication information of the point cloud to be transmitted, and decoding the media code stream according to indication attribute identification of the point cloud to be transmitted and / or the spatial domain quality indication information; Based on the determination result that the received transmission point cloud is composed of point cloud data of one spatial domain, the media code stream is decoded based on the indication attribute identification of the point cloud to be transmitted.

[0031] FIG. 6 is a structural schematic diagram of a 3D point cloud transmission device 600 according to an embodiment of the present application. The transmission device 600 includes: a point cloud analysis module 610 configured to analyze the point cloud data of the source point cloud; a quality level confirmation module 620 configured to determine a quality level of the spatial domain of the point cloud to be transmitted; a point cloud data selection module 630 configured to select point cloud data in the spatial domain of the point cloud to be transmitted based on the quality level of the spatial domain of the point cloud to be transmitted and generate a media codestream; a quality indication information checking module 640 configured to determine a quality indication information of the spatial domain of the point cloud to be transmitted; an encapsulation module 650 configured to write a spatial domain quality indication of the point cloud to be transmitted into the media codestream; a transmission module 660 configured to transmit the media code stream; The source point cloud is composed of point cloud data of different spatial regions and different quality levels, there is at least one spatial region of the point cloud to be transmitted, each spatial region includes point cloud data of at least one quality level, and quality indication information of the spatial region of the point cloud to be transmitted includes a quality level of the spatial region of the point cloud to be transmitted.

[0032] In an embodiment, the transmission device comprises an attribute description module configured to determine an indication attribute identification of the point cloud to be transmitted based on a spatial domain quality indication of the point cloud to be transmitted; and the encapsulation module 660 further configured to write an indicative attribute identification of the point cloud to be transmitted into the media codestream.

[0033] FIG. 7 is a structural schematic diagram of a 3D point cloud receiving device 700 according to an embodiment of the present application. The receiving device 700 includes: a quality level indication module 710 configured to analyze from the received media codestream a quality indication of the spatial domain of the point cloud to be transmitted; a decoding module 720 configured to decode the media codestream based on a spatial domain quality indication of the point cloud to be transmitted to obtain point cloud data of the point cloud to be transmitted; a rendering module 730 configured to render point cloud data of the point cloud to be transmitted; The point cloud data is point cloud data of different spatial regions and different quality levels of the point cloud to be transmitted, and the quality indication information of the spatial region includes a quality level of the spatial region.

[0034] In one embodiment, the receiving device 700 further includes an identification analysis module configured to analyze an indication attribute identification of the point cloud to be transmitted from the received media code stream, and determine whether the point cloud to be transmitted is composed of point cloud data of different spatial domains based on the indication attribute identification of the point cloud to be transmitted.

[0035] An embodiment of the present application provides a computer-readable storage medium having a computer program stored therein, the computer program being executed by a processor to realize the transmission method and / or the receiving method described in the above embodiment. The computer-readable storage medium may be a non-transitory computer-readable storage medium.

[0036] 3 is a flow chart of a 3D point cloud encapsulation method and consumption according to an embodiment of the present application. The encapsulation method and consumption process includes the following steps:

[0037] In S310, the transmitting side determines quality indication information of the spatial domain of the point cloud to be transmitted.

[0038] In S320, the transmitting side writes spatial domain quality indication information into a media codestream generated by the point cloud data of the selected point cloud to be transmitted.

[0039] At S330, the transmitting side transmits the media codestream to the receiving side according to the spatial domain quality indication information.

[0040] At S340, the receiver reads, parses, and renders the media code stream.

[0041] 4 is a flowchart of a 3D point cloud description method according to an embodiment of the present application. The description method includes the following steps:

[0042] In S410, the server side analyzes the point cloud data of the source point cloud.

[0043] In S420, the server side determines the pointing attribute identification of the point cloud to be transmitted based on the analysis result, and writes the pointing attribute identification into the media codestream.

[0044] At S430, the server side sends the media code stream to the user side.

[0045] The 3D point cloud transmission method according to one embodiment of the present application mainly includes a point cloud analysis step, a spatial domain quality level determination step, a point cloud data selection step, a quality indication information determination step, an attribute description step, an encapsulation step, and a transmission step.

[0046] First, the point cloud analysis step is performed in the point cloud analysis module.

[0047] The point cloud data of the source point cloud is analyzed, and one source point cloud may be composed of several spatial regions or only one spatial region, and each spatial region may have point cloud data of several quality levels or only one quality level.

[0048] Second, a spatial domain quality level determination step is performed in the point cloud analysis module.

[0049] This step is performed when one source point cloud is composed of point cloud data of different spatial regions and different quality levels. The number of source point clouds may be one or more, and each source point cloud is composed of point cloud data of different quality levels. As shown in FIG. 5, the point cloud data of different spatial regions are divided into three quality levels, namely high quality, medium quality and low quality. The number of quality levels of point cloud data is set according to specific needs and is not limited to the above three. FIG. 5 only shows one exemplary situation, where the quality levels of the two spatial regions in the upper layer are high quality, the quality levels of the two spatial regions in the middle layer are medium quality, and the quality levels of the two spatial regions in the lower layer are low quality. In other embodiments, the quality levels of the spatial regions of each layer may be different.

[0050] Point cloud data can be divided into multiple types, such as geometry, occupancy, feature, attribute, auxiliary, color, texture, material, normal vector, reflection intensity, etc. One component track is composed of one or more types of point cloud data, and in one spatial domain, multiple component track groups can be formed by combinations of different component tracks, and one component track group usually corresponds to one reproducible data stream in the spatial domain. At the transmitting side (sending side), it is usually only necessary to send one component track group to the receiving side for rendering playback, and different component tracks in the spatial domain can be formed by selecting point cloud data with different quality levels, and the component track group for transmission in the spatial domain is updated or replaced in real time. Therefore, the premise of selecting point cloud data is to first determine the quality level of the spatial domain, and the key point of determining the quality level of the spatial domain is to determine the joint quality level of the component track group in the spatial domain, and the joint quality level of the component track group in the spatial domain includes the quality level of each component track constituting the component track group.

[0051] Determining the spatial domain quality level includes determining a quality level of each component track of a component track group and determining a joint quality level of the component track group in the spatial domain, where the two sub-steps are in no particular order, and once one of the quality level of each component track and the joint quality level of the component track group is determined, the other is also determined.

[0052] If there is only one source point group, a quality level is determined for each spatial region of the source point group. If there are multiple source point groups, a quality level is determined for each spatial region of the source point group.

[0053] In one exemplary embodiment, the user moves his / her head, and the user's position and the user's viewing angle change constantly. The user side or the server side determines the quality level of each spatial region based on the information such as the user position and the viewing angle, and selects the point cloud data. The user feedback information may include request information for indicating the user position and / or the viewing angle.

[0054] In one exemplary embodiment, when checking the quality level of each spatial region, the spatial region corresponding to the viewing angle center position is set to a high quality level, and the spatial region corresponding to the viewing angle edge position is set to a low quality level. For example, for a three-dimensional point cloud model of a person, the user's viewing angle is constantly changing, and when the user's viewing angle is concentrated on the person's upper body region such as the head, the person corresponding region can be set to a high quality region, and the remaining region can be set to a medium or low quality region. Similarly, when the user's viewing angle is concentrated on the person's lower body region such as the shoes, the person corresponding region can be set to a high quality region, and the remaining region can be set to a medium or low quality region.

[0055] In an exemplary embodiment, when checking the quality level of each spatial region, the spatial region corresponding to the position where the user pays attention is set to a high quality level, and the remaining spatial regions are set to a low quality level. For example, for a 3D point cloud model of a large scene such as a virtual reality museum, a user can interact with the surrounding 3D point cloud scene and nearby 3D point cloud objects in an exhibition area, and the exhibition area where the user pays attention is constantly changing with the viewing angle of the user, at this time the exhibition area where the user pays attention is set to a high quality region, and the remaining regions are set to a medium or low quality region. The user feedback information may include request information for indicating the position and / or information where the user pays attention.

[0056] Third, a point cloud data selection step is performed in a point cloud data selection module.

[0057] Based on the joint quality level of the spatial domain component track groups and / or the quality level of each component track, point cloud data of each component track is selected to form point cloud data of the spatial domain component track groups, and a media code stream is generated.

[0058] In one exemplary embodiment, each source point cloud is divided into different spatial regions, and point cloud data of component track groups of the spatial regions is selected based on the joint quality level of the component track groups of the spatial regions and / or the quality level of each component track, and a media code stream is generated.

[0059] In an exemplary embodiment, each source point cloud is pre-divided into a plurality of spatial regions, and the component track groups in each spatial region are respectively encoded according to different joint quality levels. For example, if the point cloud data is divided into three quality levels, in the spatial region of each source point cloud, the component tracks respectively select high quality, medium quality and low quality point cloud data. According to different combinations of each component track, the component track groups in the spatial region have a plurality of different joint quality levels, and different component track groups are pre-encoded respectively. After determining the joint quality levels of the component track groups in the spatial region, the media code streams of the pre-encoded component track groups can be directly selected.

[0060] When only the media code stream of one component track group in one spatial domain needs to be transmitted, the media code stream of the component track group is the media code stream of the spatial domain.When the media code streams of multiple component track groups in one spatial domain need to be transmitted, the media code streams of the multiple component track groups are the media code streams of the spatial domain.

[0061] When there is only one source point group, the media code streams of each spatial region of the source point group are combined to form the media code stream of the source point group.When there are multiple source point groups, the media code streams of each spatial region of each source point group are combined to form the media code streams of multiple source point groups.

[0062] 4. The quality indication information determination step performed by the quality indication information confirmation module. After the point cloud data is selected, it is necessary to generate quality indication information of the spatial domain to indicate the selected point cloud data. In addition to the quality level information (joint quality level of the component track group of the spatial domain, quality level of the component track), the quality indication information further includes any one or more of the following: the number of spatial domains, the identification of the spatial domain, the position of the spatial domain, the dimension of the spatial domain, the size of the spatial domain, the number of component tracks of the spatial domain, the identification of the component track of the spatial domain, the type of the component track group of the spatial domain, the number of component track groups of the spatial domain, the identification of the component track group of the spatial domain, and an identifier of whether the spatial domain dimension is identified.

[0063] If there is only one source point cloud, determine quality indication information for each spatial region of the source point cloud. If there are multiple source point clouds, determine quality indication information for each spatial region of each source point cloud respectively.

[0064] In an exemplary embodiment, when dividing one source point cloud into different spatial regions, the number of spatial regions is determined. The number of spatial regions of one source point cloud may be the same as or different from the number of quality levels of point cloud data included in the source point cloud. In the former case, each spatial region has a different quality level. In the latter case, multiple spatial regions may be confirmed to have the same quality level.

[0065] In one exemplary embodiment, when dividing a source point cloud into different spatial regions, the positions of the spatial regions are determined, which may be represented by three-dimensional coordinates.

[0066] In an exemplary embodiment, the dimension of the spatial domain is determined when dividing a source point cloud into different spatial domains, where a spatial domain unit may be a point cloud data, or a rectangular parallelepiped, cylindrical or spherical spatial domain.

[0067] In one exemplary embodiment, when dividing a source point cloud into different spatial regions, the sizes of the spatial regions are determined, and each spatial region may be arranged to have the same size or different sizes.

[0068] In one exemplary embodiment, the size of the spatial region is determined when dividing a source point cloud into different spatial regions, and the spatial regions may be arranged to overlap or not overlap each other.

[0069] The associated fields or characters added may be as follows: region_quality_ranking: indicates the quality level of a single spatial region in the point cloud. Component_track_quality_ranking: Indicates the joint quality level of all component tracks within a component track group of one spatial region in the point cloud. num_regions indicates the number of 3D spatial regions in the point cloud. num_packed_track indicates the number of component tracks in the spatial region. packed_track_id indicates the component track identification of the spatial region. num_track_groups Indicates the number of component track groups in the spatial region. component_track_group_id Indicates the identification of the component track group of the spatial domain. 3DspatialRegionStruct: Indicates a point of a single source point cloud or a spatial region, and contains the x, y, z coordinates of the anchor point and cuboid_dx, cuboid_dy, cuboid_dz ​​indicating the region extents. The included fields are: dimensions_included_flag: an identifier indicating whether one spatial region dimension has already been identified and the dimensions of the spatial region are the same as the dimensions of the first 3DSpatialRegionStruct instance with the same 3d_region_id (e.g. in the previous sample or in V3CSpatialRegionsBox), and if the dimensions of the spatial region are already explicitly specified in at least one 3DSpatialRegionStruct instance, then dimensions_included_flag should be 0. 3d_region_id:Identifier of the spatial region. anchor: Specifies a single 3D point as the 3D spatial region anchor in the Cartesian coordinate system, specified by a 3DPoint(). The fields contained in 3DPoint() are: x, z, y: Specifies the x, z, y coordinate values ​​of a single 3D point in the Cartesian coordinate system. CuboidRegionStruct(): Specifies a rectangular region in the Cartesian coordinate system, and contains the following fields: bb_dx, bb_dy, bb_dz: A 3D region in the Cartesian coordinate system extending in the x, y, and z axes relative to the anchor point.

[0070] For convenience of explanation, the above set of instruction information description is referred to, but other information may be used when the example is implemented.

[0071] Exemplarily, one implementation is as follows: aligned(8) class V3CSpatialRegionsBox extends FullBox('vpsr', 0 ,0) { bit(1) all_tiles_in_single_track_flag; bit(7) reserved = 0; unsigned int(16) num_regions; for (i=0; i < num_regions; i++) { unsigned int(8) region_quality_ranking; 3dSpatialRegionStruct(1); if (all_tiles_in_single_track_flag) { unsigned int(8) num_track_groups; for (j=0; j < num_track_groups; j++) { unsigned int(8) component_quality_ranking; unsigned int(32) component_track_group_id; unsigned int(8) num_tiles; for (k=0; k < num_tiles; k++) { unsigned int(16) tile_id; } } } else { unsigned int(8) num_tile_tracks; for (j=0; j < num_tile_tracks; j++) { unsigned int(32) tile_track_id; } } }

[0072] Exemplarily, another implementation is as follows: aligned(8) class V3CSpatialRegionsBox extends FullBox('vpsr',0,0) { unsigned int(16) num_regions; for (i=0; i < num_regions; i++) { 3DSpatialRegionStruct(1); unsigned int(8) region_quality_ranking; unsigned int(8) num_track_groups; for (j=0; j < num_track_groups; j++) { unsigned int(32) track_group_id; unsigned int(8) component_quality_ranking; unsigned int(16) nal_group_id; } unsigned int(8) num_objects; unsigned int(8) obj_idx_length; for (j=0; j < num_objects; j++) unsigned int(obj_idx_length * 8) soi_object_idx; } } }

[0073] When the examples of this application are implemented, only one of the region_quality_ranking and component_quality_ranking may be used, or both may be used.

[0074] When the examples of the present application are implemented, one or more of the above exemplary embodiments may be selected and implemented in combination.

[0075] 5. An attribute description step performed in an attribute description module.

[0076] After the point cloud data is selected, an indication attribute identification needs to be generated to indicate whether the selected point cloud is composed of point cloud data of different spatial regions and different quality levels, where the different quality levels include one or more combinations of: (1) different types of point cloud data included in one component track have different quality levels; (2) point cloud data of component tracks included in one component track group have different quality levels; and (3) multiple component track groups in one spatial region have different joint quality levels.

[0077] In one exemplary embodiment, when the selected point cloud to be transmitted is composed of point cloud data of different spatial domains, regardless of whether the selected point cloud to be transmitted is composed of point cloud data of different quality levels, one value is assigned to the instruction attribute identification of the selected point cloud to be transmitted, and when the selected point cloud to be transmitted is composed of point cloud data of only one spatial domain, another value is assigned to the instruction attribute identification of the selected point cloud to be transmitted.

[0078] In one exemplary embodiment, the point cloud indication attribute identification for point cloud data of different spatial regions and different quality levels is v3c_srqr. One or more fields may be appropriately added to the transmission signaling according to needs, such as: V3C_information_descriptor(): Represents the descriptor of the point cloud content. descriptor_tag: Represents the type of the descriptor. descriptor_length: Specifies the byte length from the next byte of the field to the last byte of the descriptor. v3c_srpr_flag: indicates the existence of point clouds with different quality in different regions. V3CspatialRegionsBox: indicates spatial regions in a point cloud. Specifically, it is as shown in the table below. [Table 1]

[0079] If one source point cloud is composed of point cloud data in different spatial regions and different quality levels, v3c_srpr_flag is set to 1 and each spatial region is specified by V3CSpatialRegionBox. If one source point cloud is composed of point cloud data in different spatial regions and one quality level, v3c_srpr_flag is set to 1 and each spatial region is specified by V3CSpatialRegionBox. Otherwise, v3c_srpr_flag is set to 0.

[0080] A point group to be transmitted corresponds to a point attribute identification, and the point attribute identification of the point group to be transmitted is written into a media codestream formed by the point group to be transmitted.

[0081] Sixth, an encapsulation step performed in an encapsulation module.

[0082] A quality indication and / or an indication attribute identification of the spatial domain of the point cloud to be transmitted is written into a media codestream formed by point cloud data selected from the spatial domain of the point cloud to be transmitted.

[0083] The quality indication information of the spatial domain of the point cloud to be transmitted is written into the media code stream corresponding to the spatial domain of the point cloud to be transmitted, and the media code streams corresponding to each spatial domain of one point cloud to be transmitted are arranged together to form the media code stream of the point cloud to be transmitted, whereby the indication attribute identification can be written into the media code stream of the point cloud, or can be unified in the quality indication information of the spatial domain.

[0084] If the determined related point group is only one transmission target point group, write the spatial domain quality indication information and / or indication attribute identification into the media codestream formed by the selected point group data from the corresponding spatial domain. If the determined related point group is multiple transmission target point groups, perform the above encapsulation step for each transmission target point group respectively.

[0085] Seventh, a transmission step performed in a transmission module.

[0086] The media code stream is transmitted to a receiving side.

[0087] Corresponding to the transmission method, the receiving side executes a 3D point cloud receiving method, which includes the following steps: Analyzing the point cloud indication attribute identification from the received media code stream to determine whether the received point cloud is composed of point cloud data of different spatial domains, this step being performed in an identification analysis module. If the judgment result is "yes", analyze the quality indication information of each spatial region of the point cloud to be transmitted, and this step is performed in a quality indication information analysis module. Combined with the point cloud indication attribute identification and / or quality indication information of each spatial region, the media code stream is decoded to obtain the point cloud data, which step is performed in a decoding module. The decoded point cloud data is rendered and presented; this step is performed in the rendering module.

[0088] The user side decapsulates the data code stream, and when there is an overlap between different spatial domains of the received point clouds, selects and renders high quality point cloud data for the overlap based on the quality indication information of the spatial domain.

[0089] Because the human eye is only sensitive to information within a certain angle within the visual field, when a user consumes a point cloud media file, the degradation of the point cloud quality in a low-interest area does not affect the overall perceived quality. Usually, the data amount corresponding to point clouds of different qualities is different, and the data amount corresponding to point clouds of high quality is also larger. Therefore, by extending the encapsulation information of the 3D point cloud media data, different quality level information is assigned to different areas of the 3D point cloud, and point cloud data of different qualities is obtained in the encoding process according to the quality level information, so as to achieve the purpose of transmitting point cloud data of different qualities in different areas, and finally, the data amount in the transmission process can be effectively reduced, and bandwidth resources in the transmission process can be saved.

[0090] Those skilled in the art will appreciate that, in addition to implementing the system and its multiple devices, modules, and units provided by the present application in a pure computer-readable program code manner, the system and its multiple devices, modules, and units provided by the present application can also be implemented with logic programming for method steps to realize the same functions in the form of logic gates, switches, dedicated integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Therefore, the system and its multiple devices, modules, and units provided by the present application may be regarded as hardware components, and the devices, modules, and units for implementing various functions contained therein may be regarded as structures within the hardware components, and the devices, modules, and units for implementing multiple functions may be regarded as software modules for implementing a method or as structures within the hardware components.

[0091] The technical solution of the present application extends the encapsulation information of 3D point cloud media data, and adds point cloud indication attribute identification and quality indication information of each spatial domain to the signaling, and the user side can select and render high-quality point cloud data for overlapping spatial domains according to the quality indication information of the spatial domain. The method and device of the present application can reduce the amount of point cloud data in the transmission process.

Claims

1. Analyzing the point cloud data of the source point cloud and determining a quality level of the spatial domain of the point cloud to be transmitted; Selecting point cloud data in the spatial domain of the point cloud to be transmitted based on a quality level of the spatial domain of the point cloud to be transmitted and generating a media code stream; determining quality indication information of the spatial domain of the point cloud to be transmitted based on point cloud data in the spatial domain of the selected point cloud to be transmitted, and writing the quality indication information of the spatial domain of the point cloud to be transmitted into the media codestream; transmitting the media code stream; A method for transmitting a three-dimensional point cloud, wherein the original point cloud is composed of point cloud data of different spatial regions and different quality levels, the spatial region of the point cloud to be transmitted is at least one, each spatial region includes point cloud data of at least one quality level, and quality indication information of the spatial region of the point cloud to be transmitted includes a quality level of the spatial region of the point cloud to be transmitted.

2. Each of the spatial regions includes at least one component track group, each component track group includes at least one component track, each of the component tracks includes at least one type of point cloud data, each type of point cloud data includes at least one quality level, and determining the quality level of the spatial region of the point cloud to be transmitted includes: determining a quality level of a component track of each component track group of each spatial region of the point cloud to be transmitted; determining a joint quality level for each component track group in each spatial region of the point cloud to be transmitted; 2. The method of transmitting a three-dimensional point cloud as described in claim 1, wherein the joint quality level of each component track group is determined by the quality level of each component track contained in each component track group, and the quality level of each component track is determined based on the quality level of at least one type of point cloud data contained in each component track.

3. After determining a quality indication of the spatial domain of the point cloud to be transmitted, determining an indication attribute identification of the point cloud to be transmitted based on a spatial domain quality indication of the point cloud to be transmitted; and writing an indication attribute identification of the point cloud to be transmitted into the media codestream; 2. The method for transmitting a three-dimensional point cloud according to claim 1, wherein the indication attribute identification of the point cloud to be transmitted is used to indicate at least one of different spatial regions, whether the point cloud to be transmitted is composed of point cloud data of different quality levels, and spatial regions included in the point cloud to be transmitted.

4. Selecting point cloud data in the spatial domain of the point cloud to be transmitted based on a quality level of the spatial domain of the point cloud to be transmitted and generating a media code stream, The method for transmitting a three-dimensional point cloud according to claim 2, further comprising: selecting point cloud data of component tracks included in each component track group in the spatial domain of the point cloud to be transmitted based on a joint quality level of each component track group in the spatial domain of the point cloud to be transmitted; forming point cloud data of each component track group in the spatial domain of the point cloud to be transmitted; and generating a media code stream.

5. The quality indication information of the spatial domain of the point cloud to be transmitted is 2. The method of claim 1, further comprising at least one of: a number of spatial regions; an identification of the spatial region; a position of the spatial region; a dimension of the spatial region; a size of the spatial region; a number of component tracks of the spatial region; an identification of the component tracks of the spatial region; a type of component track group of the spatial region; a number of component track groups of the spatial region; and an identification of the component track group of the spatial region.

6. Analysing from the received media codestream a spatial domain quality indication of the point cloud to be transmitted; Decoding the received media code stream based on quality indication information of the spatial domain of the point cloud to be transmitted, to obtain point cloud data of the spatial domain of the point cloud to be transmitted; rendering point cloud data of a spatial domain of the point cloud to be transmitted; a quality indication information for the spatial domain of the point cloud to be transmitted based on the quality level of the spatial domain of the point cloud to be transmitted; a media code stream is generated by selecting point cloud data in the spatial domain of the point cloud to be transmitted based on the quality level of the spatial domain of the point cloud to be transmitted; a quality indication information for the spatial domain of the point cloud to be transmitted based on the selected point cloud data in the spatial domain of the point cloud to be transmitted; the quality indication information for the spatial domain of the point cloud to be transmitted is written into the generated media code stream; and the media code stream is transmitted, wherein the original point cloud is composed of point cloud data in different spatial domains and different quality levels, the spatial domain of the point cloud to be transmitted is at least one, each spatial domain includes point cloud data of at least one quality level, and the quality indication information for the spatial domain of the point cloud to be transmitted includes the quality level of the spatial domain of the point cloud to be transmitted.

7. before analyzing the spatial domain quality indication of the point cloud to be transmitted from the received media codestream, The method for receiving a three-dimensional point cloud as described in claim 6, further comprising: analyzing an instruction attribute identification of the point cloud to be transmitted from the received media code stream; and determining whether the point cloud to be transmitted is composed of point cloud data of different spatial domains based on the instruction attribute identification of the point cloud to be transmitted.

8. a point cloud analysis module configured to analyze the point cloud data of the source point cloud; a quality level verification module configured to determine a quality level of the spatial domain of the point cloud to be transmitted; a point cloud data selection module configured to select point cloud data in the spatial domain of the point cloud to be transmitted based on the quality level of the spatial domain of the point cloud to be transmitted and generate a media code stream; a quality indication information confirmation module configured to determine quality indication information of the spatial domain of the point cloud to be transmitted based on point cloud data in the spatial domain of the selected point cloud to be transmitted; an encapsulation module configured to write a spatial domain quality indication of the point cloud to be transmitted into the media codestream; a transmission module configured to transmit the media code stream; A three-dimensional point cloud transmission device, wherein the original point cloud is composed of point cloud data of different spatial regions and different quality levels, the spatial region of the point cloud to be transmitted is at least one, each spatial region includes point cloud data of at least one quality level, and quality indication information of the spatial region of the point cloud to be transmitted includes a quality level of the spatial region of the point cloud to be transmitted.

9. an attribute description module configured to determine an indication attribute identification of the point cloud to be transmitted based on a quality indication of the spatial domain of the point cloud to be transmitted; The apparatus of claim 8 , further comprising: the encapsulation module further configured to write an indicative attribute identification of the point cloud to be transmitted into the media codestream.

10. a quality indication information analysis module configured to analyze quality indication information of a spatial domain of a point cloud to be transmitted from the received media codestream; a decoding module configured to decode the received media code stream based on a quality indication of the spatial domain of the point cloud to be transmitted, to obtain point cloud data of the spatial domain of the point cloud to be transmitted; a rendering module configured to render point cloud data of a spatial domain of the point cloud to be transmitted; a quality indication information for the spatial domain of the point cloud to be transmitted based on the quality level of the spatial domain of the point cloud to be transmitted, a quality indication information for the spatial domain of the point cloud to be transmitted based on the selected point cloud data in the spatial domain of the point cloud to be transmitted to generate a media code stream, a quality indication information for the spatial domain of the point cloud to be transmitted based on the selected point cloud data in the spatial domain of the point cloud to be transmitted, a quality indication information for the spatial domain of the point cloud to be transmitted based on the selected point cloud data in the spatial domain of the point cloud to be transmitted, a quality indication information for the spatial domain of the point cloud to be transmitted based on the selected point cloud data in the spatial domain of the point cloud to be transmitted, a quality indication information for the spatial domain of the point cloud to be transmitted based on the selected point cloud data in the spatial domain of the point cloud to be transmitted, a quality indication information for the spatial domain of the point cloud to be transmitted based on the selected point cloud data in the spatial domain of the point cloud to be transmitted,

11. The three-dimensional point cloud receiving device of claim 10, further comprising an identification analysis module configured to analyze an instruction attribute identification of the point cloud to be transmitted from the received media code stream and determine whether the point cloud to be transmitted is composed of point cloud data of different spatial domains based on the instruction attribute identification of the point cloud to be transmitted.

12. 1. A computer-readable storage medium, comprising: A computer-readable storage medium having a computer program stored therein, the computer program being executed by a processor to realize the three-dimensional point cloud transmission method according to any one of claims 1 to 5 or the three-dimensional point cloud reception method according to any one of claims 6 to 7. A computer-readable storage medium.

Images