A data processing method, device, equipment and storage medium for point cloud media

Abstract

The embodiments of the present application disclose a data processing method, apparatus, device, and storage medium for point cloud media. The data processing method for point cloud media includes: obtaining the point cloud object indication information of the i-th sample group of the point cloud media, where the point cloud media includes N sample groups, and the i-th sample group is any one of the N sample groups; the i-th sample group includes point cloud objects, and the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group, where N and i are both positive integers and i ∈ [1, N]; parsing the point cloud media according to the point cloud object indication information of the i-th sample group. By using the embodiments of the present application, by indicating the point cloud objects included in the point cloud media, the parsing and processing efficiency of the point cloud media can be improved to a certain extent.

Description

Technical Field

[0001] This application relates to the field of computer technologies, in particular to the field of point cloud media technologies, and specifically relates to a method for processing data of point cloud media, an apparatus for processing data of point cloud media, a device for processing data of point cloud media, and a computer-readable storage medium. Background Art

[0002] With the continuous development of science and technology, it is currently possible to obtain a large amount of high-precision point cloud data at a relatively low cost and within a relatively short time period. The point cloud data is often transmitted between content production devices and content consumption devices in the form of point cloud media.

[0003] The transmission process of point cloud media is as follows: after encoding the point cloud media, the content production device encapsulates the encoded point cloud media to obtain an encapsulated file of the point cloud media, and the content production device transmits the encapsulated file of the point cloud media to the content consumption device; the content consumption device unpacks the encapsulated file of the point cloud media transmitted by the content production device, then decodes it, and finally the content consumption device presents the media file. Since the amount of point cloud data contained in the point cloud media is large, how to improve the parsing and processing efficiency of the point cloud media, so as to bring a better experience for the consumption of the point cloud media, is a problem that the industry has been continuously solving. Summary of the Invention

[0004] Embodiments of this application provide a method, apparatus, device, and storage medium for processing data of point cloud media. By indicating the point cloud objects included in the point cloud media, the parsing and processing efficiency of the point cloud media can be improved to a certain extent.

[0005] On the one hand, embodiments of this application provide a method for processing data of point cloud media, and the method for processing data of point cloud media includes:

[0006] Obtain the point cloud object indication information of the i-th sample group of the point cloud media. The point cloud media includes N sample groups, and the i-th sample group is any one of the N sample groups; the i-th sample group includes point cloud objects, and the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group. N and i are both positive integers and i ∈ [1, N];

[0007] Parse the point cloud media according to the point cloud object indication information of the i-th sample group.

[0008] On the other hand, embodiments of this application provide a method for processing data of point cloud media, and the method for processing data of point cloud media includes:

[0009] Generate the point cloud object indication information of the i-th sample group of the point cloud media. The point cloud media includes N sample groups, and the i-th sample group is any one of the N sample groups. The i-th sample group includes point cloud objects, and the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group. Both N and i are positive integers and i ∈ [1, N].

[0010] Transmit the point cloud object indication information of the i-th sample group to the content consumption device, so that the content consumption device parses the point cloud media according to the point cloud object indication information of the i-th sample group.

[0011] On the other hand, an embodiment of the present application provides a data processing device for point cloud media. The data processing device for point cloud media includes:

[0012] An acquisition unit, configured to acquire the point cloud object indication information of the i-th sample group of the point cloud media. The point cloud media includes N sample groups, and the i-th sample group is any one of the N sample groups. The i-th sample group includes point cloud objects, and the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group. Both N and i are positive integers and i ∈ [1, N].

[0013] A processing unit, configured to parse the point cloud media according to the point cloud object indication information of the i-th sample group.

[0014] In one implementation, the point cloud object indication information of the i-th sample group includes an object priority field, and the object priority field is used to indicate the priority of the i-th sample group. The smaller the value of the object priority field, the higher the priority of the i-th sample group, and the smaller the possibility that the i-th sample group is discarded during transmission.

[0015] The point cloud indication information of the i-th sample group further includes an object quantity field, and the object quantity field is used to indicate the number of point cloud objects included in the i-th sample group. The value of the object quantity field is M, and M is a positive integer. When M takes the value of 1, the i-th sample group includes one point cloud object, and the point cloud object in the i-th sample group corresponds to a priority, and the priority of the i-th sample group is equal to the priority of the point cloud object included in the i-th sample group. When M takes a value greater than 1, the i-th sample group includes M point cloud objects, and each point cloud object among the M point cloud objects corresponds to a priority respectively, and the priority of the i-th sample group is equal to the highest priority among the M priorities.

[0016] In one implementation, the j-th sample group is any one of the N sample groups except the i-th sample group, j is a positive integer and j ∈ [1, N]. The priority of the i-th sample group is higher than the priority of the j-th sample group. The processing unit is specifically configured to:

[0017] Parse the i-th sample group according to the point cloud object indication information of the i-th sample group first, and then parse the j-th sample group according to the point cloud object indication information of the j-th sample group;

[0018] Among them, the priority of the i-th sample group is higher than that of the j-th sample group, including: if the j-th sample group does not include a point cloud object, the priority of the i-th sample group is higher than that of the j-th sample group; or, if the j-th sample group includes a point cloud object, but the value of the object priority field included in the point cloud object indication information of the j-th sample group is greater than the value of the object priority field included in the point cloud object indication information of the i-th sample group, then the priority of the i-th sample group is higher than that of the j-th sample group.

[0019] In one implementation, the point cloud object indication information of the i-th sample group includes an object scenario field, and the object scenario field is used to indicate the application scenario to which the point cloud object included in the i-th sample group belongs; in different application scenarios, the values of the object scenario field are different; the processing unit is specifically used for:

[0020] Read the object scenario field in the point cloud object indication information of the i-th sample group, and determine the application scenario to which the point cloud object in the i-th sample group belongs according to the value of the object scenario field;

[0021] Among them, the application scenario includes at least one of the following: high-precision map scenario, real-time inspection scenario, and emergency rescue scenario.

[0022] In one implementation, the i-th sample group contains M point cloud objects, where M is a positive integer; the point cloud object indication information of the i-th sample group includes M object type fields, and the M object type fields are used to respectively indicate the types of the M point cloud objects; the values of the object type fields corresponding to different types of point cloud objects are different; let the m-th point cloud object be any one of the M point cloud objects, and the m-th object type field be any one of the M object type fields, and the m-th object type field is used to indicate the type of the m-th point cloud object; m is a positive integer and m ∈ [1, M]; the processing unit is specifically used for:

[0023] Read the m-th object type field in the point cloud object indication information of the i-th sample group, and determine the type of the m-th point cloud object in the i-th sample group according to the value of the m-th object type field;

[0024] Among them, the type includes any one of the following: scene abnormal situation, scene indication object, and target object.

[0025] In one implementation, the point cloud object indication information of the i-th sample group further includes M object description fields, and the M object description fields are used to respectively indicate the description information of M point cloud objects; the m-th object description field is any one of the M object description fields, and the value of the m-th object description field is an 8-bit string ending with a null character, and is used to indicate the description information of the m-th point cloud object; the processing unit is specifically configured to:

[0026] Read the m-th object description field in the point cloud object indication information of the i-th sample group, determine the description information of the m-th point cloud object in the i-th sample group according to the value of the m-th object description field, and respond to the description information;

[0027] Wherein, the description information includes at least one of the following: alarm information, highlighting information, and distress information.

[0028] In one implementation, the point cloud media includes a plurality of media frames, and the plurality of media frames are encapsulated into N sample groups, and each sample group includes at least one media frame; the point cloud object in the i-th sample group exists in the media frames in the i-th sample group; all the media frames in the i-th sample group form a set that can be independently encoded and decoded; the acquisition unit is specifically configured to:

[0029] Acquire the description signaling file sent by the content production device, and the description signaling file includes at least one encapsulation file description information of the point cloud media;

[0030] If the target encapsulation file description information in the description signaling file is selected, send an acquisition request to the content production device, and the acquisition request carries the target encapsulation file description information, so that the content production device returns the target encapsulation file according to the acquisition request, and the target encapsulation file includes the point cloud object indication information of the i-th sample group;

[0031] Acquire the point cloud object indication information of the i-th sample group from the target encapsulation file;

[0032] The processing unit is specifically configured to:

[0033] Independently decode the i-th sample group according to the point cloud object indication information of the i-th sample group to obtain at least one media frame in the i-th sample group.

[0034] On the other hand, an embodiment of the present application provides a data processing device for point cloud media, and the data processing device for point cloud media includes:

[0035] A processing unit for generating point cloud object indication information of the i-th sample group of point cloud media, where the point cloud media includes N sample groups, and the i-th sample group is any one of the N sample groups; the i-th sample group includes point cloud objects, and the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group, and both N and i are positive integers and i ∈ [1, N];

[0036] A transmission unit for transmitting the point cloud object indication information of the i-th sample group to a content consumption device, so that the content consumption device can parse the point cloud media according to the point cloud object indication information of the i-th sample group.

[0037] In one implementation, the point cloud media includes multiple media frames, and the multiple media frames are encapsulated into N sample groups; the processing unit is further configured to:

[0038] Perform object recognition on each media frame of the point cloud media;

[0039] If it is recognized that at least one media frame of the point cloud media contains a point cloud object, then encapsulate the at least one recognized media frame into the i-th sample group; all media frames within the i-th sample group form a set that can be independently encoded and decoded; and,

[0040] Encapsulate the other media frames of the point cloud media that are not recognized as containing point cloud objects into the other sample groups except the i-th sample group among the N sample groups.

[0041] In one implementation, the point cloud object indication information of the i-th sample group includes an object priority field, and the object priority field is used to indicate the priority of the i-th sample group; the smaller the value of the object priority field, the higher the priority of the i-th sample group, and the smaller the possibility of the i-th sample group being discarded during transmission; the point cloud indication information of the i-th sample group also includes an object quantity field, and the object quantity field is used to indicate the number of point cloud objects included in the i-th sample group; the processing unit is specifically configured to:

[0042] Identify the number of point cloud objects in the i-th sample group, and configure the value of the object quantity field in the point cloud indication information of the i-th sample group to be M according to the number of point cloud objects in the i-th sample group, where M is a positive integer; each of the M point cloud objects corresponds to a priority;

[0043] Configure the object priority field in the point cloud object indication information of the i-th sample group according to the priorities corresponding to the M point cloud objects;

[0044] Wherein, when M takes the value of 1, the i-th sample group contains one point cloud object, and the point cloud object in the i-th sample group corresponds to a priority. The priority of the i-th sample group is equal to the priority of the point cloud object contained in the i-th sample group. When M takes a value greater than 1, the i-th sample group contains M point cloud objects, and each of the M point cloud objects corresponds to a priority respectively. The priority of the i-th sample group is equal to the highest priority among the M priorities.

[0045] In one implementation, the processing unit is further configured to: if network congestion is detected, discard the corresponding sample group in the point cloud media according to the priority indicated by the object priority field in the object indication information of each sample group included in the point cloud media, and re-encapsulate the point cloud media and then send it to the content consumption device in ascending order of the priorities of each sample group.

[0046] In one implementation, the i-th sample group contains M point cloud objects, where M is a positive integer. The point cloud object indication information of the i-th sample group contains M object type fields and M object description fields. The M object type fields are used to indicate the types of the M point cloud objects respectively, and the M object description fields are used to indicate the description information of the M point cloud objects respectively. The values of the object type fields corresponding to different types of point cloud objects are different. Let the m-th point cloud object be any one of the M point cloud objects, the m-th object type field be any one of the M object type fields, and the m-th object type field is used to indicate the type of the m-th point cloud object. Let the m-th object description field be any one of the M object description fields, and the value of the m-th object description field is an 8-bit string ending with a null character, which is used to indicate the description information of the m-th point cloud object. m is a positive integer and m ∈ [1, M]. The processing unit is specifically configured to:

[0047] Identify the type of the m-th point cloud object in the i-th sample group, and configure the m-th object type field in the point cloud object indication information of the i-th sample group according to the type of the m-th point cloud object; and,

[0048] Obtain the description information of the m-th point cloud object in the i-th sample group, and configure the m-th object description field in the point cloud object indication information of the i-th sample group according to the description information of the m-th point cloud object.

[0049] In one implementation, the point cloud object indication information of the i-th sample group contains an object scene field, and the object scene field is used to indicate the application scenario to which the point cloud objects included in the i-th sample group belong. The processing unit is specifically configured to:

[0050] Obtain the application scenario to which the point cloud objects in the i-th sample group belong, and configure the object scene field in the point cloud object indication information of the i-th sample group according to the application scenario to which the point cloud objects in the i-th sample group belong.

[0051] In one implementation, the processing unit is further configured to:

[0052] Generate a description signaling file, where the description signaling file includes at least one encapsulation file description information of the point cloud media;

[0053] Send the description signaling file to the content consumption device, and receive a fetch request sent by the content consumption device, where the fetch request carries the target encapsulation file description information in the selected description signaling file;

[0054] Return the target encapsulation file to the content consumption device according to the fetch request; the target encapsulation file includes the point cloud object indication information of the i-th sample group.

[0055] On the other hand, an embodiment of the present application provides a data processing device for point cloud media, and the data processing device for point cloud media includes:

[0056] A processor, adapted to implement computer instructions; and,

[0057] A computer-readable storage medium storing computer instructions, where the computer instructions are adapted to be loaded and executed by the processor to perform the above-mentioned data processing method for point cloud media.

[0058] On the other hand, an embodiment of the present application provides a computer-readable storage medium storing computer instructions, which when read and executed by a processor of a computer device, cause the computer device to perform the above-mentioned data processing method for point cloud media.

[0059] On the other hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the above-mentioned data processing method for point cloud media.

[0060] In the embodiments of the present application, a point cloud media includes N sample groups. The i-th sample group is any one of the N sample groups. The i-th sample group contains point cloud objects, and the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group (such as priority, the application scenario it belongs to, type, etc.). During the consumption process of the point cloud media, the point cloud media can be parsed based on the point cloud object indication information of the i-th sample group of the point cloud media. By using the point cloud object indication information of the sample group to indicate various point cloud objects and the attributes of the point cloud objects in the point cloud media, the point cloud technology standard can support richer application scenarios. And according to the attributes indicated by the point cloud object indication information of the sample group, the transmission strategy of the point cloud media can be flexibly determined, effectively improving the transmission efficiency of the point cloud media under certain network conditions, and also effectively improving the parsing and processing efficiency of the content consumption device for the point cloud media, thus bringing a better experience for the consumption of the point cloud media. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0062] Figure 1 FIG. shows a schematic structural diagram of a data processing system for a point cloud media provided by an exemplary embodiment of the present application;

[0063] Figure 2a FIG. shows a schematic structural diagram of a data processing architecture for a point cloud media provided by an exemplary embodiment of the present application;

[0064] Figure 2b FIG. shows a schematic structural diagram of a sample provided by an exemplary embodiment of the present application;

[0065] Figure 2c FIG. shows a schematic structural diagram of a container containing multiple file tracks provided by an exemplary embodiment of the present application;

[0066] Figure 2d FIG. shows a schematic structural diagram of a sample provided by another exemplary embodiment of the present application;

[0067] Figure 3 FIG. shows a schematic flowchart of a data processing method for a point cloud media provided by an exemplary embodiment of the present application;

[0068] Figure 4 FIG. shows a schematic flowchart of a data processing method for a point cloud media provided by another exemplary embodiment of the present application;

[0069] Figure 5 The figure shows a schematic structural diagram of a data processing device for point cloud media provided by an exemplary embodiment of the present application;

[0070] Figure 6 The figure shows a schematic structural diagram of a data processing device for point cloud media provided by another exemplary embodiment of the present application;

[0071] Figure 7 The figure shows a schematic structural diagram of a data processing device for point cloud media provided by an exemplary embodiment of the present application.

[0072] Specific implementation

[0073] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present application.

[0074] The embodiments of the present application provide a data processing solution for point cloud media. The so-called point cloud refers to a set of discrete points that are irregularly distributed in space and express the spatial structure and surface attributes of a three-dimensional object or a three-dimensional scene. Point cloud data is the specific recording form of the point cloud. The point cloud data of each point in the point cloud may include geometric information (i.e., three-dimensional position information) and attribute information. Among them, the geometric information of each point in the point cloud refers to the Cartesian three-dimensional coordinate data of the point, and the attribute information of each point in the point cloud may include, but is not limited to, at least one of the following: color information, material information, and laser reflection intensity information. Usually, each point in the point cloud has the same number of attribute information; for example, each point in the point cloud has two attribute information, namely color information and laser reflection intensity information; or each point in the point cloud has three attribute information, namely color information, material information, and laser reflection intensity information.

[0075] With the progress and development of science and technology, it is currently possible to obtain a large amount of high-precision point cloud data at a relatively low cost and within a short time period. The acquisition methods of point cloud data can include, but are not limited to, at least one of the following: ① Generated by computer devices. Computer devices can generate point cloud data based on virtual three-dimensional objects and virtual three-dimensional scenes. ② Obtained by 3D (3-Dimension) laser scanning. Through 3D laser scanning, point cloud data of static real-world three-dimensional objects or three-dimensional scenes can be obtained, and millions of point cloud data can be obtained per second. ③ Obtained by 3D photogrammetry. Through 3D photographic equipment (i.e., a set of cameras or a camera device with multiple lenses and sensors), the visual scenes of the real world are collected to obtain the point cloud data of the visual scenes of the real world. Through 3D photography, point cloud data of dynamic real-world three-dimensional objects or three-dimensional scenes can be obtained. ④ Obtaining point cloud data of biological tissue organs through medical devices. In the medical field, point cloud data of biological tissue organs can be obtained through medical devices such as magnetic resonance imaging (MRI), computed tomography (CT), and electromagnetic positioning information.

[0076] The so-called point cloud media refers to a point cloud media file formed by point cloud data. The point cloud media includes multiple media frames, and each media frame in the point cloud media is composed of point cloud data. The point cloud media can flexibly and conveniently express the spatial structure and surface attributes of three-dimensional objects or three-dimensional scenes, so it is widely used. The main application scenarios of point cloud media can be classified into two major categories: the first category is machine perception of point clouds, such as autonomous navigation systems (ANS), real-time inspection systems, geographic information systems (GIS), vision sorting robots, disaster relief robots, etc.; the second category is human eye perception of point clouds, such as digital cultural heritage, free viewpoint broadcasting, computer-aided design (CAD), three-dimensional immersive communication, three-dimensional immersive interaction, three-dimensional reconstruction of biological tissue organs, etc.

[0077] In addition, the data processing solution for point cloud media provided by the embodiments of this application can indicate the point cloud objects included in the point cloud media. A point cloud object refers to an object existing in the media frame of the point cloud media, that is, a special object recognized from the media frame of the point cloud media in certain application scenarios. The types of point cloud objects can include any one of the following: scene abnormal situations (such as abnormal high-voltage wire nodes, abnormal street lights, etc. detected in a real-time patrol scene), scene indication objects (such as traffic lights, cameras, etc. recognized in a high-precision map scene), target objects (such as organisms waiting for rescue (such as people, dogs, etc.) recognized in a disaster relief scene). The data processing solution for point cloud media provided by the embodiments of this application indicates the i-th sample group containing point cloud objects in the point cloud media during the production process of the point cloud media, and generates point cloud object indication information for the i-th sample group; the point cloud media includes multiple media frames, the point cloud media includes N sample groups, N is a positive integer; the multiple media frames included in the point cloud media are encapsulated into N sample groups, each sample group includes at least one media frame, the i-th sample group is any one of the N sample groups, i is a positive integer and i ∈ [1, N]; the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group, and the attributes include at least one of the following: the number of point cloud objects, the type of point cloud objects, the description information of point cloud objects, the application scenario to which the point cloud objects belong, and the priority of point cloud objects; during the consumption process of the point cloud media, the point cloud media can be parsed based on the point cloud object indication information of the i-th sample group of the point cloud media; by using the point cloud object indication information of the sample group to indicate various point cloud objects and the attributes of the point cloud objects in the point cloud media, this enables the point cloud technology standard to support richer application scenarios; and based on the attributes indicated by the point cloud object indication information of the sample group, the transmission strategy of the point cloud media can be flexibly determined, effectively improving the transmission efficiency of the point cloud media under certain network conditions, and can also effectively improve the parsing and processing efficiency of the content consumption device for the point cloud media, thus bringing a better experience for the consumption of the point cloud media.

[0078] Based on the above description, please refer to Figure 1 , Figure 1The figure shows a schematic architecture diagram of a data processing system for point cloud media provided by an exemplary embodiment of the present application. The data processing system 10 for point cloud media includes a content consumption device 101 and a content production device 102. Among them, the content production device 102 refers to the computer device used by the provider of point cloud media (such as the content producer of point cloud media). This computer device can be a terminal (such as a PC (Personal Computer), a smart mobile device (such as a smart phone), etc.), a server, a movable platform (such as an unmanned aerial vehicle (UAV), a robot, etc.), and other devices with the ability to encode and encapsulate point cloud media; the content consumption device 101 refers to the computer device used by the user of point cloud media (such as a user). This computer device can be a terminal (such as a PC (Personal Computer), a smart mobile device (such as a smart phone), a VR (Virtual Reality) device (such as a VR helmet, VR glasses), etc.), and other devices with the ability to unpack and decode point cloud media. The content production device 102 and the content consumption device 101 can be directly or indirectly connected through wired communication or wireless communication. The embodiments of the present application do not limit this here.

[0079] Figure 2a The figure shows a schematic architecture diagram of a data processing architecture for point cloud media provided by an exemplary embodiment of the present application. Next, the data processing solution for point cloud media provided by the embodiments of the present application will be introduced in combination with Figure 1 the data processing system for point cloud media shown in the figure and Figure 2a the data processing architecture for point cloud media shown in the figure. The data processing process for point cloud media includes the data processing process on the content production device side and the data processing process on the content consumption device side. The specific processing process is as follows:

[0080] I. Data processing process on the content production device side:

[0081] (1) Process of obtaining point cloud data.

[0082] In one implementation, from the perspective of the acquisition method of point cloud data, the acquisition methods of point cloud data can be divided into two types: acquiring by capturing the visual scene of the real world through a capturing device and generating by a computer device. In one implementation, the capturing device can be a hardware component set in the content production device. For example, the capturing device is a camera, a sensor, etc. of the terminal. The capturing device can also be a hardware device connected to the content production device. For example, a camera connected to the server, etc. The capturing device is used to provide the acquisition service of point cloud data for the content production device. The capturing device can include but is not limited to any of the following: imaging device, sensing device, scanning device; among them, the imaging device can include an ordinary camera, a stereo camera, a light field camera, etc.; the sensing device can include a laser device, a radar device, etc.; the scanning device can include a 3D laser scanning device, etc. The number of capturing devices can be multiple, and these capturing devices are deployed at some specific positions in the real space to simultaneously capture the point cloud data at different angles in this space, and the captured point cloud data is synchronized both in time and space. In another implementation, the computer device can generate point cloud data according to the virtual three-dimensional object and the virtual three-dimensional scene. Due to the different acquisition methods of point cloud data, the corresponding compression encoding methods for the point cloud data acquired by different methods may also be different.

[0083] (2) Encoding and encapsulation process of point cloud data.

[0084] In one implementation, the content production device can use the Geometry-Based Point Cloud Compression (GPCC) encoding method or the Video-Based Point Cloud Compression (VPCC) encoding method based on traditional video encoding to encode the acquired point cloud data, and obtain the GPCC bitstream or VPCC bitstream of the point cloud data.

[0085] In one implementation, taking the GPCC encoding method as an example, the content production device uses a file track to encapsulate the GPCC bitstream of the encoded point cloud data; the so-called file track refers to the encapsulation container of the GPCC bitstream of the encoded point cloud data; the GPCC bitstream can be encapsulated in a single file track, and the GPCC bitstream can also be encapsulated into multiple file tracks. The specific situations of the GPCC bitstream encapsulated in a single file track and the GPCC bitstream encapsulated in multiple file tracks are as follows:

[0086] ① The GPCC bitstream is encapsulated in a single file track. When the GPCC bitstream is transmitted in a single file track, it is required that the GPCC bitstream be declared and represented according to the transmission rules of the single file track. The GPCC bitstream encapsulated in a single file track does not require further processing and can be encapsulated by ISOBMFF (International Organization for Standardization Base Media File Format). Specifically, each sample encapsulated in a single file track contains one or more GPCC components. A sample refers to a set of encapsulated structures of one or more point clouds and is the encapsulation unit in the process of point cloud media encapsulation. Point cloud media contains multiple samples, and a sample is usually a media frame of point cloud media. Each sample consists of one or more Type-Length-Value Byte Stream Format (TLV) encapsulation structures. Figure 2b shows a schematic structural diagram of a sample provided by an exemplary embodiment of the present application, as Figure 2b shown. When performing single file track transmission, the sample in this file track consists of a GPCC parameter set TLV, a geometry bitstream TLV, and an attribute bitstream TLV, and this sample is encapsulated into a single file track.

[0087] ② The GPCC bitstream is encapsulated in multiple file tracks. When the encoded GPCC geometry bitstream and the encoded GPCC attribute bitstream are transmitted in different file tracks, each sample in the file track contains at least one TLV encapsulation structure. The TLV encapsulation structure carries the data of a single GPCC component, and the encoded GPCC geometry bitstream and the encoded GPCC attribute bitstream are not simultaneously included in the TLV encapsulation structure. Figure 2c shows a schematic structural diagram of a container containing multiple file tracks provided by an exemplary embodiment of the present application, as Figure 2c shown. The encapsulated packet 1 transmitted in file track 1 contains the encoded GPCC geometry bitstream and does not contain the encoded GPCC attribute bitstream; the encapsulated packet 2 transmitted in file track 2 contains the encoded GPCC attribute bitstream and does not contain the encoded GPCC geometry bitstream. Since the content consumption device should first decode the encoded GPCC geometry bitstream during decoding, and the decoding of the encoded GPCC attribute bitstream depends on the decoded geometry information, encapsulating different GPCC component bitstreams in separate file tracks enables the content consumption device to access the file track carrying the encoded GPCC geometry bitstream before the encoded GPCC attribute bitstream. Figure 2dShows a schematic structural diagram of a sample provided by another exemplary embodiment of the present application, as Figure 2d shown, when performing multiple file track transmissions, the encoded GPCC geometric bitstream and the encoded GPCC attribute bitstream are transmitted in different file tracks. The sample in this file track consists of a GPCC parameter set TLV and a geometric bitstream TLV, and the sample does not contain an attribute bitstream TLV. This sample is encapsulated in any one of the multiple file tracks.

[0088] In one implementation, the acquired point cloud data is encoded and encapsulated by a content production device to form an encapsulated file of point cloud media. This encapsulated file of point cloud media can be the entire media file or a media segment in the media file; and the content production device records the metadata of this encapsulated file of point cloud media according to the file format requirements of the point cloud media using Media Presentation Description (i.e., description signaling file) (MPD). Here, the metadata is a general term for information related to the presentation of the point cloud media. This metadata can include description information about the media content, description information about the viewport, and signaling information related to the presentation of the media content, etc. The content production device sends the MPD to the content consumption device so that the content consumption device requests to obtain the encapsulated file of the point cloud media according to the relevant description information in the MDP. Specifically, the encapsulated file of the point cloud media and the MDP are sent from the content production device to the content consumption device through a transmission mechanism (such as DASH (Dynamic Adaptive Streaming over HTTP, dynamic adaptive streaming transmission), SMT (Smart Media Transport, intelligent media transport)).

[0089] II. Data processing process on the content consumption device side:

[0090] (1) The process of unpacking and decoding point cloud data.

[0091] In one implementation, the content consumption device can obtain the encapsulated file of the point cloud media through the MDP sent by the content production device. The process of unpacking the file on the content consumption device side is the reverse of the process of encapsulating the file on the content production device side. The content consumption device unpacks the encapsulated file of the point cloud media according to the file format requirements of the point cloud media to obtain an encoded bitstream (i.e., GPCC bitstream or VPCC bitstream). The decoding process on the content consumption device side is the reverse of the encoding process on the content production device side. The content consumption device decodes the encoded bitstream to restore the point cloud data.

[0092] (2) The process of rendering point cloud data.

[0093] In one implementation, the content consumption device renders the point cloud data obtained by decoding the GPCC bitstream according to the metadata related to rendering and window in the MDP, and the rendering is completed, thus presenting the visual scene corresponding to the point cloud data.

[0094] In the embodiments of the present application, for the content production device side, first, the acquisition device samples the visual scene of the real world to obtain the point cloud data corresponding to the visual scene of the real world; then, the obtained point cloud data is encoded by the GPCC encoding method or the VPCC encoding method to obtain a GPCC bitstream or a VPCC bitstream (both the GPCC bitstream and the VPCC bitstream include an encoded geometric bitstream and an encoded attribute bitstream); then, the GPCC bitstream or the VPCC bitstream is encapsulated to obtain an encapsulated file of the point cloud media (including a media file or a media segment); the content production device can also encapsulate the metadata into the media file or the media segment, and send the encapsulated file of the point cloud media to the content consumption device through a transmission mechanism (such as a dynamic adaptive streaming transmission mechanism). For the content consumption device side, first, receive the encapsulated file of the point cloud media sent by the content production device; then, perform a de-encapsulation process on the encapsulated file of the point cloud media to obtain the encoded GPCC bitstream (or VPCC bitstream) and the metadata; then, parse the metadata in the encoded GPCC bitstream or VPCC bitstream (that is, perform a decoding process on the encoded GPCC bitstream or VPCC bitstream to obtain the point cloud data); finally, based on the viewing (window) direction of the current user, render the decoded point cloud data and display it on the content consumption device. It should be noted that the viewing (window) direction of the current user is determined by the head tracking and visual tracking functions. In addition to using the renderer to render the point cloud data in the viewing (window) direction of the current user, the audio decoder can also be used to decode and optimize the audio in the viewing (window) direction of the current user. By encoding and encapsulating the collected point cloud data by the content production device, the storage and transmission of the point cloud data are realized; the content production device sends the encapsulated file of the point cloud media obtained by encapsulation to the content consumption device, realizing the publication and sharing of the point cloud data; the content consumption device performs de-encapsulation, decoding and consumption on the encapsulated file of the point cloud media, so that the visual scene of the real world is presented on the content consumption device.

[0095] It can be understood that the data processing system of the point cloud media described in the embodiments of the present application is to more clearly illustrate the technical solutions of the embodiments of the present application, and does not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those of ordinary skill in the art know that with the evolution of the system architecture and the emergence of new service scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0096] As can be seen from the above data processing process of the point cloud media, the content production device needs to encode the point cloud media and package it into a packaged file of the point cloud media before it can be sent to the content consumption device. Correspondingly, the content consumption device needs to unpack and decode the packaged file of the point cloud media before it can render and present the point cloud media. The data processing system for the point cloud media provided by the embodiments of the present application supports data boxes (Boxes), such as ISOBMFF data boxes. A data box refers to a data block including metadata or an object including metadata, that is, the data box contains the metadata of the point cloud media; the point cloud media can be associated with multiple data boxes. For example, if the point cloud media includes N sample groups, the point cloud media is associated with N data boxes, and the i-th sample group corresponds to the i-th data box. The embodiments of the present application expand the data boxes supported by the data processing system for the point cloud media, and encapsulate the point cloud object indication information of the i-th sample group including the point cloud object in the data box; the point cloud object indication information of the i-th sample group is the PointCloudObjectIndicationGroupEntry class in the data box, and the point cloud object indication information of the i-th sample group includes at least one of the following fields: object_scene field (object_sceiario), object_priority field (object_priority), object_count field (object_count), object_type field (object_type), and object_description field (object_description); the object_scene field is used to indicate the application scenario to which the point cloud objects included in the i-th sample group belong, the object_priority field is used to indicate the priority of the i-th sample group, the object_count field is used to indicate the number of point cloud objects included in the i-th sample group, the object_type field is used to indicate the type of point cloud objects included in the i-th sample group, and the object_description field is used to indicate the description information of the point cloud objects included in the i-th sample group; the syntax of the point cloud object indication group entry class in the data box can be seen in Table 1:

[0097] Table 1

[0098]

[0099]

[0100] The semantics of the syntax shown in Table 1 above are as follows ①-⑤:

[0101] ① The object scenario field object_scenario indicates the application scenario to which the point cloud objects included in the i-th sample group belong; in different application scenarios, the values of the object scenario field are different. The correspondence between the values of the object scenario field and the application scenarios is shown in Table 2. When the value of the object scenario field is 0, it indicates that the application scenario to which the point cloud objects included in the i-th sample group belong is the high-precision map scenario; when the value of the object scenario field is 1, it indicates that the application scenario to which the point cloud objects included in the i-th sample group belong is the real-time inspection scenario; when the value of the object scenario field is 2, it indicates that the application scenario to which the point cloud objects included in the i-th sample group belong is the disaster relief scenario. It should be noted that there are other extended values for the object scenario field. That is to say, when the point cloud objects exist in other application scenarios except the above three application scenarios (i.e., the high-precision map scenario, the real-time inspection scenario, and the disaster relief scenario), only the value of the object scenario field needs to be extended to indicate other application scenarios.

[0102] Table 2

[0103] Value of the object scene field Meaning 0 High-precision map scene 1 Real-time inspection scene 2 Emergency rescue scene Others Reserved

[0104] ② The object priority field object_priority indicates the priority of the i-th sample group; the smaller the value of the object priority field, the higher the priority of the i-th sample group, the smaller the possibility that the i-th sample group is discarded during transmission, and the earlier the parsing order of the i-th sample group during parsing; the larger the value of the object priority field, the lower the priority of the i-th sample group, the greater the possibility that the i-th sample group is discarded during transmission, and the later the parsing order of the i-th sample group during parsing. Suppose there are M point cloud objects included in the i-th sample group, where M is a positive integer. When M = 1, the i-th sample group contains one point cloud object, and the point cloud object in the i-th sample group corresponds to a priority, and the priority of the i-th sample group is equal to the priority of the point cloud object included in the i-th sample group; when M > 1, the i-th sample group contains M point cloud objects, and each of the M point cloud objects corresponds to a priority, and the priority of the i-th sample group is equal to the highest priority among the M priorities.

[0105] ③ The object count field object_count indicates the number of point cloud objects included in the i-th sample group; suppose there are M point cloud objects included in the i-th sample group, then the value of the object count field is M. For example, if there is 1 point cloud object included in the i-th sample group, the value of the object count field is 1; if there are 10 point cloud objects included in the i-th sample group, the value of the object count field is 10.

[0106] ④ The object type field "object_type" indicates the type of the point cloud object included in the i-th sample group. Suppose there are M point cloud objects included in the i-th sample group. Then, the point cloud object indication information of the i-th sample group contains M object type fields, and the M object type fields are used to indicate the types of the M point cloud objects respectively. Suppose the m-th point cloud object is any one of the M point cloud objects, and the m-th object type field is any one of the M object type fields. The m-th object type field is used to indicate the type of the m-th point cloud object. The values of the object type fields corresponding to different types of point cloud objects are different. The correspondence between the values of the object type field and the types of point cloud objects is shown in Table 3. When the value of the object type field is 0, it indicates that the type of the point cloud object is a scene abnormal situation. When the value of the object type field is 1, it indicates that the type of the point cloud object is a scene indication object. When the value of the object type field is 2, it indicates that the type of the point cloud object is a target object. It should be noted that there are other extended values for the object type field. That is to say, when the type of the point cloud object is other types except the above three types (i.e., scene abnormal situation, scene indication object, target object), only the value of the object type field needs to be extended to indicate other types.

[0107] Table 3

[0108] Value of the object scene field Meaning 0 Scene abnormal situation 1 Scene indication object 2 Target object Others Reserved

[0109] ⑤ The object description field "object_description" indicates the description information of the point cloud object included in the i-th sample group. Suppose there are M point cloud objects included in the i-th sample group. Then, the point cloud object indication information of the i-th sample group contains M object description fields, and the M object description fields are used to indicate the description information of the M point cloud objects respectively. Suppose the m-th point cloud object is any one of the M point cloud objects, and the m-th object description field is any one of the M object description fields. The value of the m-th object description field is an 8-bit (Unicode Transformation Format-8, UTF-8) string ending with a null character, and is used to indicate the description information of the m-th point cloud object.

[0110] As can be seen from Table 2, after the content production device indicates the i-th sample group containing point cloud objects in the point cloud media, it generates the point cloud object indication information of the i-th sample group. The point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group (such as priority, the application scenario it belongs to, type, etc.); the content consumption device can parse the point cloud media based on the point cloud object indication information of the i-th sample group of the point cloud media, and use the point cloud object indication information of the sample group to indicate various point cloud objects and the attributes of the point cloud objects in the point cloud media, which enables the point cloud technology standard to support richer application scenarios; and according to the attributes indicated by the point cloud object indication information of the sample group, the transmission strategy of the point cloud media can be flexibly determined, effectively improving the transmission efficiency of the point cloud media under certain network conditions, and can also effectively improve the parsing and processing efficiency of the content consumption device for the point cloud media; in addition, during the process of transmitting the point cloud media to the content consumption device, if network congestion is detected in the transmission network, the corresponding sample groups in the point cloud media can be discarded in the order of the priorities from low to high indicated by the object priority fields in the object indication information of each sample group included in the point cloud media, and the point cloud media is re-encapsulated and then sent to the content consumption device, thereby saving transmission bandwidth and further improving the transmission efficiency of the point cloud media; furthermore, there are other extended values in the object scenario field and the object type field of the point cloud object indication information, further enriching the application scenarios supported by the point cloud technology standard and the types of point cloud objects, thus bringing a better experience for the consumption of the point cloud media.

[0111] Based on the above description, please refer to Figure 3 , Figure 3 which shows a schematic flowchart of a data processing method for a point cloud media provided by an exemplary embodiment of the present application. This method can be executed by Figure 1 the content consumption device 101 in the embodiment shown. The data processing method for the point cloud media includes the following steps S301 to step S302:

[0112] Step S301, obtain the point cloud object indication information of the i-th sample group of the point cloud media. The point cloud media includes N sample groups, the i-th sample group is any one of the N sample groups, the i-th sample group includes point cloud objects, and both N and i are positive integers and i ∈ [1, N].

[0113] Step S302, parse the point cloud media according to the point cloud object indication information of the i-th sample group.

[0114] In one implementation, the PointCloudObjectIndicationGroupEntry of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group, where the attributes may include at least one of the following: the number of point cloud objects, the type of point cloud objects, the description information of point cloud objects, the application scenario to which the point cloud objects belong, and the priority of point cloud objects.

[0115] In one implementation, the object_priority field is included in the PointCloudObjectIndicationGroupEntry of the i-th sample group, and the object_priority field is used to indicate the priority of the i-th sample group; the priority of the i-th sample group is determined according to the priorities of the point cloud objects included in the i-th sample group. Specifically, assume that the i-th sample group includes M point cloud objects, where M is a positive integer. When M = 1, the i-th sample group contains one point cloud object, and the point cloud object in the i-th sample group corresponds to a priority, and the priority of the i-th sample group is equal to the priority of the point cloud object included in the i-th sample group; when M > 1, the i-th sample group contains M point cloud objects, and each of the M point cloud objects corresponds to a priority, and the priority of the i-th sample group is equal to the highest priority among the M priorities.

[0116] In one implementation, the smaller the value of the object_priority field in the PointCloudObjectIndicationGroupEntry of the i-th sample group, the higher the priority of the i-th sample group; the larger the value of the object_priority field in the PointCloudObjectIndicationGroupEntry of the i-th sample group, the lower the priority of the i-th sample group. If there are situations such as limited storage space of the content consumption device or limited processing capacity of the content consumption device, the content consumption device can preferentially parse the sample groups with higher priorities in the point cloud media, that is, the higher the priority of the i-th sample group, the earlier the parsing order of the i-th sample group during parsing; the lower the priority of the i-th sample group, the later the parsing order of the i-th sample group during parsing. In this implementation, during the parsing process of the point cloud media, the sample groups with higher priorities in the point cloud media can be preferentially parsed, and then the sample groups with lower priorities in the point cloud media can be parsed. For example, the j-th sample group is any one of the N sample groups other than the i-th sample group, where j is a positive integer and j ∈ [1, N]. The j-th sample group includes point cloud objects, and the value of the object_priority field included in the PointCloudObjectIndicationGroupEntry of the j-th sample group is greater than the value of the object_priority field included in the PointCloudObjectIndicationGroupEntry of the i-th sample group. Then the priority of the i-th sample group is higher than the priority of the j-th sample group. During the parsing process of the point cloud media, the i-th sample group is preferentially parsed according to the PointCloudObjectIndicationGroupEntry of the i-th sample group, and then the j-th sample group is parsed according to the PointCloudObjectIndicationGroupEntry of the j-th sample group.

[0117] In one implementation, the priority of the sample group containing point cloud objects in the point cloud media is higher than that of the sample group not containing point cloud objects in the point cloud media; in this implementation, during the parsing process of the point cloud media, the sample group containing point cloud objects in the point cloud media is preferentially parsed, and then the sample group not containing point cloud objects in the point cloud media is parsed. For example, the j-th sample group is any one of the N sample groups except the i-th sample group, j is a positive integer and j ∈ [1, N], and the j-th sample group does not contain point cloud objects, and the i-th sample group contains point cloud objects, then the priority of the i-th sample group is higher than that of the j-th sample group. During the parsing process of the point cloud media, the i-th sample group is preferentially parsed according to the point cloud object indication information of the i-th sample group, and then the j-th sample group is parsed according to the point cloud object indication information of the j-th sample group.

[0118] In one implementation, the point cloud object indication information of the i-th sample group includes an object count field (object_count), and the object count field is used to indicate the number of point cloud objects contained in the i-th sample group; assuming that the i-th sample group contains M point cloud objects, the value of the object count field is M.

[0119] In one implementation, the point cloud object indication information of the i-th sample group includes an object scenario field (object_sceiario), and the object scenario field is used to indicate the application scenario to which the point cloud objects contained in the i-th sample group belong. In different application scenarios, the values of the object scenario field are different; in this implementation, during the parsing process of the point cloud media, the object scenario field in the point cloud object indication information of the i-th sample group is read, and the application scenario to which the point cloud objects in the i-th sample group belong is determined according to the value of the object scenario field. Among them, the application scenario may include at least one of the following: high-precision map scenario, real-time inspection scenario, and disaster relief scenario. For example, according to the value 0 of the object scenario field in the point cloud object indication information of the i-th sample group, it is determined that the application scenario to which the point cloud objects in the i-th sample group belong is the high-precision map scenario.

[0120] In one implementation, the point cloud object indication information of the i-th sample group includes an object type field (object_type), and the object type field is used to indicate the type of the point cloud object included in the i-th sample group. Specifically, the i-th sample group contains M point cloud objects, where M is a positive integer; the point cloud object indication information of the i-th sample group includes M object type fields, and the M object type fields are used to respectively indicate the types of the M point cloud objects, and the values of the object type fields corresponding to different types of point cloud objects are different; let the m-th point cloud object be any one of the M point cloud objects, and the m-th object type field be any one of the M object type fields, and the m-th object type field is used to indicate the type of the m-th point cloud object, where m is a positive integer and m ∈ [1, M]; in this implementation, during the parsing process of the point cloud media, the m-th object type field in the point cloud object indication information of the i-th sample group is read, and the type of the m-th point cloud object in the i-th sample group is determined according to the value of the m-th object type field; among them, the type can include any one of the following: scene abnormal situation, scene indication object, and target object. For example, according to the value 0 of the m-th object type field in the point cloud object indication information of the i-th sample group, it is determined that the type of the m-th point cloud object in the i-th sample group is a scene abnormal situation.

[0121] In one implementation, the point cloud object indication information of the i-th sample group includes an object description field (object_description), and the object description field is used to indicate the description information of the point cloud objects included in the i-th sample group. Specifically, there are M point cloud objects included in the i-th sample group, where M is a positive integer; the point cloud object indication information of the i-th sample group includes M object description fields, and the M object description fields are used to respectively indicate the description information of the M point cloud objects; let the m-th point cloud object be any one of the M point cloud objects, and the m-th object description field be any one of the M object description fields. The value of the m-th object description field is an 8-bit string ending with a null character and is used to indicate the description information of the m-th point cloud object, where m is a positive integer and m ∈ [1, M]; in this implementation, during the parsing process of the point cloud media, the m-th object description field in the point cloud object indication information of the i-th sample group is read, and the description information of the m-th point cloud object in the i-th sample group is determined according to the value of the m-th object description field, and the description information is responded to; among them, the description information may include at least one of the following: alarm information, highlighting information, and distress information. For example, according to the value "alarm" of the m-th object description field, it is determined that the description information of the m-th point cloud object in the i-th sample group is alarm information, and this alarm information is responded to, triggering the local alarm system; another example is that according to the value "traffic light" of the m-th object description field, it is determined that the description information of the m-th point cloud object in the i-th sample group is highlighting information, and this highlighting information is responded to, highlighting the traffic light in the point cloud media; still another example is that according to the value "SOS" of the m-th object description field, it is determined that the description information of the m-th point cloud object in the i-th sample group is distress information, and this distress information is responded to, automatically dialing the rescue phone.

[0122] In one implementation, the point cloud media includes a plurality of media frames. The plurality of media frames are encapsulated into N sample groups. Each sample group includes at least one media frame. The point cloud objects in the i-th sample group exist in the media frames within the i-th sample group. All the media frames within the i-th sample group form a set that can be independently encoded and decoded. The content consumption device obtains a description signaling file (MDP) sent by the content production device. The description signaling file includes at least one encapsulated file description information of the point cloud media. If the target encapsulated file description information in the description signaling file is selected, a request for obtaining is sent to the content production device. The request for obtaining carries the target encapsulated file description information, so that the content production device returns the target encapsulated file according to the request for obtaining. The target encapsulated file description information may include description information of the media content included in the target encapsulated file, description information of the viewport, and signaling information related to the presentation of the media content included in the target encapsulated file, etc. The target encapsulated file includes the i-th sample group and the point cloud object indication information of the i-th sample group. The content consumption device obtains the point cloud object indication information of the i-th sample group from the target encapsulated file. The content consumption device independently decodes the i-th sample group according to the point cloud object indication information of the i-th sample group to obtain at least one media frame within the i-th sample group.

[0123] In the embodiments of the present application, the point cloud media includes N sample groups. The i-th sample group is any one of the N sample groups. The i-th sample group contains point cloud objects. The object priority field in the point cloud object indication information of the i-th sample group is used to indicate the priority of the i-th sample group. The object quantity level field in the point cloud object indication information of the i-th sample group is used to indicate the quantity of the point cloud objects included in the i-th sample group. The object type field in the point cloud object indication information of the i-th sample group is used to indicate the type of the point cloud objects included in the i-th sample group. The object scene field in the point cloud object indication information of the i-th sample group is used to indicate the application scenario to which the point cloud objects included in the i-th sample group belong. The object description field in the point cloud object indication information of the i-th sample group is used to indicate the description information of the point cloud objects included in the i-th sample group. During the consumption process of the point cloud media, the point cloud media can be parsed based on the point cloud object indication information of the i-th sample group of the point cloud media. By using the point cloud object indication information of the sample group to indicate various point cloud objects and the attributes of the point cloud objects in the point cloud media, this enables the point cloud technology standard to support richer application scenarios. And based on the attributes indicated by the point cloud object indication information of the sample group, the transmission strategy of the point cloud media can be flexibly determined, effectively improving the transmission efficiency of the point cloud media under certain network conditions, and also effectively improving the parsing and processing efficiency of the content consumption device for the point cloud media, thereby bringing a better experience for the consumption of the point cloud media.

[0124] Please refer to Figure 4 , Figure 4The flowchart shows a data processing method for point cloud media provided by another exemplary embodiment of the present application. This method can be executed by Figure 1 the content production device 102 in the illustrated embodiment. The data processing method for point cloud media includes the following steps S401 to S402:

[0125] Step S401: Generate point cloud object indication information for the i-th sample group of the point cloud media. The point cloud media includes N sample groups, and the i-th sample group is any one of the N sample groups. The i-th sample group includes point cloud objects. Both N and i are positive integers, and i ∈ [1, N].

[0126] In one implementation, the point cloud media includes multiple media frames, and the multiple media frames are encapsulated into N sample groups. Specifically, the content production device uses an object recognition algorithm (such as a target recognition algorithm, an image recognition algorithm, an image processing algorithm, etc.) to perform object recognition on each media frame of the point cloud media. If it is recognized that at least one media frame of the point cloud media contains a point cloud object, the recognized at least one media frame is encapsulated into the i-th sample group. All the media frames within the i-th sample group form a set that can be independently encoded and decoded. Also, the content production device encapsulates the other media frames in the point cloud media that are not recognized as containing point cloud objects into the other sample groups except the i-th sample group among the N sample groups. That is to say, the content production device can encapsulate the media frames containing point cloud objects in the point cloud media into the i-th sample group, and encapsulate the media frames not containing point cloud objects in the point cloud media into the other sample groups except the i-th sample group among the N sample groups. In another implementation, the content production device can also encapsulate the media frames containing point cloud objects in the point cloud media into P sample groups respectively, and encapsulate the media frames not containing point cloud objects in the point cloud media into the N - P sample groups except the P sample groups among the N sample groups. P is a positive integer greater than 1, and P ≤ N.

[0127] In one implementation, the point cloud indication information of the i-th sample group includes an object quantity field, and the object quantity field is used to indicate the quantity of point cloud objects contained in the i-th sample group. The content production device recognizes the quantity of point cloud objects in the i-th sample group and configures the object quantity field in the point cloud indication information of the i-th sample group according to the quantity of point cloud objects in the i-th sample group. For example, if the content production device recognizes that there are M point cloud objects in the i-th sample group, it configures the value of the object quantity field in the point cloud indication information of the i-th sample group to M according to the recognized quantity of point cloud objects in the i-th sample group.

[0128] In one implementation, the i-th sample group contains M point cloud objects, where M is a positive integer. Each of the M point cloud objects in the M point cloud objects corresponds to a priority. The M priorities corresponding to the M point cloud objects. The object priority field in the point cloud object indication information of the i-th sample group is used to indicate the priority of the i-th sample group. The content production device determines the highest priority among the M priorities and configures the object priority field in the point cloud object indication information of the i-th sample group according to the highest priority. The smaller the value of the object priority field, the higher the priority of the i-th sample group, and the smaller the possibility that the i-th sample group is discarded during transmission. The larger the value of the object priority field, the lower the priority of the i-th sample group, and the greater the possibility that the i-th sample group is discarded during transmission. For example, the i-th sample group contains 3 point cloud objects, namely the first point cloud object, the second point cloud object, and the third point cloud object. The priority of the first point cloud object is higher than that of the second point cloud object, and the priority of the second point cloud object is higher than that of the third point cloud object. The higher the priority corresponding to the point cloud object, the smaller the priority value of the point cloud object. The first point cloud object corresponds to a priority value of 0, the second point cloud object corresponds to a priority value of 1, and the third point cloud object corresponds to a priority value of 2. If the highest priority among the 3 priorities corresponding to the 3 point cloud objects is the priority of the first point cloud object, then the priority of the i-th sample group is the priority of the first point cloud object. The content production device configures the value of the object priority field in the point cloud object indication information of the i-th sample group to 0 according to the priority value corresponding to the first point cloud object.

[0129] In one implementation, the i-th sample group contains M point cloud objects, where M is a positive integer. The point cloud object indication information of the i-th sample group contains M object type fields, and the M object type fields are used to respectively indicate the types of the M point cloud objects. The values of the object type fields corresponding to different types of point cloud objects are different. Let the m-th point cloud object be any one of the M point cloud objects, and the m-th object type field be any one of the M object type fields. The m-th object type field is used to indicate the type of the m-th point cloud object, where m is a positive integer and m ∈ [1, M]. The content production device identifies the type of the m-th point cloud object in the i-th sample group and configures the m-th object type field in the point cloud object indication information of the i-th sample group according to the type of the m-th point cloud object. For example, when the content production device identifies that the type of the m-th point cloud object in the i-th sample group is a scene anomaly, it configures the value of the m-th object type field in the point cloud object indication information of the i-th sample group to 0 according to the type of the m-th point cloud object.

[0130] In one implementation, the point cloud object indication information of the i-th sample group includes an object scene field, and the object scene field is used to indicate the application scenario to which the point cloud objects included in the i-th sample group belong; the content production device obtains the application scenario to which the point cloud objects in the i-th sample group belong, and configures the object scene field in the point cloud object indication information of the i-th sample group according to the application scenario to which the point cloud objects in the i-th sample group belong. For example, if the content production device obtains that the application scenario to which the point cloud objects in the i-th sample group belong is a high-precision map scenario, it configures the value of the object scene field in the point cloud object indication information of the i-th sample group to 0 according to the application scenario to which the point cloud objects in the i-th sample group belong.

[0131] In one implementation, the i-th sample group includes M point cloud objects, where M is a positive integer; the point cloud object indication information of the i-th sample group includes M object description fields, and the M object type fields are used to respectively indicate the description information of the M point cloud objects; let the m-th point cloud object be any one of the M point cloud objects, and the m-th object description field be any one of the M object description fields, and the m-th object description field is used to indicate the description information of the m-th point cloud object, where m is a positive integer and m ∈ [1, M]; the content production device obtains the description information of the m-th point cloud object in the i-th sample group, and configures the m-th object description field in the point cloud object indication information of the i-th sample group according to the description information of the m-th point cloud object. For example, if the description information of the m-th point cloud object in the i-th sample group obtained by the content production device is an alarm message, it configures the value of the m-th object description field in the point cloud object indication information of the i-th sample group to alarm according to the obtained description information of the m-th point cloud object.

[0132] Step S402: Transmit the point cloud object indication information of the i-th sample group to the content consumption device, so that the content consumption device parses the point cloud media according to the point cloud object indication information of the i-th sample group.

[0133] In one implementation, the content production device generates a description signaling file, and the description signaling file includes at least one encapsulation file description information of the point cloud media; the content production device issues the description signaling file to the content consumption device, and receives a fetch request sent by the content consumption device, where the fetch request carries the target encapsulation file description information in the selected description signaling file; the content production device returns the target encapsulation file to the content consumption device according to the fetch request, and the target encapsulation file includes the i-th sample group and the point cloud object indication information of the i-th sample group, so that the content consumption device parses the i-th sample group according to the point cloud object indication information of the i-th sample group.

[0134] In one implementation, during the process of the content production device transmitting point cloud media to the content consumption device, if the content production device detects network congestion in the transmission network, limited storage space of the content consumption device, or limited processing capacity of the content consumption device, the content production device discards the corresponding sample groups in the point cloud media according to the priorities indicated by the object priority fields in the object indication information of each sample group included in the point cloud media, and re-packages the point cloud media and then sends it to the content consumption device in ascending order of the priorities of the sample groups. In another implementation, the transmission network for the content production device to transmit the point cloud media to the content consumption device includes multiple intermediate nodes. If the first intermediate node (any intermediate node in the transmission network) detects network congestion, the first intermediate node discards the corresponding sample groups in the point cloud media in ascending order of the priorities of the sample groups, and re-packages the point cloud media and then sends it to the second intermediate node (any intermediate node in the transmission network other than the first intermediate node). Among them, the transmission network for the content production device to transmit the point cloud media to the content consumption device can be a CDN (Content Delivery Network).

[0135] In the embodiments of the present application, the point cloud media includes N sample groups. The i-th sample group is any one of the N sample groups, and the i-th sample group contains point cloud objects. During the production process of the point cloud media, the content production device configures the object scene field, object priority field, object quantity field, object type field, and object description field included in the point cloud object indication information of the i-th sample group in the point cloud media to generate the point cloud object indication information of the i-th sample group. The point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group; during the consumption process of the point cloud media, the point cloud media can be parsed based on the point cloud object indication information of the i-th sample group of the point cloud media; various point cloud objects and the attributes of the point cloud objects in the point cloud media are indicated by the point cloud object indication information of the sample group, which enables the point cloud technology standard to support richer application scenarios; and based on the attributes indicated by the point cloud object indication information of the sample group, the transmission strategy of the point cloud media can be flexibly determined, effectively improving the transmission efficiency of the point cloud media under certain network conditions, and also effectively improving the parsing and processing efficiency of the content consumption device for the point cloud media, thus bringing a better experience for the consumption of the point cloud media. In addition, during the process of the point cloud media being transmitted to the content consumption device, if the content production device or an intermediate node in the transmission network detects network congestion in the transmission network, the corresponding sample groups in the point cloud media can be discarded according to the priorities indicated by the object priority fields in the object indication information of each sample group included in the point cloud media, and the point cloud media is re-packaged and then sent to the content consumption device, thereby saving transmission bandwidth and further improving the transmission efficiency of the point cloud media.

[0136] Combined with the following Figure 3 embodiments and Figure 4 the content described in the embodiments, for a specific application scenario, an example description of the data processing solution for the point cloud media provided in the embodiments of the present application is given.

[0137] For example, the content production device is a drone, and the application scenario is a real-time inspection scenario. When the drone inspects the high-voltage wire node, it captures a point cloud media, which contains multiple media frames. The drone performs object recognition on each media frame included in the point cloud media and identifies that there is 1 point cloud object in all the media frames included in the point cloud media. This point cloud object is an abnormal high-voltage wire node, the type of this point cloud object is a scene abnormal situation, and the priority value corresponding to this point cloud object is 0. During the production process of the point cloud media, the drone encapsulates all the media frames included in the point cloud media into a sample group (i.e., the target sample group), and all the media frames within the target sample group form a set that can be independently encoded and decoded. The drone also generates the point cloud object indication information of the target sample group. Specifically, the drone configures the value of the object scene field in the point cloud object indication information of the target sample group to 1 according to the application scenario to which the point cloud object in the target sample group belongs; configures the value of the object priority field in the point cloud object indication information of the target sample group to 0 according to the priority value corresponding to the point cloud object in the target sample group; configures the value of the object quantity field in the point cloud indication information of the target sample group to 1 according to the number of point cloud objects identified in the target sample group; configures the value of the object type field in the point cloud object indication information of the target sample group to 0 according to the type of the point cloud object in the target sample group; configures the value of the object description field in the point cloud object indication information of the target sample group to "alarm" according to the description information of the point cloud object obtained in the target sample group. The corresponding relationship between each field included in the point cloud object indication information of the target sample group and the value of each field is shown in Table 4.

[0138] Table 4

[0139] Field Value Object scene field object_sceiario 1 Object priority field object_priority 0 Object count field object_count 1 Object type field object_type 0 Object description field object_description alarm

[0140] The drone encapsulates the point cloud object indication information of the target sample group into the encapsulated file of the point cloud media, and transmits the encapsulated file of the point cloud media to the terminal (such as the terminal used by maintenance personnel). The terminal obtains the target sample group and the point cloud object indication information of the target sample group from the encapsulated file of the point cloud media, and parses the target sample group according to the point cloud object indication information of the target sample group to obtain all media frames included in the target sample group. The terminal determines that the description information of the point cloud object of the target sample group is an alarm message according to each media frame obtained by parsing and the value "alarm" of the object description field included in the point cloud object indication information of the target sample group. The terminal responds to the alarm message and triggers the system alarm of the terminal to prompt the maintenance personnel to go to the corresponding location to repair the abnormal high-voltage wire node.

[0141] For another example, the content production device is a drone, and the application scenario is a high-precision map scenario. When the drone collects high-precision map materials, it captures point cloud media, and the point cloud media contains 60 media frames. The drone performs object recognition on each media frame included in the point cloud media and identifies that 2 point cloud objects are included in 30 of the media frames included in the point cloud media, and no point cloud objects are included in the other 30 media frames. The 2 point cloud objects are the first point cloud object and the second point cloud object respectively. The first point cloud object is a traffic signal light, the type of the first point cloud object is a scene indication object, and the priority value corresponding to the first point cloud object is 0. The second point cloud object is a vehicle, the type of the second point cloud object is reserved, and the priority value corresponding to the second point cloud object is 2. The priority of the first point cloud object is higher than that of the second point cloud object. During the production process of the point cloud media, the drone encapsulates 30 media frames containing point cloud objects in the point cloud media into the first sample group, and encapsulates the other 30 media frames not containing point cloud objects in the point cloud media into the second sample group. All the media frames in the first sample group form a set that can be independently encoded and decoded, and all the media frames in the second sample group also form a set that can be independently encoded and decoded. The drone also generates point cloud object indication information for the first sample group. Specifically, the drone configures the value of the object scene field in the point cloud object indication information of the first sample group to 0 according to the application scenario to which the point cloud objects in the first sample group belong; configures the value of the object priority field in the point cloud object indication information of the first sample group to 0 according to the priority value corresponding to the first point cloud object in the first sample group; configures the value of the object quantity field in the point cloud indication information of the first sample group to 2 according to the quantity of the point cloud objects identified in the first sample group; configures the value of the first object type field in the point cloud object indication information of the first sample group to 1 according to the type of the first point cloud object in the first sample group, and configures the value of the second object type field in the point cloud object indication information of the first sample group to other according to the type of the second point cloud object in the first sample group; configures the value of the object description field in the first point cloud object indication information of the first sample group to traffic light according to the description information of the first point cloud object obtained in the first sample group, and configures the value of the object description field in the second point cloud object indication information of the first sample group to other according to the description information of the second point cloud object obtained in the first sample group. The corresponding relationship between each field included in the point cloud object indication information of the first sample group and the value of each field is shown in Table 5.

[0142] Table 5

[0143] Field Value Object scene field object_sceiario 0 Object priority field object_priority 0 Object count field object_count 2 First object type field object_type 1 First object description field object_description traffic light Second object type field object_type Others Second object description field object_description Others

[0144] The drone encapsulates the point cloud object indication information of the first sample group into the encapsulation file of the point cloud media, and transmits the encapsulation file of the point cloud media (including the first sample group, the second sample group, and the point cloud object indication information of the first sample group) to the terminal. The transmission network through which the drone transmits the encapsulation file of the point cloud media to the terminal includes an intermediate node (i.e., the target intermediate node). If the target intermediate node detects network congestion in the transmission network, the target intermediate node discards the second sample group in ascending order of the priorities of the first sample group and the second sample group, and re-encapsulates the point cloud media and then sends it to the terminal. The terminal obtains the first sample group and the point cloud object indication information of the first sample group from the encapsulation file of the point cloud media, and parses the first sample group according to the point cloud object indication information of the first sample group to obtain 30 media frames included in the first sample group. The terminal determines that the description information of the first point cloud object in the first sample group is highlighted information based on the 30 media frames obtained by parsing and the value traffic light of the first object description field included in the point cloud object indication information of the first sample group. The terminal responds to the highlighted information and highlights the traffic signal in the point cloud media.

[0145] Please refer to Figure 5 , Figure 5 shows a schematic structural diagram of a data processing device for point cloud media provided by an exemplary embodiment of the present application. The data processing device 50 for point cloud media can be used to execute Figure 3 the corresponding steps in the data processing method for point cloud media shown. The data processing device 50 for point cloud media includes the following units:

[0146] An obtaining unit 501, configured to obtain the point cloud object indication information of the i-th sample group of the point cloud media. The point cloud media includes N sample groups, and the i-th sample group is any one of the N sample groups. The i-th sample group includes point cloud objects, and the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group. Both N and i are positive integers and i ∈ [1, N];

[0147] A processing unit 502, configured to parse the point cloud media according to the point cloud object indication information of the i-th sample group.

[0148] In one implementation, the point cloud object indication information of the i-th sample group includes an object priority field, and the object priority field is used to indicate the priority of the i-th sample group. The smaller the value of the object priority field, the higher the priority of the i-th sample group, and the smaller the possibility that the i-th sample group is discarded during transmission;

[0149] The point cloud indication information of the i-th sample group further includes an object quantity field, which is used to indicate the quantity of point cloud objects included in the i-th sample group; the value of the object quantity field is M, and M is a positive integer; when M takes the value of 1, the i-th sample group contains one point cloud object, and the point cloud object in the i-th sample group corresponds to a priority, and the priority of the i-th sample group is equal to the priority of the point cloud object included in the i-th sample group; when M takes a value greater than 1, the i-th sample group contains M point cloud objects, and each of the M point cloud objects corresponds to a priority, and the priority of the i-th sample group is equal to the highest priority among the M priorities.

[0150] In one implementation, the j-th sample group is any one of the N sample groups except the i-th sample group, j is a positive integer and j ∈ [1, N]; the priority of the i-th sample group is higher than the priority of the j-th sample group; the processing unit 502 is specifically configured to:

[0151] First, parse the i-th sample group according to the point cloud object indication information of the i-th sample group, and then parse the j-th sample group according to the point cloud object indication information of the j-th sample group;

[0152] Among them, the priority of the i-th sample group being higher than the priority of the j-th sample group includes: if the j-th sample group does not include a point cloud object, then the priority of the i-th sample group is higher than the priority of the j-th sample group; or, if the j-th sample group includes a point cloud object, but the value of the object priority field included in the point cloud object indication information of the j-th sample group is greater than the value of the object priority field included in the point cloud object indication information of the i-th sample group, then the priority of the i-th sample group is higher than the priority of the j-th sample group.

[0153] In one implementation, the point cloud object indication information of the i-th sample group includes an object scenario field, which is used to indicate the application scenario to which the point cloud object included in the i-th sample group belongs; in different application scenarios, the values of the object scenario field are different; the processing unit 502 is specifically configured to:

[0154] Read the object scenario field in the point cloud object indication information of the i-th sample group, and determine the application scenario to which the point cloud object in the i-th sample group belongs according to the value of the object scenario field;

[0155] Among them, the application scenario includes at least one of the following: high-precision map scenario, real-time inspection scenario, and emergency rescue scenario.

[0156] In one implementation, the i-th sample group contains M point cloud objects, where M is a positive integer; the point cloud object indication information of the i-th sample group contains M object type fields, and the M object type fields are used to respectively indicate the types of the M point cloud objects; the values of the object type fields corresponding to different types of point cloud objects are different; let the m-th point cloud object be any one of the M point cloud objects, and the m-th object type field be any one of the M object type fields, and the m-th object type field is used to indicate the type of the m-th point cloud object; m is a positive integer and m ∈ [1, M]; the processing unit 502 is specifically configured to:

[0157] Read the m-th object type field in the point cloud object indication information of the i-th sample group, and determine the type of the m-th point cloud object in the i-th sample group according to the value of the m-th object type field;

[0158] Wherein, the type includes any one of the following: scene abnormal conditions, scene indication objects, and target objects.

[0159] In one implementation, the point cloud object indication information of the i-th sample group further contains M object description fields, and the M object description fields are used to respectively indicate the description information of the M point cloud objects; the m-th object description field is any one of the M object description fields, and the value of the m-th object description field is an 8-bit string ending with a null character, which is used to indicate the description information of the m-th point cloud object; the processing unit 502 is specifically configured to:

[0160] Read the m-th object description field in the point cloud object indication information of the i-th sample group, determine the description information of the m-th point cloud object in the i-th sample group according to the value of the m-th object description field, and respond to the description information;

[0161] Wherein, the description information includes at least one of the following: alarm information, highlighting information, and distress information.

[0162] In one implementation, the point cloud media includes multiple media frames, and the multiple media frames are encapsulated into N sample groups, and each sample group includes at least one media frame; the point cloud objects in the i-th sample group exist in the media frames in the i-th sample group; all the media frames in the i-th sample group form a set that can be independently encoded and decoded; the acquisition unit 501 is specifically configured to:

[0163] Obtain the description signaling file sent by the content production device, and the description signaling file includes at least one encapsulation file description information of the point cloud media;

[0164] If the target encapsulation file description information in the signaling file is selected, a fetch request is sent to the content production device, and the target encapsulation file description information is carried in the fetch request, so that the content production device returns the target encapsulation file according to the fetch request. The target encapsulation file includes the point cloud object indication information of the i-th sample group;

[0165] Fetch the point cloud object indication information of the i-th sample group from the target encapsulation file;

[0166] The processing unit 502 is specifically configured to:

[0167] Independently decode the i-th sample group according to the point cloud object indication information of the i-th sample group to obtain at least one media frame within the i-th sample group.

[0168] According to an embodiment of the present application, Figure 5 Each unit in the point cloud media data processing device 50 shown can be separately or all combined into one or several other units to form, or a certain (some) unit can be further split into multiple smaller units with functional division to form, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above units are divided based on logical functions. In practical applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of the present application, the point cloud media data processing device 50 may also include other units. In practical applications, these functions can also be assisted by other units and can be realized by the cooperation of multiple units. According to another embodiment of the present application, it is possible to run a computer program (including program code) capable of executing the respective steps involved in the corresponding method shown in Figure 3 on a general computing device of a general computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), and a read-only storage medium (ROM), to construct the point cloud media data processing device 50 shown in Figure 5 and to implement the point cloud media data processing method of the embodiments of the present application. The computer program can be recorded on, for example, a computer-readable storage medium and loaded into Figure 1 the content consumption device 101 of the point cloud media data processing system shown and run therein.

[0169] In the embodiments of the present application, the point cloud media includes N sample groups. The i-th sample group is any one of the N sample groups. The i-th sample group contains point cloud objects. The object priority field in the point cloud object indication information of the i-th sample group is used to indicate the priority of the i-th sample group. The object quantity field in the point cloud object indication information of the i-th sample group is used to indicate the number of point cloud objects included in the i-th sample group. The object type field in the point cloud object indication information of the i-th sample group is used to indicate the type of the point cloud objects included in the i-th sample group. The object scene field in the point cloud object indication information of the i-th sample group is used to indicate the application scenario to which the point cloud objects included in the i-th sample group belong. The object description field in the point cloud object indication information of the i-th sample group is used to indicate the description information of the point cloud objects included in the i-th sample group. During the consumption process of the point cloud media, the point cloud media can be parsed according to the point cloud object indication information of the i-th sample group of the point cloud media. By using the point cloud object indication information of the sample group to indicate various point cloud objects and the attributes of the point cloud objects in the point cloud media, the point cloud technology standard can support richer application scenarios. And according to the attributes indicated by the point cloud object indication information of the sample group, the transmission strategy of the point cloud media can be flexibly determined, effectively improving the transmission efficiency of the point cloud media under certain network conditions, and also effectively improving the parsing and processing efficiency of the content consumption device for the point cloud media, thus bringing a better experience for the consumption of the point cloud media.

[0170] Please refer to Figure 6 , Figure 6 FIG. shows a schematic structural diagram of a data processing device for point cloud media provided by another exemplary embodiment of the present application. The data processing device 60 for point cloud media can be used to execute Figure 4 the corresponding steps in the data processing method for point cloud media shown. The data processing device 60 for point cloud media includes the following units:

[0171] A processing unit 601, configured to generate point cloud object indication information of the i-th sample group of the point cloud media. The point cloud media includes N sample groups. The i-th sample group is any one of the N sample groups. The i-th sample group includes point cloud objects. The point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group. Both N and i are positive integers and i ∈ [1, N];

[0172] A transmission unit 602, configured to transmit the point cloud object indication information of the i-th sample group to a content consumption device, so that the content consumption device parses the point cloud media according to the point cloud object indication information of the i-th sample group.

[0173] In one implementation, the point cloud media includes multiple media frames, and the multiple media frames are encapsulated into N sample groups. The processing unit 601 is further configured to:

[0174] Perform object recognition on each media frame of the point cloud media;

[0175] If it is recognized that at least one media frame of the point cloud media contains a point cloud object, encapsulate the at least one recognized media frame into the i-th sample group; all media frames within the i-th sample group form a set that can be independently encoded and decoded; and,

[0176] Encapsulate the other media frames in the point cloud media that are not recognized as containing point cloud objects into the other sample groups except the i-th sample group among the N sample groups.

[0177] In one implementation, the object indication information of the i-th sample group includes an object priority field, and the object priority field is used to indicate the priority of the i-th sample group; the smaller the value of the object priority field, the higher the priority of the i-th sample group, and the smaller the possibility that the i-th sample group is discarded during transmission; the point cloud indication information of the i-th sample group further includes an object quantity field, and the object quantity field is used to indicate the number of point cloud objects contained in the i-th sample group; the processing unit 601 is specifically configured to:

[0178] Identify the number of point cloud objects in the i-th sample group, and configure the value of the object quantity in the point cloud indication information of the i-th sample group to be M according to the number of point cloud objects in the i-th sample group, where M is a positive integer; each of the M point cloud objects corresponds to a priority;

[0179] Configure the object priority field in the point cloud object indication information of the i-th sample group according to the priorities corresponding to the M point cloud objects;

[0180] Wherein, when M is equal to 1, the i-th sample group contains one point cloud object, and the point cloud object in the i-th sample group corresponds to a priority, and the priority of the i-th sample group is equal to the priority of the point cloud object contained in the i-th sample group; when M is greater than 1, the i-th sample group contains M point cloud objects, and each of the M point cloud objects corresponds to a priority, and the priority of the i-th sample group is equal to the highest priority among the M priorities.

[0181] In one implementation, the processing unit 601 is further configured to: if network congestion is detected, discard the corresponding sample groups in the point cloud media according to the priorities indicated by the object priority fields in the object indication information of each sample group included in the point cloud media, and re-encapsulate the point cloud media and then send it to the content consumption device in the order of the priorities of each sample group from low to high.

[0182] In one implementation, the i-th sample group contains M point cloud objects, where M is a positive integer; the point cloud object indication information of the i-th sample group includes M object type fields and M object description fields. The M object type fields are used to respectively indicate the types of the M point cloud objects, and the M object description fields are used to respectively indicate the description information of the M point cloud objects; the values of the object type fields corresponding to different types of point cloud objects are different; let the m-th point cloud object be any one of the M point cloud objects, the m-th object type field be any one of the M object type fields, and the m-th object type field is used to indicate the type of the m-th point cloud object; let the m-th object description field be any one of the M object description fields, and the value of the m-th object description field is an 8-bit string ending with a null character, which is used to indicate the description information of the m-th point cloud object; m is a positive integer and m ∈ [1, M]; the processing unit 601 is specifically configured to:

[0183] Identify the type of the m-th point cloud object in the i-th sample group, and configure the m-th object type field in the point cloud object indication information of the i-th sample group according to the type of the m-th point cloud object; and,

[0184] Obtain the description information of the m-th point cloud object in the i-th sample group, and configure the m-th object description field in the point cloud object indication information of the i-th sample group according to the description information of the m-th point cloud object.

[0185] In one implementation, the point cloud object indication information of the i-th sample group includes an object scene field, and the object scene field is used to indicate the application scenario to which the point cloud objects included in the i-th sample group belong; the processing unit 601 is specifically configured to:

[0186] Obtain the application scenario to which the point cloud objects in the i-th sample group belong, and configure the object scene field in the point cloud object indication information of the i-th sample group according to the application scenario to which the point cloud objects in the i-th sample group belong.

[0187] In one implementation, the processing unit 601 is further configured to:

[0188] Generate a description signaling file, and the description signaling file includes at least one encapsulated file description information of the point cloud media;

[0189] Send the description signaling file to the content consumption device, and receive a fetch request sent by the content consumption device. The fetch request carries the target encapsulated file description information in the selected description signaling file;

[0190] Return the target encapsulated file to the content consumption device according to the fetch request; the target encapsulated file includes the point cloud object indication information of the i-th sample group.

[0191] According to an embodiment of the present application, Figure 6Each unit in the data processing device 60 of the point cloud media shown can be separately or all combined into one or several other units to form, or a certain one (or some) of the units can be further split into multiple smaller units with more specific functions to form. This can achieve the same operations without affecting the realization of the technical effects of the embodiments of this application. The above units are divided based on logical functions. In actual applications, the function of one unit can also be realized by multiple units, or the functions of multiple units can be realized by one unit. In other embodiments of this application, the data processing device 60 of the point cloud media can also include other units. In actual applications, these functions can also be assisted by other units and can be realized through the cooperation of multiple units. According to another embodiment of this application, it can be achieved by running a computer program (including program code) that can execute the respective steps involved in the corresponding method shown in Figure 4 on a general computing device of a general computer including processing elements and storage elements such as a central processing unit (CPU), a random access storage medium (RAM), a read-only storage medium (ROM), etc., to construct a data processing device 60 of the point cloud media as shown in Figure 6 , and to implement the data processing method of the point cloud media in the embodiments of this application. The computer program can be recorded on, for example, a computer-readable storage medium and loaded into Figure 1 the content production device 102 of the data processing system of the point cloud media shown through the computer-readable storage medium and run therein.

[0192] In an embodiment of the present application, a point cloud medium includes N sample groups. The i-th sample group is any one of the N sample groups, and the i-th sample group contains point cloud objects. During the production process of the point cloud medium, a content production device configures an object scene field, an object priority field, an object quantity field, an object type field, and an object description field included in the point cloud object indication information of the i-th sample group in the point cloud medium to generate the point cloud object indication information of the i-th sample group. The point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group. During the consumption process of the point cloud medium, the point cloud medium can be parsed based on the point cloud object indication information of the i-th sample group of the point cloud medium. By using the point cloud object indication information of the sample group to indicate various point cloud objects and the attributes of the point cloud objects in the point cloud medium, the point cloud technology standard can support richer application scenarios. Moreover, based on the attributes indicated by the point cloud object indication information of the sample group, the transmission strategy of the point cloud medium can be flexibly determined, effectively improving the transmission efficiency of the point cloud medium under certain network conditions, and also effectively improving the parsing and processing efficiency of the content consumption device for the point cloud medium, thereby bringing a better experience for the consumption of the point cloud medium. In addition, during the process of transmitting the point cloud medium to the content consumption device, if the content production device or an intermediate node in the transmission network detects network congestion in the transmission network, it can discard the corresponding sample groups in the point cloud medium according to the priorities indicated by the object priority fields in the object indication information of each sample group included in the point cloud medium, and re-encapsulate the point cloud medium and then send it to the content consumption device, thereby saving transmission bandwidth and further improving the transmission efficiency of the point cloud medium.

[0193] Please refer to Figure 7 , Figure 7 FIG. shows a schematic structural diagram of a data processing device for a point cloud medium provided by an exemplary embodiment of the present application. The data processing device 70 for the point cloud medium includes at least a processor 701 and a computer-readable storage medium 702. Among them, the processor 701 and the computer-readable storage medium 702 can be connected through a bus or other means. The computer-readable storage medium 702 can be stored in a memory. The computer-readable storage medium 702 is used to store a computer program, and the computer program includes computer instructions. The processor 701 is used to execute the computer instructions stored in the computer-readable storage medium 702. The processor 701 (or CPU (Central Processing Unit)) is the computing core and control core of the data processing device 70 for the point cloud medium. It is suitable for implementing one or more computer instructions, specifically suitable for loading and executing one or more computer instructions to implement the corresponding method flow or corresponding function.

[0194] The embodiment of the present application also provides a computer-readable storage medium (Memory). The computer-readable storage medium is a memory device in the data processing device 70 of the point cloud media, and is used to store programs and data. It can be understood that the computer-readable storage medium 702 here can include both the built-in storage medium in the data processing device 70 of the point cloud media, and of course can also include the extended storage medium supported by the data processing device 70 of the point cloud media. The computer-readable storage medium provides a storage space, and the operating system of the data processing device 70 of the point cloud media is stored in this storage space. And, one or more computer instructions suitable for being loaded and executed by the processor 701 are also stored in this storage space. These computer instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium 702 here can be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk memory; optionally, it can also be at least one computer-readable storage medium located far from the aforementioned processor 701.

[0195] In one implementation, the data processing device 70 of the point cloud media can be Figure 1 the content consumption device 101 in the point cloud media data processing system shown in the figure; a first computer instruction is stored in the computer-readable storage medium 702; the first computer instruction stored in the computer-readable storage medium 702 is loaded and executed by the processor 701 to implement Figure 3 the corresponding steps in the method embodiment shown in the figure; in a specific implementation, the first computer instruction in the computer-readable storage medium 702 is loaded and executed by the processor 701 as follows:

[0196] Obtain the point cloud object indication information of the i-th sample group of the point cloud media. The point cloud media includes N sample groups, and the i-th sample group is any one of the N sample groups; the i-th sample group includes point cloud objects, and the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group. Both N and i are positive integers and i ∈ [1, N];

[0197] Parse the point cloud media according to the point cloud object indication information of the i-th sample group.

[0198] In one implementation, the point cloud object indication information of the i-th sample group includes an object priority field, and the object priority field is used to indicate the priority of the i-th sample group; the smaller the value of the object priority field, the higher the priority of the i-th sample group, and the smaller the possibility that the i-th sample group is discarded during transmission;

[0199] The point cloud indication information of the i-th sample group further includes an object quantity field, which is used to indicate the number of point cloud objects included in the i-th sample group; the value of the object quantity field is M, and M is a positive integer; when M takes the value of 1, the i-th sample group contains one point cloud object, and the point cloud object in the i-th sample group corresponds to a priority, and the priority of the i-th sample group is equal to the priority of the point cloud object included in the i-th sample group; when M takes a value greater than 1, the i-th sample group contains M point cloud objects, and each of the M point cloud objects corresponds to a priority, and the priority of the i-th sample group is equal to the highest priority among the M priorities.

[0200] In one implementation, the j-th sample group is any one of the N sample groups except the i-th sample group, j is a positive integer and j ∈ [1, N]; the priority of the i-th sample group is higher than the priority of the j-th sample group; the first computer instruction in the computer-readable storage medium 702 is loaded and specifically executed by the processor 701 as follows:

[0201] First, parse the i-th sample group according to the point cloud object indication information of the i-th sample group, and then parse the j-th sample group according to the point cloud object indication information of the j-th sample group;

[0202] Among them, the priority of the i-th sample group being higher than the priority of the j-th sample group includes: if the j-th sample group does not include a point cloud object, then the priority of the i-th sample group is higher than the priority of the j-th sample group; or, if the j-th sample group includes a point cloud object, but the value of the object priority field included in the point cloud object indication information of the j-th sample group is greater than the value of the object priority field included in the point cloud object indication information of the i-th sample group, then the priority of the i-th sample group is higher than the priority of the j-th sample group.

[0203] In one implementation, the point cloud object indication information of the i-th sample group includes an object scenario field, which is used to indicate the application scenario to which the point cloud object included in the i-th sample group belongs; in different application scenarios, the values of the object scenario field are different; the first computer instruction in the computer-readable storage medium 702 is loaded and specifically executed by the processor 701 as follows:

[0204] Read the object scenario field in the point cloud object indication information of the i-th sample group, and determine the application scenario to which the point cloud object in the i-th sample group belongs according to the value of the object scenario field;

[0205] Among them, the application scenario includes at least one of the following: high-precision map scenario, real-time inspection scenario, and emergency rescue scenario.

[0206] In one implementation, the i-th sample group contains M point cloud objects, where M is a positive integer; the point cloud object indication information of the i-th sample group contains M object type fields, and the M object type fields are used to respectively indicate the types of the M point cloud objects; the values of the object type fields corresponding to different types of point cloud objects are different; let the m-th point cloud object be any one of the M point cloud objects, and the m-th object type field be any one of the M object type fields, and the m-th object type field is used to indicate the type of the m-th point cloud object; m is a positive integer and m ∈ [1, M]; the first computer instructions in the computer-readable storage medium 702 are loaded by the processor 701 and specifically execute the following steps:

[0207] Read the m-th object type field in the point cloud object indication information of the i-th sample group, and determine the type of the m-th point cloud object in the i-th sample group according to the value of the m-th object type field;

[0208] Among them, the type includes any one of the following: scene abnormal situation, scene indication object, and target object.

[0209] In one implementation, the point cloud object indication information of the i-th sample group further contains M object description fields, and the M object description fields are used to respectively indicate the description information of the M point cloud objects; the m-th object description field is any one of the M object description fields, and the value of the m-th object description field is an 8-bit string ending with a null character, which is used to indicate the description information of the m-th point cloud object; the first computer instructions in the computer-readable storage medium 702 are loaded by the processor 701 and specifically execute the following steps:

[0210] Read the m-th object description field in the point cloud object indication information of the i-th sample group, and determine the description information of the m-th point cloud object in the i-th sample group according to the value of the m-th object description field, and respond to the description information;

[0211] Among them, the description information includes at least one of the following: alarm information, highlighting information, and distress information.

[0212] If the point cloud object indication information of the i-th sample group indicates that the application scenario to which the point cloud objects included in the i-th sample group belong is a disaster relief scenario, and the type of the m-th point cloud object is a target object, then the priority of the m-th point cloud object is higher than the priorities of other point cloud objects.

[0213] In one implementation, the point cloud media includes multiple media frames, and the multiple media frames are encapsulated into N sample groups, and each sample group includes at least one media frame; the point cloud objects in the i-th sample group exist in the media frames in the i-th sample group; all the media frames in the i-th sample group form a set that can be independently encoded and decoded; the first computer instructions in the computer-readable storage medium 702 are loaded by the processor 701 and specifically execute the following steps:

[0214] Obtain a description signaling file sent by a content production device, where the description signaling file includes at least one encapsulation file description information of point cloud media;

[0215] If the target encapsulation file description information in the description signaling file is selected, send a obtain request to the content production device, where the obtain request carries the target encapsulation file description information, so that the content production device returns a target encapsulation file according to the obtain request, and the target encapsulation file includes point cloud object indication information of the i-th sample group;

[0216] Obtain the point cloud object indication information of the i-th sample group from the target encapsulation file;

[0217] The first computer instructions in the computer-readable storage medium 702 are specifically executed by the processor 701 as follows:

[0218] Independently decode the i-th sample group according to the point cloud object indication information of the i-th sample group to obtain at least one media frame within the i-th sample group.

[0219] In one implementation, the data processing device 70 of the point cloud media may be Figure 1 The content production device 102 in the data processing system of the point cloud media shown; the computer-readable storage medium 702 stores second computer instructions; the second computer instructions stored in the computer-readable storage medium 702 are loaded and executed by the processor 701 to implement Figure 4 The corresponding steps in the method embodiment shown; in a specific implementation, the second computer instructions in the computer-readable storage medium 702 are loaded and executed by the processor 701 as follows:

[0220] Generate point cloud object indication information of the i-th sample group of the point cloud media, where the point cloud media includes N sample groups, and the i-th sample group is any one of the N sample groups; the i-th sample group includes point cloud objects, and the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group, and N and i are both positive integers and i ∈ [1, N];

[0221] Transmit the point cloud object indication information of the i-th sample group to a content consumption device, so that the content consumption device parses the point cloud media according to the point cloud object indication information of the i-th sample group.

[0222] In one implementation, the point cloud media includes multiple media frames, and the multiple media frames are encapsulated into N sample groups; the second computer instructions in the computer-readable storage medium 702 are further executed by the processor 701 as follows:

[0223] Perform object recognition on each media frame of the point cloud media;

[0224] If at least one media frame of the point cloud media contains a point cloud object, encapsulate the identified at least one media frame into the i-th sample group; all media frames within the i-th sample group form a set that can be independently encoded and decoded; and,

[0225] Encapsulate the other media frames in the point cloud media that are not identified as containing point cloud objects into the other sample groups except the i-th sample group among the N sample groups.

[0226] In one implementation, the object priority field is included in the point cloud object indication information of the i-th sample group, and the object priority field is used to indicate the priority of the i-th sample group; the smaller the value of the object priority field, the higher the priority of the i-th sample group, and the smaller the possibility of the i-th sample group being discarded during transmission; the point cloud indication information of the i-th sample group also includes an object quantity field, and the object quantity field is used to indicate the number of point cloud objects included in the i-th sample group; the second computer instruction in the computer-readable storage medium 702 is specifically executed by the processor 701 as follows:

[0227] Identify the number of point cloud objects in the i-th sample group, and configure the value of the object quantity field in the point cloud indication information of the i-th sample group to be M according to the number of point cloud objects in the i-th sample group, where M is a positive integer; each of the M point cloud objects corresponds to a priority;

[0228] Configure the object priority field in the point cloud object indication information of the i-th sample group according to the priorities corresponding to the M point cloud objects;

[0229] Wherein, when M is 1, the i-th sample group contains one point cloud object, and the point cloud object in the i-th sample group corresponds to a priority, and the priority of the i-th sample group is equal to the priority of the point cloud object included in the i-th sample group; when M is greater than 1, the i-th sample group contains M point cloud objects, and each of the M point cloud objects corresponds to a priority, and the priority of the i-th sample group is equal to the highest priority among the M priorities.

[0230] In one implementation, the second computer instruction in the computer-readable storage medium 702 is further executed by the processor 701 as follows: If network congestion is detected, discard the corresponding sample groups in the point cloud media in the order of the priorities of each sample group from low to high according to the priorities indicated by the object priority fields in the object indication information of each sample group included in the point cloud media, and re-encapsulate the point cloud media and then send it to the content consumption device.

[0231] In one implementation, the i-th sample group contains M point cloud objects, where M is a positive integer; the point cloud object indication information of the i-th sample group contains M object type fields and M object description fields. The M object type fields are used to respectively indicate the types of the M point cloud objects, and the M object description fields are used to respectively indicate the description information of the M point cloud objects; the values of the object type fields corresponding to different types of point cloud objects are different; let the m-th point cloud object be any one of the M point cloud objects, the m-th object type field be any one of the M object type fields, and the m-th object type field is used to indicate the type of the m-th point cloud object; let the m-th object description field be any one of the M object description fields, and the value of the m-th object description field is an 8-bit string ending with a null character, which is used to indicate the description information of the m-th point cloud object; m is a positive integer and m ∈ [1, M]; the second computer instructions in the computer-readable storage medium 702 are loaded and specifically executed by the processor 701 as follows:

[0232] Identify the type of the m-th point cloud object in the i-th sample group, and configure the m-th object type field in the point cloud object indication information of the i-th sample group according to the type of the m-th point cloud object; and,

[0233] Obtain the description information of the m-th point cloud object in the i-th sample group, and configure the m-th object description field in the point cloud object indication information of the i-th sample group according to the description information of the m-th point cloud object.

[0234] In one implementation, the point cloud object indication information of the i-th sample group contains an object scene field, and the object scene field is used to indicate the application scenario to which the point cloud objects included in the i-th sample group belong; the second computer instructions in the computer-readable storage medium 702 are loaded and specifically executed by the processor 701 as follows:

[0235] Obtain the application scenario to which the point cloud objects in the i-th sample group belong, and configure the object scene field in the point cloud object indication information of the i-th sample group according to the application scenario to which the point cloud objects in the i-th sample group belong.

[0236] In one implementation, the i-th sample group contains M point cloud objects, where M is a positive integer; the point cloud object indication information of the i-th sample group contains M object description fields, and the M object description fields are used to respectively indicate the description information of the M point cloud objects; let the m-th point cloud object be any one of the M point cloud objects, the m-th object description field be any one of the M object description fields, and the value of the m-th object description field is an 8-bit string ending with a null character, which is used to indicate the description information of the m-th point cloud object; the second computer instructions in the computer-readable storage medium 702 are loaded and specifically executed by the processor 701 as follows:

[0237] Obtain the description information of the m-th point cloud object within the i-th sample group, and configure the m-th object description field in the point cloud object indication information of the i-th sample group according to the description information of the m-th point cloud object.

[0238] In one implementation, the second computer instructions in the computer-readable storage medium 702 are loaded and executed by the processor 701 to perform the following steps:

[0239] Generate a description signaling file, where the description signaling file includes at least one encapsulated file description information of the point cloud media;

[0240] Send the description signaling file to the content consumption device, and receive a fetch request sent by the content consumption device. The fetch request carries the target encapsulated file description information in the selected description signaling file;

[0241] Return the target encapsulated file to the content consumption device according to the fetch request; the target encapsulated file includes the point cloud object indication information of the i-th sample group.

[0242] In the embodiments of the present application, the point cloud media includes N sample groups. The i-th sample group is any one of the N sample groups. The i-th sample group contains point cloud objects. The point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group (such as priority, the application scenario to which it belongs, type, etc.); during the consumption process of the point cloud media, the point cloud media can be parsed based on the point cloud object indication information of the i-th sample group of the point cloud media; various point cloud objects and the attributes of the point cloud objects in the point cloud media are indicated through the point cloud object indication information of the sample group, which enables the point cloud technology standard to support richer application scenarios; and according to the attributes indicated by the point cloud object indication information of the sample group, the transmission strategy of the point cloud media can be flexibly determined, effectively improving the transmission efficiency of the point cloud media under certain network conditions, and can also effectively improve the parsing and processing efficiency of the content consumption device for the point cloud media, thereby bringing a better experience for the consumption of the point cloud media.

[0243] According to one aspect of the present application, there is provided a computer program product or a computer program. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the data processing method of the point cloud media provided in the above various optional manners.

[0244] As described above, it is only the specific implementation manner of the present application. However, the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the said claims.

Claims

1. A data processing method for point cloud media, characterized in that, The method includes: Obtaining the point cloud object indication information of the i-th sample group of the point cloud media, where the point cloud media includes multiple samples, and one sample refers to one media frame, and multiple media frames are encapsulated into N sample groups; the i-th sample group is obtained by encapsulating at least one media frame in which a point cloud object is identified, and the point cloud object refers to an object identified from the media frames of the point cloud media; the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group, and both N and i are positive integers and i ∈ [1, N]; Parsing the point cloud media according to the point cloud object indication information of the i-th sample group.

2. The method according to claim 1, characterized in that, The point cloud object indication information of the i-th sample group includes an object priority field, and the object priority field is used to indicate the priority of the i-th sample group; the smaller the value of the object priority field, the higher the priority of the i-th sample group, and the smaller the possibility that the i-th sample group is discarded during transmission; The point cloud indication information of the i-th sample group further includes an object quantity field, and the object quantity field is used to indicate the number of point cloud objects included in the i-th sample group; the value of the object quantity field is M, and M is a positive integer; when M is 1, the i-th sample group includes one point cloud object, and the point cloud object in the i-th sample group corresponds to one priority, and the priority of the i-th sample group is equal to the priority of the point cloud object included in the i-th sample group; when M is greater than 1, the i-th sample group includes M point cloud objects, and each of the M point cloud objects corresponds to one priority respectively, and the priority of the i-th sample group is equal to the highest priority among the M priorities.

3. The method according to claim 2, wherein The j-th sample group is any one of the N sample groups other than the i-th sample group, and j is a positive integer and j ∈ [1, N]; The priority of the i-th sample group is higher than the priority of the j-th sample group; The parsing the point cloud media according to the point cloud object indication information of the i-th sample group includes: Parsing the i-th sample group preferentially according to the point cloud object indication information of the i-th sample group, and then parsing the j-th sample group according to the point cloud object indication information of the j-th sample group; Among them, the priority of the i-th sample group being higher than the priority of the j-th sample group includes: if the j-th sample group does not include a point cloud object, then the priority of the i-th sample group is higher than the priority of the j-th sample group; or, if the j-th sample group includes a point cloud object, but the value of the object priority field included in the point cloud object indication information of the j-th sample group is greater than the value of the object priority field included in the point cloud object indication information of the i-th sample group, then the priority of the i-th sample group is higher than the priority of the j-th sample group.

4. The method according to claim 1, wherein The point cloud object indication information of the i-th sample group includes an object scene field, and the object scene field is used to indicate the application scenario to which the point cloud objects included in the i-th sample group belong; Under different application scenarios, the values of the object scene field are different; Parsing the point cloud media according to the point cloud object indication information of the i-th sample group includes: Reading the object scene field in the point cloud object indication information of the i-th sample group, and determining the application scenario to which the point cloud object in the i-th sample group belongs according to the value of the object scene field; Wherein, the application scenario includes at least one of the following: high-precision map scenario, real-time inspection scenario, and disaster relief scenario.

5. The method according to claim 1, wherein The i-th sample group contains M point cloud objects, where M is a positive integer; the point cloud object indication information of the i-th sample group contains M object type fields, and the M object type fields are used to respectively indicate the types of the M point cloud objects; the values of the object type fields corresponding to different types of point cloud objects are different; let the m-th point cloud object be any one of the M point cloud objects, and the m-th object type field be any one of the M object type fields, and the m-th object type field is used to indicate the type of the m-th point cloud object; m is a positive integer and m ∈ [1, M]; Parsing the point cloud media according to the point cloud object indication information of the i-th sample group includes: Reading the m-th object type field in the point cloud object indication information of the i-th sample group, and determining the type of the m-th point cloud object in the i-th sample group according to the value of the m-th object type field; Wherein, the type of the point cloud object includes any one of the following: scene abnormal situation, scene indication object, and target object.

6. The method according to claim 5, wherein The point cloud object indication information of the i-th sample group further contains M object description fields, and the M object description fields are used to respectively indicate the description information of the M point cloud objects; the m-th object description field is any one of the M object description fields, and the value of the m-th object description field is an 8-bit string ending with a null character, and is used to indicate the description information of the m-th point cloud object; Parsing the point cloud media according to the point cloud object indication information of the i-th sample group further includes: Reading the m-th object description field in the point cloud object indication information of the i-th sample group, and determining the description information of the m-th point cloud object in the i-th sample group according to the value of the m-th object description field, and responding to the description information; Wherein, the description information includes at least one of the following: alarm information, highlighting information, and distress information.

7. The method according to claim 1, wherein Each sample group includes at least one media frame; the point cloud objects in the i-th sample group exist in the media frames in the i-th sample group; All the media frames in the i-th sample group form a set that can be independently encoded and decoded; Obtaining the point cloud object indication information of the i-th sample group of the point cloud media includes: Obtaining a description signaling file sent by the content production device, and the description signaling file includes at least one encapsulation file description information of the point cloud media; If the target encapsulation file description information in the described description signaling file is selected, a fetch request is sent to the content production device, and the target encapsulation file description information is carried in the fetch request, so that the content production device returns a target encapsulation file according to the fetch request, and the target encapsulation file includes the point cloud object indication information of the i-th sample group; Fetch the point cloud object indication information of the i-th sample group from the target encapsulation file; The parsing of the point cloud media according to the point cloud object indication information of the i-th sample group includes: Independently decode the i-th sample group according to the point cloud object indication information of the i-th sample group to obtain at least one media frame within the i-th sample group.

8. A data processing method for point cloud media, characterized in that, The method includes: Generate the point cloud object indication information of the i-th sample group of the point cloud media, where the point cloud media includes multiple samples, one sample refers to one media frame, and multiple media frames are encapsulated into N sample groups; the i-th sample group is encapsulated by at least one media frame that identifies a point cloud object, and the point cloud object refers to an object identified from the media frames of the point cloud media; the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud objects included in the i-th sample group, and both N and i are positive integers and i ∈ [1, N]; Transmit the point cloud object indication information of the i-th sample group to the content consumption device, so that the content consumption device parses the point cloud media according to the point cloud object indication information of the i-th sample group.

9. The method according to claim 8, wherein The method further includes: Perform object recognition on each media frame of the point cloud media; If it is recognized that at least one media frame of the point cloud media contains a point cloud object, encapsulate the recognized at least one media frame into the i-th sample group; all the media frames within the i-th sample group form a set that can be independently encoded and decoded; and, Encapsulate the other media frames in the point cloud media that are not recognized as containing point cloud objects into the other sample groups except the i-th sample group among the N sample groups respectively.

10. The method according to claim 8, wherein The point cloud object indication information of the i-th sample group includes an object priority field, and the object priority field is used to indicate the priority of the i-th sample group; the smaller the value of the object priority field, the higher the priority of the i-th sample group, and the smaller the possibility that the i-th sample group is discarded during transmission; the point cloud indication information of the i-th sample group further includes an object quantity field, and the object quantity field is used to indicate the number of point cloud objects included in the i-th sample group; The generating of the point cloud object indication information of the i-th sample group of the point cloud media includes: Identify the number of point cloud objects within the i-th sample group, and configure the value of the object quantity field in the point cloud indication information of the i-th sample group to be M according to the number of point cloud objects within the i-th sample group, where M is a positive integer; each of the M point cloud objects corresponds to a priority; Configure the object priority field in the point cloud object indication information of the i-th sample group according to the priorities corresponding to the M point cloud objects; Wherein, when M takes the value of 1, the i-th sample group contains one point cloud object, and the point cloud object in the i-th sample group corresponds to a priority. The priority of the i-th sample group is equal to the priority of the point cloud object contained in the i-th sample group. When M takes a value greater than 1, the i-th sample group contains M point cloud objects, and each of the M point cloud objects corresponds to a priority respectively. The priority of the i-th sample group is equal to the highest priority among the M priorities.

11. The method according to claim 10, wherein The method further includes: If network congestion is detected, then according to the priorities indicated by the object priority fields in the object indication information of each sample group included in the point cloud media, discard the corresponding sample groups in the point cloud media in ascending order of the priorities of each sample group, and re-encapsulate the point cloud media and send it to the content consumption device.

12. The method according to claim 8, wherein, The i-th sample group contains M point cloud objects, where M is a positive integer. The point cloud object indication information of the i-th sample group includes M object type fields and M object description fields. The M object type fields are used to indicate the types of the M point cloud objects respectively, and the M object description fields are used to indicate the description information of the M point cloud objects respectively. The values of the object type fields corresponding to different types of point cloud objects are different. Let the m-th point cloud object be any one of the M point cloud objects, and the m-th object type field be any one of the M object type fields. The m-th object type field is used to indicate the type of the m-th point cloud object. Let The m-th object description field is any one of the M object description fields. The value of the m-th object description field is an 8-bit string ending with a null character and is used to indicate the description information of the m-th point cloud object. m is a positive integer and m ∈ [1, M]; Generating the point cloud object indication information of the i-th sample group of the point cloud media includes: Identifying the type of the m-th point cloud object in the i-th sample group, and configuring the m-th object type field in the point cloud object indication information of the i-th sample group according to the type of the m-th point cloud object. And Obtaining the description information of the m-th point cloud object in the i-th sample group, and configuring the m-th object description field in the point cloud object indication information of the i-th sample group according to the description information of the m-th point cloud object.

13. The method according to claim 8, wherein The point cloud object indication information of the i-th sample group includes an object scene field, and the object scene field is used to indicate the application scenario to which the point cloud objects included in the i-th sample group belong. Generating the point cloud object indication information of the i-th sample group of the point cloud media includes: Obtaining the application scenario to which the point cloud objects in the i-th sample group belong, and configuring the object scene field in the point cloud object indication information of the i-th sample group according to the application scenario to which the point cloud objects in the i-th sample group belong.

14. A data processing device for point cloud media, characterized in that, The data processing device of the point cloud media includes: An acquisition unit, configured to acquire the point cloud object indication information of the i-th sample group of the point cloud media, where the point cloud media includes a plurality of samples, one sample refers to one media frame, and a plurality of media frames are encapsulated into N sample groups; the i-th sample group is obtained by encapsulating at least one media frame in which a point cloud object is identified, and the point cloud object refers to an object identified from the media frames of the point cloud media; the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud object included in the i-th sample group, and both N and i are positive integers and i ∈ [1, N]; A processing unit, configured to parse the point cloud media according to the point cloud object indication information of the i-th sample group.

15. A data processing device for point cloud media, characterized in that, The data processing device of the point cloud media includes: A processing unit, configured to generate the point cloud object indication information of the i-th sample group of the point cloud media, where the point cloud media includes a plurality of samples, one sample refers to one media frame, and a plurality of media frames are encapsulated into N sample groups; the i-th sample group is obtained by encapsulating at least one media frame in which a point cloud object is identified, and the point cloud object refers to an object identified from the media frames of the point cloud media; the point cloud object indication information of the i-th sample group is used to indicate the attributes of the point cloud object included in the i-th sample group, and both N and i are positive integers and i ∈ [1, N]; A transmission unit, configured to transmit the point cloud object indication information of the i-th sample group to a content consumption device, so that the content consumption device parses the point cloud media according to the point cloud object indication information of the i-th sample group.

16. A data processing device for point cloud media, characterized in that, The data processing device of the point cloud media includes: A processor, adapted to implement computer instructions; and, A computer-readable storage medium storing computer instructions, the computer instructions being adapted to be loaded and executed by the processor to perform the data processing method of the point cloud media according to any one of claims 1 to 13.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions, the computer instructions being adapted to be loaded and executed by a processor to perform the data processing method of the point cloud media according to any one of claims 1 to 13.

18. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the data processing method of the point cloud media according to any one of claims 1 to 13.

Images