Scene description file parsing method and apparatus

Abstract

Some embodiments of the present application provide a scene description file parsing method and apparatus. The method comprises: acquiring a first node description module from a node list of a scene description file of a three-dimensional scene to be rendered, the first node description module being a node description module corresponding to a node representing a target digital human; acquiring a facial landmark array from a digital human node array of the first node description module; and on the basis of the facial landmark array, acquiring description information of a facial landmark of the target digital human.

Description

Method and apparatus for parsing scene description files

[0001] This application claims priority to Chinese patent applications filed on July 9, 2024, No. 202410916902.3; No. 202410916879.8; No. 202410916908.0; No. 202410917302.9; and No. 202410917292.9, all of which are incorporated herein by reference. Technical Field

[0002] Some embodiments of this application relate to the field of video processing technology. More specifically, they relate to a method and apparatus for parsing scene description files. Background Technology

[0003] Currently, some standards define default facial landmarks for digital human models in Scene Description Files (SDs), indicating that these landmarks can be used to implement digital human animation and related processing. However, the scene description files extended based on gilt2.0 do not declare information related to facial landmarks. Therefore, immersive media scene description frameworks cannot currently use facial landmarks to implement digital human animation and related processing. Summary of the Invention

[0004] This application provides an exemplary embodiment of a method and apparatus for parsing scene description files, which solves the problem that the immersive media scene description framework cannot be used to implement digital human animation and related processing based on facial markers because the scene description file does not declare relevant information about facial markers.

[0005] In a first aspect, some embodiments of this application provide a method for parsing a scene description file, including: obtaining a first node description module from a node list of a scene description file of a three-dimensional scene to be rendered, wherein the first node description module is a node description module corresponding to a node representing a target digital human; obtaining a facial marker point array from a digital human node array of the first node description module; and obtaining description information of the facial marker points of the target digital human based on the facial marker point array.

[0006] As can be seen from the above technical solutions, the scene description file parsing method provided in some embodiments of this application first obtains the first node description module corresponding to the node representing the target digital human from the node list of the scene description file of the 3D scene to be rendered, then obtains the facial marker point array from the digital human node array of the first node description module, and obtains the description information of the facial marker points of the target digital human based on the facial marker point array. Since some embodiments of this application can obtain the node description module corresponding to the target digital human from the scene description file, obtain the facial marker point array corresponding to the target digital human from the node description module corresponding to the target digital human, and obtain the description information of the facial marker points of the target digital human based on the facial marker point array corresponding to the target digital human, the embodiments of this application can parse the scene description file to obtain the description information of the facial marker points of the target digital human, and realize the animation and related processing of the target digital human based on the description information of the facial marker points of the target digital human. Therefore, some embodiments of this application can solve the problem that the scene description file does not declare the relevant information of the facial marker points, which leads to the immersive media scene description framework being unable to realize digital human animation and related processing based on the facial marker points. Attached Figure Description

[0007] Figure 1 shows a schematic diagram of the structure of the immersive media scene description framework in some embodiments of this application;

[0008] Figure 2 shows a schematic diagram of how skeletal points drive the rotation of skin apex in some embodiments of this application;

[0009] Figure 3 illustrates a schematic diagram of an animation based on facial blending shapes in some embodiments of this application;

[0010] Figure 4 shows a schematic diagram of the index values ​​of facial marker points in some embodiments of this application;

[0011] Figure 5 shows a schematic diagram illustrating the application scenarios of facial markers in some embodiments of this application;

[0012] Figure 6 shows a schematic diagram of the structure of the scene description file in some other embodiments of this application;

[0013] Figure 7 shows a schematic diagram of the structure of the scene description file in some other embodiments of this application;

[0014] Figure 8 shows a schematic diagram of a digital human pipeline in some embodiments of this application;

[0015] Figure 9 shows a flowchart of the steps of the scene description file parsing method in some embodiments of this application; Detailed Implementation

[0016] Some embodiments of this application relate to a scene description framework for immersive media. Referring to the immersive media scene description framework shown in FIG1, in order to enable the display engine 11 to focus on media rendering, the immersive media scene description framework decouples media file access and processing from media file rendering, and designs a Media Access Function (MAF) 12 to be responsible for media file access and processing functions. A Media Access Function Application Programming Interface (API) is also designed, and the display engine 11 and the Media Access Function 12 interact with each other through the Media Access Function API. The display engine 11 can issue instructions to the Media Access Function 12 through the Media Access Function API, and the Media Access Function 12 can also request instructions from the display engine 11 through the Media Access Function API.

[0017] The general workflow of an immersive media scene description framework may include: 1) Display engine 11 obtains the scene description file provided by the immersive media service provider. 2) Display engine 11 parses the scene description file, obtains the access address of the media file, the attribute information of the media file (media type and encoding / decoding parameters, etc.), and the format requirements of the processed media file, and calls the media access function API to pass all or part of the information obtained from parsing the scene description file to media access function 12. 3) Media access function 12, based on the information passed by display engine 11, requests to download the specified media file from the media resource server or obtains the specified media file from the local machine, and establishes a corresponding pipeline for the media file. Subsequently, the media file is processed in the pipeline through decapsulation, decryption, decoding, post-processing, etc., to convert the media file from the encapsulated format to the format specified by display engine 11. 4) Media access function 12 stores the output data obtained after all processing in the specified cache. 5) Display engine 11 reads the fully processed data from the specified cache and renders the media file based on the data read from the cache.

[0018] With the rapid development of immersive media technologies and the increasing acceptance of online work methods, immersive media technologies are becoming more widespread in people's work and lives. Compared to relying on large display screens to enhance immersion, Virtual Reality (VR) / Augmented Reality (AR) devices have the advantage of conforming to human physiological structure and can create vivid and realistic scenes through binocular stereoscopic vision. Therefore, VR / AR devices are more likely to serve as the hardware foundation for immersive media in the future. A key aspect of VR / AR technology is user representation, or simply avatar. When users wear VR / AR devices, they expect their real or virtual image to appear in the virtual scene presented by the headset and to move synchronously with their movements in the virtual scene. This is a crucial function of avatars and a vital element in enhancing immersion.

[0019] In the field of computer graphics (CG-based), a digital human is represented by the static geometric topology of a 3D mesh model, consisting of geometric points and a skeleton. The dynamic attributes of a digital human can be divided into two categories: limb movements, such as walking, running, jumping, turning, and picking up objects; and facial expression movements, such as smiling, frowning, and widening eyes. Limb movements have lower requirements for precision, while facial expression movements require far greater accuracy and complexity.

[0020] In glTF 2.0-based 3D media resource formats, animations are defined and implemented using syntax elements within the animation description module. As an example, the following is a JSON example of an animation description module:

[0021] The animation description module uses the node index syntax element and its value ("node":1) to indicate that the node to be animated is the node described by the second node description module in the node list. It uses the path syntax element and its value ("path":"translation") to indicate that the animation method is translation. It uses the syntax elements in "samplers" to indicate the keyframe data of the animation. Specifically, it uses the input syntax element and its value ("input":4) to indicate the input keyframe, uses the interpolation syntax element and its value ("interpolation":"LINEAR") to indicate the interpolation method of the keyframe, and uses the output syntax element and its value ("output":5) to indicate the interpolation data at the corresponding time.

[0022] The dynamic attributes of limbs are achieved through skinning to animate the body. Skinning establishes the relationship between skeletal points (joints) and skin vertices. Each skeletal point is associated with many skin vertices, and the influence weight of each skin vertex is different. Through this relationship, the overall movement of the digital human can be controlled by controlling the rotation vectors of the skeletal points. That is, in the glTF 2.0-based 3D media resource format, by setting the animation type of "path" in "target" to "rotation", and finding the rotation information data of the corresponding nodes (joints) to be animated through "samplers", the motion information of the skeleton is obtained. Then, through skinning technology, the skeleton drives the skin animation, thereby realizing the animation of the digital human.

[0023] For example, referring to Figure 2, the skeleton point 41 is associated with skin vertices 42 to 49. When the rotation vector of the skeleton point 41 is rotation1 = [0.0, 0.0, 0.0, 0.1], the rotation vectors rotation2 to rotation9 of the skin vertices 42 to 49 can be calculated based on the association between the skeleton point 41 and the skin vertices 42 to 49. Then, the skin vertices 42 to 49 are rotated according to the rotation vectors rotation2 to rotation9 of the skin vertices 42 to 49 respectively.

[0024] Facial dynamic attributes refer to facial animation-related attributes such as deformation and expression changes. Because facial skin changes are subtle and detailed, animate them using the same skeletal skinning method as the body, which would be redundant and cumbersome. Therefore, CG tools and glTF 2.0 both support animation based on facial blend shapes. A blend shape refers to a deformation relative to a base shape, usually represented as vertex displacements. Multiple blend shapes can be defined for the face, each considered an "expression base." Facial skin deformation can be achieved by combining different blend shapes and their corresponding weights. In other words, in a glTF 2.0-based 3D media resource format, by setting the "path" animation style in the "target" to "weights," and using "samplers" to find the blend shape weights keyframe data of the mesh under the node to be animated, facial animation is achieved by combining the weights of each blend shape. For example, for a smiley face (morph target 1) and a normal face (base shape), an animation of the transition between the smiley face and the normal face can be generated by generating a blend shape.

[0025] For example, as shown in FIG3, a transition animation 53 between facial expressions 51 and 52 can be generated by blend shape based on the weights of facial expressions 51 and 52, and a transition animation 55 between facial expressions 51 and 54 can also be generated by blend shape based on the weights of facial expressions 51 and 54.

[0026] Another facial animation method supported by CG tools and glTF 2.0 is the facial landmark method. Facial landmarks typically refer to key feature points in a facial image, such as the positions of the eyes, nose, and mouth. The positions of these key points can be represented by a set of coordinates, the number of which depends on the specific task. Figure 4 shows Morgan_landmarks as defined in Annex H of ISO / IEC 23090-14 Amd2. As shown in Figure 4, Morgan_landmarks defined in Annex H of ISO / IEC 23090-14 Amd2 includes 68 facial landmarks, with index numbers ranging from 0 to 67. The semantic meaning of a facial landmark can be determined based on its index number. For example, if the index number of a facial landmark is 36, its semantic meaning is the right corner of the right eye. Another example: if the index number of a facial landmark is 14, its semantic meaning is the left earlobe.

[0027] Currently, the method of reconstructing dynamic facial expressions of digital humans using facial markers is quite prominent in the relevant technical field. The overall process of the facial marker method may include the following steps (1) to (4):

[0028] (1) Predefine a neutral digital human face 3D model and predefine a set of facial markers on the model.

[0029] The neutral digital human facial 3D model can be a static 3D model, whose topology can be a 3D mesh or a 3D point cloud, etc. Facial markers typically cover spatial locations with rich variations and significant impact on facial expressions, such as the eyes, eyebrows, mouth, nose, and jaw. Currently, there is no uniform rule for the total number of facial markers, but 68 points are commonly used. Of course, 21, 29, 98, 106, and 186 points can also be used. The larger the total number of facial markers, the more accurate the facial markers will be, but the corresponding data volume and computational load will also be greater. Therefore, in practical applications, the number of facial markers can be set according to the accuracy requirements of the facial 3D model and the performance of the relevant equipment.

[0030] (2) Obtain the input data at the target time and obtain a set of motion vectors of facial markers based on the input data.

[0031] The types of input data and the methods of processing it are highly diverse. For example, when the input data is depth video captured by an RGBD camera, the motion vectors of facial markers can be obtained by using computer vision and neural network methods to locate the facial markers. For instance, using a facial marker extraction network, the spatial coordinates of the facial markers can be obtained, and then the difference between these coordinates and those from previous moments can be used to obtain the motion vectors of the facial markers at the target moment. When the input data is text representing the emotions of a digital human, the motion vectors of the facial markers can be obtained by using a facial expression blending model. This involves first using a weighted superposition of basic expressions to obtain a realistic complex expression, and then locating the facial markers and calculating the motion vectors on this complex face.

[0032] (3) Using the motion vectors of this set of facial markers, obtain the motion vectors of all vertices in the digital human face 3D model.

[0033] When considering the amount of data and the number of points, facial markers are equivalent to sparse data, while all vertices of the static 3D model of a digital human face are equivalent to dense data. The mapping of motion vectors from facial markers to all vertices of the facial model is a mapping from sparse data to dense data. This step can use computer vision and neural network methods, such as using motion diffusion networks, to map the motion vectors of sparse data to the motion vectors of dense data.

[0034] (4) Superimpose the motion vectors of all vertices onto the neutral 3D model of the digital human face to obtain the data of the digital human's dynamic facial expression at the target time.

[0035] The process of superimposing vertex motion vectors onto a neutral 3D digital facial model varies depending on the topology of the static 3D model. When the topology is a 3D point cloud, the motion vector of each point is directly summed with the spatial coordinates of the corresponding point in the 3D point cloud, and the resulting spatial coordinates are the superimposed result. Additionally, the attribute information attached to the 3D point cloud needs to be mapped; that is, the spatial coordinates of the points in the point cloud change, but the attribute information remains unchanged. When the topology is a 3D mesh, the motion vector of each point is directly summed with the spatial coordinates of the corresponding vertex in the 3D mesh, and the resulting spatial coordinates are the superimposed result. Furthermore, the attribute information attached to the 3D mesh needs to be mapped one-to-one; the attribute information remains unchanged, and the connection relationships between vertices remain unchanged. That is, vertices that were previously connected still have the same connection relationships after coordinate transformation.

[0036] Through the above steps (1) to (4), the data of the digital human's dynamic facial expressions at the target time can be obtained. By continuously repeating the above process, the data of the digital human's dynamic facial expressions at each time can be obtained. By using the data from these different times to drive the dynamic neutral digital human facial 3D model, the reconstruction and mapping of the digital human's dynamic facial expressions can be realized. In some embodiments, after the above step (4), the dynamic facial expressions of the neutral digital human can also be mapped onto the stylized digital human face, making the visual experience more realistic.

[0037] In the scene description framework of some embodiments of this application, the 3D scene service provider needs to provide a scene description file to the display engine, which then parses the scene description file. This scene description file includes, but is not limited to: description information of the entire 3D scene, description information of the digital human in the 3D scene, and information such as the location and index of facial marker points of the digital human in the 3D scene.

[0038] Furthermore, facial landmarks can be used to implement digital human animation and related processing, and play an important role in multiple fields. Figure 5 illustrates a scenario for implementing digital human animation and related processing based on facial landmarks. As shown in Figure 5, application scenarios for implementing digital human animation and related processing based on facial landmarks include: 71. Face Tracking: Accurate feature localization helps extract facial descriptive features, improving recognition accuracy. They can also be used for face alignment, ensuring that face images are standardized during preprocessing. 72. Face Retargeting: Facial landmark algorithms locate key feature points of the original and target faces, such as the positions of eyes, nose, and mouth. Using the located facial landmarks, the deformation mapping relationship between the source and target faces can be calculated. After determining the deformation mapping relationship, the texture (i.e., pixel information) of the source face can be mapped onto the geometry of the target face. 73. Face Alignment: Face alignment is a technique that locates the coordinates of key facial features on a face. Its goal is to accurately align and standardize the positions of facial features in a face image. It can also enable downstream tasks such as "makeup" 731, "adding props" 732, and "expression recognition and modification" 733. 74. Face Animation: Establishes the relationship between facial markers and the vertices of a 3D mesh model. Based on the new positions of the facial markers, a deformation field is calculated. This deformation field describes the mapping relationship from the original 3D mesh model to the deformed 3D mesh model, thus the deformation of the 3D mesh model is obtained from the change in facial markers. 75. Others: Other application scenarios for digital human animation and related processing based on facial markers.

[0039] Referring to Figure 6, which is a schematic diagram of the scene description file structure in some embodiments of this application, the scene description file may include MPEG media (MPEG_media) 801, a scene description module (scene) 802, and a node description module (node) 803. The node description module (node) 803 may include a digital human node array ("MPEG_node_avatar":{}) 8031. In some embodiments of this application, a digital human in a three-dimensional scene will be represented by a node, described by the corresponding node description module (node), and multiple levels of child nodes will be attached under this node. The node description modules corresponding to the child nodes will describe the specific parts of the digital human's body layer by layer. Through the hierarchical structure, the deepest unit in the upper body can be each finger joint. For example: Using the buttocks as a node to represent a digital human (layer 1), the node representing the digital human includes three child nodes: spine, left thigh, and right thigh (layer 2); selecting the child node representing the spine and continuing to extend it deeper, the child node representing the spine includes a child node representing the chest (layer 3); selecting the child node representing the chest and continuing to extend it deeper, the child node representing the chest includes a child node representing the upper chest (layer 4); selecting the child node representing the upper chest and continuing to extend it deeper, the child node representing the upper chest includes three child nodes representing the left shoulder, right shoulder, and neck (layer 5); selecting the child node representing the neck and continuing to extend it deeper, it includes a child node representing the head (layer 6).

[0040] In the node and child node hierarchy describing the specific parts of the digital human's body, each node and child node may have one or more 3D meshes attached to it, serving as 3D models of these body parts. Furthermore, using the displacement, rotation, scaling, and other parameters of the nodes and child nodes, these 3D models of body parts can be realistically pieced together to form a complete 3D model of the digital human.

[0041] Whether a node description module includes the extension "MPEG_node_avatar":{}" indicates whether the node corresponding to that node is used to represent a digital human. Specifically, if a node description module in a scene description file includes the extension "MPEG_node_avatar":{}", then it is determined that the node corresponding to that node description module represents a digital human.

[0042] It should be noted that in the scene description file, a digital human is represented by a node, and multiple levels of child nodes can be attached to this node. The specific parts of the digital human's body are described layer by layer by the node description module corresponding to the child nodes. In order to avoid information redundancy, in some embodiments of this application, only the digital human node array ("MPEG_node_avatar":{}) is added to the node description module corresponding to the top-level node representing the digital human, and the digital human node array is not added to the node description module corresponding to the child nodes attached to it.

[0043] In some embodiments, when a node represents a digital human, a digital human node array ("MPEG_node_avatar":{}) can be added to the extension list ("extensions":{}) of the node description module corresponding to that node. As shown in Figure 6, the digital human node array ("MPEG_node_avatar":{}) 8031 ​​in the node description module 803 may include: an active identifier syntax element ("isAvatar") 80311, a digital human type syntax element ("type") 80312, and a mapping list ("mappings":[]) 80313.

[0044] The active identifier syntax element ("isAvatar") in the digital human node array ("MPEG_node_avatar":{}) of the node description module (node) can be used to indicate whether the digital human represented by the node corresponding to the node description module (node) is active (whether the digital human needs to be rendered during the rendering process, and whether the display engine, media access function, etc. in the scene description box need to process the relevant data and information from the digital human).

[0045] In some embodiments, indicating whether the digital person represented by the node corresponding to the node description module is active via the activity identifier syntax element ("isAvatar") may include: determining that the digital person represented by the node corresponding to the node description module is active when the value of the activity identifier syntax element ("isAvatar") is a first value, and determining that the digital person represented by the node corresponding to the node description module is inactive when the value of the activity identifier syntax element ("isAvatar") is a second value. In some embodiments, the first value and the second value may be true and false, respectively. In other embodiments, the first value and the second value may be 1 and 0, respectively.

[0046] In some embodiments, the data type of the value of the active identifier syntax element ("isAvatar") can be boolean.

[0047] The digital human type syntax element ("type") in the digital human node array ("MPEG_node_avatar":{}) of the node description module (node) can be used to indicate the representation scheme of the digital human represented by the node corresponding to the node description module (node).

[0048] In some embodiments, the digital human type syntax element ("type") uses a Uniform Resource Name (URN) to describe the representation scheme of the digital human. The representation scheme of a digital human can be understood as the technical solution / architecture to which the digital human belongs. This technical solution / architecture defines the media format / type and other information of all body components of the digital human. Using this information, media access functions can establish corresponding pipelines for the reconstruction / driving of the corresponding digital human body components, or for the reconstruction / driving of the entire digital human body. For example, when the digital human represented by the node corresponding to the node description module (node) is an MPEG reference digital human, the value of the digital human type syntax element ("type") can be set to the Uniform Resource Name of the MPEG reference digital human. The URN of the MPEG reference digital human is urn:mpeg:sd:2023:avatar. Therefore, when the digital human represented by the node corresponding to the node description module (node) is an MPEG reference digital human, the digital human type syntax element and its value are: "type":"urn:mpeg:sd:2023:avatar".

[0049] In some embodiments, the data type of the value of the digital human type syntax element ("type") can be string.

[0050] As shown in Figure 6, the mapping list ("mappings":[]) 80313 may include: a part name syntax element ("path") 803131 and a node index syntax element ("node") 803132. The mapping list ("mappings":[]) in the digital human node array ("MPEG_node_avatar":{}) of the node description module (node) describes the mapping between digital human child nodes and the hierarchical standard names of digital human body parts. The data type of the mapping list ("mappings":[]) is an array containing two syntax elements: a part name syntax element ("path") and a node index syntax element ("node"). The part name syntax element ("path") can be used to describe the standard name of the digital human body part corresponding to the node / child node of the node description module, such as " / full_body / upper_body / head / face / eye_right", etc. The node index syntax element ("node") can be used to index the nodes that describe the body parts identified by the part name syntax element ("path"), so that the body part names correspond to the nodes.

[0051] In summary, the syntax elements in the digital human node array ("MPEG_node_avatar":{}) of the node description module (node) are shown in Table 1 below:

[0052] Table 1

[0053] The syntax elements in the mapping list ("mappings":[]) of the digital human node array ("MPEG_node_avatar":{}) of the node description module (node) are shown in Table 2 below:

[0054] Table 2

[0055] In some embodiments, the syntax elements in the digital human node array ("MPEG_node_avatar":{}) of the node description module (node) may not include the active identifier syntax element ("isAvatar"). When the syntax elements in the digital human node array ("MPEG_node_avatar":{}) of the node description module (node) do not include the active identifier syntax element ("isAvatar"), the syntax elements in the digital human node array ("MPEG_node_avatar":{}) of the node description module (node) are shown in Table 3 below:

[0056] Table 3

[0057] Figure 6 shows a scene description file that includes the definition of a digital human node array ("MPEG_node_avatar":{}) and provides a related reference model. The node description module, including the digital human node array and the reference model, provides a standardized way to define the geometry and semantics of the digital human's geometric description. Furthermore, it can include definitions of interaction between the digital human and the scene. However, it currently does not describe how to animate and encode the digital human; the animation of the digital human is based on the animation description module in the scene description file. The animation description module indexes the node description module through internal syntax elements to define combinations of local translation, rotation, and scaling transformations for the corresponding node, transforming or defining the weights of the node's morph targets.

[0058] Currently, relevant technical standards define the facial landmarks of the default digital human model in scene description files, and indicate that these landmarks can be used to implement digital human animation and related processing. However, scene description files do not declare information related to facial landmarks; therefore, scene description frameworks cannot currently implement digital human animation and related processing based on facial landmarks. In view of this, some embodiments of this application further propose the following technical solutions:

[0059] A facial marker array ("landmarks":{}) is added to the digital human node array ("MPEG_node_avatar":{}) in the node description module (node). The content of the facial marker array defines the level type of the facial markers, the accessor for accessing location coordinates, and the accessor for accessing index information. In some embodiments, the data type of the content of the facial marker array ("landmarks":{}) is a data object. In some embodiments, the usage of the facial marker array ("landmarks":{}) is not mandatory. In some embodiments, the facial marker array ("landmarks":{}) has no default value.

[0060] Based on the above scheme, the syntax elements in the digital human node array ("MPEG_node_avatar":{}) of the node description module (node) are shown in Table 4 or Table 5 below:

[0061] Table 4

[0062] Table 5

[0063] In some embodiments, the face marker array ("landmarks":{}) includes a level type syntax element ("level_type"), and the number and type of face markers are declared through the level type syntax element ("level_type").

[0064] In some embodiments, the data type of the value of the level type syntax element ("level_type") is integer. In some embodiments, the usage of the level type syntax element ("level_type") is optional. In some embodiments, the default value of the level type syntax element ("level_type") is 0.

[0065] In some embodiments, the facial landmark array ("landmarks":{}) further includes an index syntax element ("indices"), which declares the index value of the accessor description module corresponding to the accessor used to access the index information of the facial landmarks. This allows the scene description framework to access the index information of the facial landmarks based on the declaration of the index syntax element, and to obtain the semantic information of the facial landmarks based on the index information. In some embodiments, the data type of the value of the index syntax element ("indices") is integer. In some embodiments, the usage of the index syntax element ("indices") is mandatory. In some embodiments, the index syntax element ("indices") has no default value.

[0066] In some embodiments, the face marker array ("landmarks":{}) further includes a position coordinate syntax element ("position"), which declares the index value of the accessor description module corresponding to the accessor used to access the index information of the face markers, thereby enabling the scene description framework to access the position coordinates of the face markers based on the declaration of the position coordinate syntax element. In some embodiments, the data type of the value of the position coordinate syntax element ("position") is integer. In some embodiments, the usage of the position coordinate syntax element ("position") is mandatory. In some embodiments, the position coordinate syntax element ("position") has no default value.

[0067] Based on the above scheme, the syntax elements in the face marker array ("landmarks":{}) of the digital human node array ("MPEG_node_avatar":{}) of the node description module (node) may include one or more of the elements in Table 6 below:

[0068] Table 6

[0069] In some embodiments, the correspondence between the value of the level type syntax element ("level_type") and the number and type of facial markers can be shown in Table 7 below:

[0070] Table 7

[0071] Referring to Figure 7, which is a schematic diagram of the structure of a scene description file supporting the definition of facial marker points provided in some embodiments of this application, the scene description file shown in Figure 6 further includes, based on the scene description file shown in Figure 6:

[0072] The facial marker array ("landmarks":{})80314 is set in the digital human node array ("MPEG_node_avatar":{})8031 in the node description module (node) 803. The facial marker array ("landmarks":{})80314 may include: a level_type syntax element ("level_type")803141, an index syntax element ("indices")803142, and a position coordinate syntax element ("position")803143.

[0073] As shown in Figure 6 or 9, the scene description file may include an accessor description module (accessor) 805, a buffer slice description module (bufferView) 806, and a buffer description module (buffer) 807. The method for describing the output data of the media access function through the accessor description module (accessor) 805, the buffer slice description module (bufferView) 806, and the buffer description module (buffer) 807 may include: a mesh description module (mesh) 804 representing the dynamic 3D model of a digital human face in the described 3D mesh points to a specific accessor description module (accessor) 805; this accessor description module (accessor) 805 then points to the corresponding buffer slice description module (bufferView) 806; this buffer slice description module (bufferView) 806 then points to the corresponding buffer description module (buffer) 807. This description method enables the storage and indexing of digital human facial expression model data. Specifically, the buffer described by the buffer description module (buffer) 807 can be directly read by the display engine, and the stored data is data that can be directly used for rendering. In some embodiments of this application, the data stored in the buffer includes the index information and position coordinates of digital human facial markers, as well as a dynamic 3D model of the digital human. The buffer slice description module (bufferView) 806 is responsible for slicing the data in the buffer, which can be achieved using two parameters: the starting byte offset (byteOffset) and the byte length (byteLength). The accessor description module (accessor) is responsible for adding additional information to the data in the buffer slice, such as the data type, the quantity of a certain type of data, and the numerical range of a certain type of data. The mesh description module (mesh) 804, used to describe the 3D mesh representing the digital human face, will point to the accessor description module (accessor) 805 to retrieve the dynamic 3D model of the digital human to be rendered. The facial marker array, which can be used to describe the descriptive information of the digital human facial markers, will point to the accessor description module (accessor) 805 to retrieve the index information and position coordinates of the digital human facial markers.

[0074] Since the data related to digital facial markers is a time-varying medium, the accessor used to access this data needs to be capable of accessing time-varying media. Therefore, the accessor description module (accessor) 805 describing the accessor used to access the data related to digital facial markers should include an MPEG ring buffer ("MPEG_accessor_timed":{}) extension, and use the MPEG ring buffer to transform the described accessor into a time-varying accessor. When the accessor description module (accessor) 805 includes an MPEG ring buffer ("MPEG_accessor_timed":{}), the accessor description module (accessor) 805 contains two buffer slice index syntax elements ("bufferView"), one of which is outside the MPEG ring buffer, and the other is inside the MPEG ring buffer. The buffer slice index syntax element ("bufferView") outside the MPEG circular buffer will point to the relevant data of digital human facial markers or the dynamic 3D model of the digital human. The buffer slice index syntax element ("bufferView") inside the MPEG circular buffer will point to the header of the time-varying accessor. The header of the time-varying accessor, also known as the parameters of the time-varying accessor, describes the indexing method, data type, and data quantity of the time-varying data stored in the time-varying accessor. These parameters change over time and are therefore appended to the time-varying accessor as a header. The header may include: the buffer slice index syntax element ("bufferView") outside the MPEG circular buffer, the data type syntax element ("componentType"), and the value of the accessor type syntax element ("type") at different times.

[0075] Since the data related to digital facial markers is a time-varying medium, the buffer used to cache this data needs to be capable of caching time-varying media. Therefore, the buffer description module (buffer) 807, which describes the buffer used to cache the data related to digital facial markers, should include an MPEG circular buffer extension ("MPEG_buffer_circular":{}) and transform the buffer into a circular buffer using the MPEG circular buffer extension. The MPEG circular buffer ("MPEG_buffer_circular":{}) can contain syntax elements such as the media index syntax element ("media"), the track index syntax element ("tracks"), and the number of stages syntax element ("count"). The media index syntax element ("media") and its value will point to the media file described in the MPEG media. The track index syntax element ("tracks") and its value can be used to describe the track information of the source data of the data cached in the buffer. The number of stages syntax element ("count") and its value can be used to specify the number of storage stages in the circular buffer.

[0076] Through the above two extensions, different stages in the ring buffer store data of the time-varying medium at different times, and the different storage stages are separated in a slice manner by cache slicing. The time-varying accessor will access data from different cache slices (storage stages) at different times.

[0077] As shown in Figure 6 or 9, the scene description file may include a camera description module (camera) 808, a material description module (material) 809, a texture description module (texture) 810, a sampler description module (sampler) 811, a texture mapping description module (image) 812, a skin description module (skin) 813, and an animation description module (animation) 814.

[0078] For example, the scenario description file provided in the above embodiments will be described below with reference to a specific scenario description file.

[0079] The curly braces between lines 1 and 150 in the example above contain the main content of the scene description file provided in the above embodiment. The scene description file may include: digital asset description module (asset), extensionUsed list (extensionUsed), MPEG media (MPEG_media), scene declaration (scene), scene list (scenes), node list (nodes), mesh list (meshes), accessor list (accessors), bufferview list (bufferViews), and buffer list (buffers). The content of each part and the information contained in each part from the perspective of parsing are explained below.

[0080] 1. Digital Asset Description Module (asset): The digital asset description module is represented by lines 2-4. The "version": "2.0" in line 3 of the digital asset description module indicates that the scene description file is written based on glTF version 2.0, which is also the reference version for the scene description standard. From a parsing perspective, the display engine can determine which parser to use to parse the scene description file based on the digital asset description module.

[0081] 2. Extension Used: Lines 6-11 show the extension used list. Since the extension used list includes four extensions: MPEG media, MPEG buffer circular, MPEG accessor timed, and MPEG node avatar, it can be determined that the scene description file uses these four MPEG extensions. From a parsing perspective, the display engine can infer in advance from the content of the extension used list that the subsequent parsing involves these four MPEG extensions: MPEG media, MPEG buffer circular, MPEG accessor timed, and MPEG node avatar.

[0082] 3. MPEG Media (MPEG_media): Lines 13-33 define the MPEG media. The MPEG media declares the media files contained in the 3D scene. Line 19, "mimeType":"text / plain", indicates the file type of the container file corresponding to the first media file, and line 20, "uri":"texts / indices.txt", indicates the access address of the first media file. Line 27, "mimeType":"text / plain", indicates the file type of the container file corresponding to the second media file, and line 28, "uri":"texts / indices.txt", indicates the access address of the second media file. From a parsing perspective, the display engine can determine the existence of two media files in the 3D scene to be rendered by parsing the MPEG media, and learn the methods for accessing and parsing these media files.

[0083] 4. Scene Declaration: The scene declaration is on line 35. Since a scene description file can theoretically include multiple 3D scenes, the above scene description file first indicates through "scene":0 on line 35 that the scene described by the scene description file is the first 3D scene in the scene list, that is, the scene described by the scene description module enclosed in the curly braces on lines 38 to 41.

[0084] 5. Scene List (scenes): The scene list is on lines 37-41. The scene list contains only one curly brace, indicating that the scene description file contains one scene description module, and also that the scene description file contains only one 3D scene. Within the curly brace (scene description module), "nodes":[0] on line 39 indicates that the 3D scene contains only one node with index 0. From a parsing perspective, this scene list clarifies that the entire scene description framework should select the first 3D scene in the scene list for subsequent processing and rendering, clarifies the overall structure of the 3D scene, and points to the more detailed next-level node description module.

[0085] 6. Node List (nodes): The node list is located in lines 43-68. The node list contains only one curly brace, indicating that it includes only one node description module (node). This 3D scene has only one node, and this node is the same node as the node with an index value of 0 contained in the scene description module. The two are associated through an index. In this unique node description module, line 45, "mesh":"0", indicates that the 3D mesh attached to the node is the 3D mesh described by the first mesh description module in the mesh list, which corresponds to the mesh description module of the next layer. Line 47, "MPEG_node_avatar", indicates that the node described by this node description module represents a digital human. Line 48, "isAvatar":True, indicates that the digital human represented by the node described by this node description module is active. Line 49, "type":"urn:mpeg:sd:2023:avatar", indicates that the representation scheme of the digital human represented by the node described by this node description module is the MPEG reference digital human represented by urn:mpeg:sd:2023:avatar. Line 52, "path":"full_body / upper_body / arm_left", indicates that the hierarchical standard name of the body part of the digital human in this mapping relationship is the left arm. Line 53, "node":0", indicates that the index value of the node description module corresponding to this mapping relationship is 1. The path in line 56, "path":"full_body / upper_body / arm_right", indicates that the hierarchical standard name of the digital human body part in this mapping relationship is the right arm. The node in line 57, "node":2", indicates that the index value of the node description module corresponding to the node in this mapping relationship is 2. Based on the level type syntax element and its value "level_type":1 in the face marker array ("landmarks":{}), the type and number of face markers can be determined to be the 68 face markers provided in MOGAN. The index syntax element and its value "indices":0 indicate that the index information of the face markers can be accessed through the accessor corresponding to the first accessor description module in the accessor list ("accessors":[]). The position coordinate syntax element and its value "position":1 indicate that the position coordinates of the face markers can be accessed through the accessor corresponding to the second accessor description module in the accessor list ("accessors":[]).From a parsing perspective, this node list indicates that the unique node in the 3D scene is attached to a 3D mesh, and that this 3D mesh is the 3D mesh described by the first mesh description module in the mesh list. This node represents a digital human, and the representation scheme of this digital human is the MPEG Reference Digital Human. The facial markers of this digital human are the 68 facial markers provided in MOGAN. The index information of the digital human's facial markers can be accessed through the accessor corresponding to the first accessor description module in the accessor list, and the position coordinates of the digital human's facial markers can be accessed through the accessor corresponding to the second accessor description module in the accessor list.

[0086] 7. Mesh List: The mesh list is located on lines 70-81. In the mesh description module, line 72, "primitives," indicates that the 3D mesh has primitives. Lines 75, "attributes," and 77, "mode," indicate the presence of attributes and modes within these primitives. Line 75, "position": 0, indicates that the 3D mesh has geometric coordinate data, and the accessor describing the accessor for accessing these coordinates is the first accessor in the accessor list. Furthermore, line 77, "mode": 4, confirms that the 3D mesh topology is a triangular mesh. From an analytical perspective, this description module determines the actual data types and topological type of the 3D mesh.

[0087] 8. Buffer List: The buffer list is located on lines 132-149. The buffer list contains two curly braces, indicating that the scene description file includes two buffer description modules, and the display of this 3D scene requires the use of two buffers. Both buffer description modules use the MPEG circular buffer extension ("MPEG_buffer_circular":{}), indicating that the buffers described by these two buffer description modules are circular buffers obtained using the MPEG extension. According to line 137 of the MPEG circular buffer ("MPEG_buffer_circular":{}), "media:0" indicates that the data source in the first circular buffer is the first media file declared within the MPEG media (MPEG_media), and according to line 145 of the MPEG circular buffer ("MPEG_buffer_circular":{}), "media:1" indicates that the data source in the second circular buffer is the second media file declared within the MPEG media (MPEG_media). From a parsing perspective, the buffer list maps declared media files in MPEG media (MPEG_media) to buffers, or in other words, allows buffers to reference previously declared but unused media files. It's important to note that the media files referenced here are unprocessed encapsulated files. These media files need to be processed by media access functions to extract directly usable rendering data, such as the 3D coordinates mentioned in the mesh description module (mesh) of the mesh list.

[0088] 9. Buffer Slice List (bufferViews): The buffer slice list is located on lines 113-130. Each buffer slice description module contains four parallel curly braces, indicating that the data of the declared media files in the MPEG media is divided into four buffer slices. The first curly brace (the first buffer slice description module) first points to the buffer description module with index 0, i.e., the first buffer description module in the buffer list. Then, the data slice range of the corresponding buffer slice is limited to bytes 0-272 by the two parameters: byte offset (0) and byte length (272). The information provided by the other buffer slice description modules is similar to that provided by the first buffer slice description module, and will not be described in detail here to avoid redundancy.

[0089] 10. Accessors List: The accessors list is located on lines 84-111. The accessors list contains two curly braces, indicating that this scene description file should only include two accessor description modules. The display of this 3D scene requires access to media data through these two accessors. Furthermore, both accessor description modules contain MPEG time-varying accessors ("MPEG_accessor_timed":{}), indicating that both accessors point to MPEG-defined time-varying media. Within the first set of curly braces (the first accessor description module), line 86's "bufferView":0 indicates that the accessor corresponding to this accessor description module should retrieve data from the cache corresponding to the first cache slice description module in the cache slice list ("bufferViews":[]). Line 88's "componentType":5125 indicates that the data type of the accessor corresponding to this accessor description module is an unsigned integer. Line 89's "type":"SCALAR" indicates that the accessor type of the accessor corresponding to this accessor description module is a string. Line 90's "Count":68 indicates that the number of data items the accessor corresponding to this accessor description module needs to access is 68. Line 94's "bufferView":2 indicates that the header information of the accessor corresponding to this accessor description module should be retrieved from the cache corresponding to the third cache slice description module in the cache slice list ("bufferViews":[]). Within the second set of curly braces (the second accessor description module), line 99's "bufferView..." The line `:1` indicates that the accessor corresponding to this accessor description module should obtain data from the cache corresponding to the first cache slice description module in the cache slice list ("bufferViews":[]). Line 101, `componentType":5126`, indicates that the data type of the accessor corresponding to this accessor description module is floating-point. Line 102, `type":"VEC3"`, indicates that the accessor type of the accessor corresponding to this accessor description module is a three-dimensional vector. Line 103, `count":3`, indicates that the number of data items the accessor corresponding to this accessor description module needs to access is 3. Line 107, `bufferView":3`, indicates that the header information of the accessor corresponding to this accessor description module should be obtained from the cache corresponding to the fourth cache slice description module in the cache slice list ("bufferViews":[]). From a parsing perspective, this accessor list completes the definition of the data required for rendering. For example, data types missing in the cache slice description modules are defined in the corresponding accessor description modules in the accessor list.

[0090] Referring to Figure 8, which is a schematic diagram of an Avatar Media Pipeline provided in some embodiments of this application, the Avatar Media Pipeline 100 may include an Avatar Landmarks Decoder 101 and an Avatar Reconstruction module 102. The workflow of the Avatar Media Pipeline 100 includes: after receiving the data format of the Avatar Landmarks sent by the display engine 11 via the Media Access Function API, firstly, initializing the Avatar Media Pipeline 100 to create a pipeline for processing media files corresponding to the Landmarks; secondly, obtaining the data stream (Landmarks stream) of the media files corresponding to the Landmarks from the cloud server; performing decoding and reconstruction operations through the Avatar Landmarks Decoder 101 and the Avatar Reconstruction module 102 in the created pipeline; converting the media files corresponding to the Landmarks into the required format; and writing the converted data into a cache through the cache management module 13 so that the display engine 11 can read the data and complete the rendering and display of the 3D scene and the Avatar in the 3D scene based on the read data.

[0091] Based on a digital human representation format that supports digital human models, digital human facial marker data can be extracted from a user's video capture by, for example, training a neural network (facial keypoint detection algorithm). This data can then be transmitted to the receiver as ISOBMFF track samples in a transmission format compatible with scene description files.

[0092] As mentioned above, ISOBMFF's Samples of a Track format is suitable for data that needs to be synchronized with the timeline in the media file; ISOBMFF's Items format can be used to store data that is not directly associated with the timeline, while the data related to digital facial markers needs to change and be updated dynamically and needs to be synchronized with the timeline. Therefore, in some embodiments of this application, ISOBMFF's Samples of a Track format is selected to store and transmit media files that include data related to digital facial markers.

[0093] When using the ISOBMFF Samples of a Track format to store and transmit media files containing data related to digital human facial markers, the ISOBMFF file can include two parts: a facial marker sample header (Landmarks sample entry) and a facial marker sample content (Landmarks sample format). The contents of the Landmarks sample entry and Landmarks sample format are described in detail below.

[0094] The Landmarks sample entry section defines the head information of the facial marker sample, including the number and type of facial markers. The Landmarks sample entry can be considered the metadata of the facial marker sample. The syntax of the Landmarks sample entry is as follows:

[0095] Among them, unsigned int(3) level_type specifies the number and type of facial landmarks used, as shown in Table 7 above. Different integer values ​​correspond to different numbers and types of facial landmarks; bit(5) reserved is a reserved bit that can be used for future expansion. It should be set to 0 when there is no expansion.

[0096] The Landmarks sample format section mainly defines the index information and location coordinates of digital human face markers.

[0097] In some embodiments, the syntax of the Landmarks sample format can be as follows:

[0098] Among them, unsigned int(32)sample_index specifies the index value of the facial marker sample, which can be used to distinguish different samples; unsigned int(32)landmarks_count specifies the number of facial markers contained in the facial marker sample; unsigned int(32)position_index[i] specifies the index information of the i-th facial marker in the facial marker sample, which can be used to uniquely identify or reference a specific facial marker; float(32)[3]position_value[i] specifies the floating-point three-dimensional position coordinates (x,y,z) of the i-th facial marker in the facial marker sample.

[0099] In other embodiments, the syntax of the Landmarks sample format may be as follows:

[0100] Among them, unsigned int(32)sample_index specifies the identifier of the facial marker sample, which can be used to distinguish different samples; unsigned int(32)landmarks_count specifies the number of facial markers contained in the facial marker sample; float(32)delta_x[i] specifies the floating-point position change of the i-th facial marker in the facial marker sample in the x-axis direction relative to the previous frame; float(32)delta_y[i] specifies the floating-point position change of the i-th facial marker in the facial marker sample in the y-axis direction relative to the previous frame; and float(32)delta_z[i] specifies the floating-point position change of the i-th facial marker in the facial marker sample in the z-axis direction relative to the previous frame.

[0101] In some embodiments, the precision value in LandmarksSampleHeader is conditional as follows: if the value of the time-varying syntax element (Immutable) in the MPEG time-varying accessor (MPEG_accessor_timed) of the accessor description module corresponding to the accessor is True, and there is no accessor slice syntax element (bufferView) in the MPEG time-varying accessor, then the value of the count syntax element (count) in the accessor description module is the number of face markers specified by unsigned int(3)level_type in the Landmarks sample entry; otherwise, it is obtained from the buffer slice (timed accessor information header) indicated by the accessor slice syntax element (bufferView) in the MPEG time-varying accessor.

[0102] In some embodiments, the LandmarksSample is processed as follows: 1. The sample_index is represented in the corresponding frame of the circular buffer. 2. The number of data that the landmarks_count needs to access as an accessor is represented in the time-varying accessor header field of the corresponding frame of the circular buffer. 3. For each face marker, position_index[i] should be represented in the corresponding frame of the circular buffer. 4. For each face marker, position_value[i] should be represented in the corresponding frame of the circular buffer, containing the specific position data of the face marker.

[0103] Some embodiments of this application also provide a method for parsing a scene description file. Referring to FIG9, the method for parsing a scene description file may include the following steps S121 to S123:

[0104] S121. Obtain the first node description module from the node list ("nodes":[]) of the scene description file of the 3D scene to be rendered. The first node description module is the node description module corresponding to the node representing the target digital human.

[0105] In some embodiments, the first node description module obtained from the scene description file may be as follows:

[0106] In some embodiments, the module for obtaining the target node description from the scene description file may also be as follows:

[0107] S122. Obtain the face marker array ("landmarks":{}) from the digital human node array ("MPEG_node_avatar":{}) of the first node description module.

[0108] Following the examples above, in some embodiments, the facial marker array ("landmarks":{}) obtained from the digital human node array ("MPEG_node_avatar":{}) of the first node description module can be as follows:

[0109] Following the examples above, in some embodiments, the facial marker array ("landmarks":{}) obtained from the digital human node array ("MPEG_node_avatar":{}) of the first node description module can also be as follows:

[0110] S123. Obtain the description information of the facial landmarks (landmarks) of the target digital human according to the facial landmark array ("landmarks":{}).

[0111] In this embodiment of the application, the description information of the facial landmarks of the target digital human may include at least one of the following: the level and number of the facial landmarks of the target digital human, the index value of the accessor description module corresponding to the accessor for accessing the index information of the facial landmarks of the target digital human, and the index value of the accessor description module corresponding to the accessor for accessing the position coordinates of the facial landmarks of the target digital human.

[0112] In some embodiments, the facial marker array ("landmarks":{}) may include a level type syntax element ("level_type") and the value of the level type syntax element. Step S123 (obtaining description information of the facial markers of the target digital human based on the facial marker array) may include: obtaining the number and type of facial markers of the target digital human based on the value of the level type syntax element ("level_type").

[0113] In some embodiments, obtaining the level type of the facial markers of the target digital human based on the value of the level type syntax element may include: obtaining the number and type of the facial markers of the target digital human based on the value of the level type syntax element and the mapping relationship shown in Table 7.

[0114] Based on the mapping relationship shown in Table 7 above, when the level type syntax element and its value are "level_type":0, the facial markers of the target digital human can be determined to be 68 facial markers provided by MOGAN; when the level type syntax element and its value are "level_type":1, the facial markers of the target digital human can be determined to be 21 facial markers provided by the AFLW dataset; when the level type syntax element and its value are "level_type":2, the facial markers of the target digital human can be determined to be 29 facial markers provided by the LFPW dataset; and when the level type syntax element and its value are "level_type":3, the facial markers of the target digital human can be determined to be 98 facial markers provided by the WFLW dataset.

[0115] In some embodiments, the facial landmark array ("landmarks":{}) may include an index syntax element ("indices") and the value of the index syntax element. Step S123 (obtaining the description information of the facial landmarks of the target digital human based on the facial landmark array) may include: obtaining the index value of the accessor description module corresponding to the first accessor in the accessor list ("accessors":[]) of the scene description file based on the value of the index syntax element ("indices"). Wherein, the first accessor is an accessor used to access the index information of the facial landmarks of the target digital human.

[0116] For example, the index syntax element ("indices") and its value "indices":2 can determine that the accessor description module corresponding to the first accessor is the third accessor description module in the accessor list ("accessors":[]).

[0117] In some embodiments, the facial marker array ("landmarks":{}) may include: a position coordinate syntax element ("position") and the value of the position coordinate syntax element. Step S123 (obtaining the description information of the facial markers of the target digital human based on the facial marker array) may include: obtaining the index value of the accessor description module corresponding to the second accessor in the accessor list ("accessors":[]) of the scene description file based on the value of the position coordinate syntax element ("position"). The second accessor is an accessor used to access the position coordinates of the facial markers of the target digital human.

[0118] For example, when the position coordinate syntax element ("position") and its value is "position":1, it can be determined that the accessor description module corresponding to the second accessor is the second accessor description module in the accessor list ("accessors":[]).

[0119] In some embodiments, the position coordinates of the facial markers of the target digital human obtained by accessing the second accessor are the absolute position coordinates of the facial markers of the target digital human in three-dimensional space.

[0120] In some embodiments, the position coordinates of the facial marker points of the target digital human obtained by accessing the second accessor are the relative position coordinates of the facial marker points of the target digital human in three-dimensional space. For example, the position coordinates of the facial marker points of the target digital human can be the offset between the position coordinates of the facial marker points of the target digital human in the current frame and the position coordinates of the facial marker points of the target digital human in the previous frame.

[0121] For example, the facial marker array ("landmarks":{}) obtained from the digital human node array ("MPEG_node_avatar":{}) of the first node description module is as follows:

[0122] Alternatively, the facial marker array ("landmarks":{}) obtained from the digital human node array ("MPEG_node_avatar":{}) of the first node description module is as follows:

[0123] Therefore, the facial marker description information that can be obtained by parsing the facial marker array ("landmarks":{}) may include: the facial markers of the target digital human are 68 facial markers provided by MOGAN; the index information of the target digital human can be accessed through the accessor corresponding to the first accessor description module in the accessor list ("accessors":[]); and the position coordinates of the target digital human can be accessed through the accessor corresponding to the second accessor description module in the accessor list ("accessors":[]).

[0124] The scene description file parsing method of some embodiments of this application first obtains the first node description module corresponding to the node representing the target digital human from the node list of the scene description file of the 3D scene to be rendered. Then, it obtains the facial marker point array from the digital human node array of the first node description module, and obtains the description information of the facial marker points of the target digital human based on the facial marker point array. Since the embodiments of this application can obtain the node description module corresponding to the target digital human from the scene description file, obtain the facial marker point array corresponding to the target digital human from the node description module corresponding to the target digital human, and obtain the description information of the facial marker points of the target digital human based on the facial marker point array, the embodiments of this application can parse the scene description file to obtain the description information of the facial marker points of the target digital human, and realize the animation and related processing of the target digital human based on the description information of the facial marker points of the target digital human. Therefore, the embodiments of this application can solve the problem that the scene description file does not declare the relevant information of the facial marker points, which leads to the immersive media scene description framework being unable to realize digital human animation and related processing based on the facial marker points.

[0125] The method for parsing scene description files in some embodiments of this application may further include: obtaining a digital human type syntax element ("type") and the value of the digital human type syntax element from the digital human node array ("MPEG_node_avatar":{}) of the first node description module; and obtaining the representation scheme of the target digital human based on the value of the digital human type syntax element ("type").

[0126] For example, when the digital human type syntax element and its value are "type":"urn:mpeg:sd:2023:avatar", the representation scheme of the target digital human can be determined as MPEG reference digital human based on the value of the digital human type syntax element (type).

[0127] The method for parsing scene description files in some embodiments of this application may further include the following steps (1) and (2):

[0128] Step (1): Obtain the mapping list ("mappings":[]) corresponding to the target number from the digital human node array ("MPEG_node_avatar":{}) of the first node description module.

[0129] For example, obtaining the mapping list ("mappings":[]) corresponding to the target digit from the digit human node array ("MPEG_node_avatar":{}) of the first node description module can be as follows:

[0130] Step 2: Based on the mapping list ("mappings":[]) corresponding to the target digital human, obtain the digital human body parts represented by each digital human component of the target digital human and the node description modules in the node list corresponding to each digital human component of the target digital human.

[0131] In some embodiments, obtaining the digital human body parts represented by each digital human component of the target digital human and the node description module in the node list corresponding to each digital human component of the target digital human based on the mapping list ("mappings":[]) corresponding to the target digital human may include the following steps 1) to 3):

[0132] Step 1) Obtain the mapping array corresponding to each digital human component of the target digital human from the mapping list ("mappings":[]) corresponding to the target digital human.

[0133] For example, the mapping array corresponding to one digital human component of the target digital human can be as follows:

[0134] Step 2) Obtain the digital human body parts represented by each digital human component of the target digital human based on the value of the part name syntax element ("path") in the mapping array corresponding to each digital human component of the target digital human.

[0135] For example, if the part name syntax element in the mapping array corresponding to a digital human component of the target digital human is "path":"full_body / upper_body / head / face", then the digital human body part represented by the digital human component can be determined to be the face based on the value of the part name syntax element in the mapping array corresponding to the digital human component.

[0136] Step 3) Based on the value of the node index syntax element ("node") in the mapping array corresponding to each digital human component of the target digital human, obtain the node description module in the node list corresponding to each digital human component of the target digital human.

[0137] For example, if the node index syntax element in the mapping array corresponding to a digital human component of the target digital human is "node":2, then the node description module corresponding to the digital human component can be determined to be the third node description module in the node list based on the value of the index syntax element in the mapping array corresponding to the digital human component.

[0138] The method for parsing the scene description file in some embodiments of this application may further include: determining whether the target digital human is an active digital human based on the value of the active identifier syntax element (isAvatar) in the digital human node array.

[0139] In some embodiments, determining whether the target digital person is an active digital person based on the value of the active identifier syntax element in the digital person node array of the target node description module may include: if the value of the active identifier syntax element in the digital person node array of the target node description module is 1 or true, then the target digital person is determined to be an active digital person; if the value of the active identifier syntax element in the digital person node array of the target node description module is 0 or false, then the target digital person is determined to be an inactive digital person.

[0140] For example, the node description module corresponding to the first node of the target digital human is shown below:

[0141] Therefore, according to "mesh":0 in line n+01, we know that the first node mounts the 3D mesh as the 3D mesh described by the first mesh description module in the mesh list; according to "isAvatar":True in line n+04, we know that the target digital human is an active digital human; according to "type":"urn:mpeg:sd:2023:avatar" in line n+05, we know that the representation scheme of the digital human represented by the node described by this node description module is MPEG reference digital human; according to "path":"full_body / upper_body / arm_left" in line n+08, we know that the target digital human includes a digital human component representing the left arm of the digital human; according to "node":1 in line n+09, we know that the digital human component representing the left arm is the node corresponding to the second node description module in the node list; according to line n+12... The path "full_body / upper_body / arm_left" indicates that the target digital human includes a digital human component representing the right arm. Based on line n+13's "node":1, the digital human component representing the right arm corresponds to the third node description module in the node list. Line n+17's "level_type":0 indicates that the target digital human's facial markers are the 68 facial markers provided by MOGAN. Line n+18's "indices":0 indicates that the accessor used to access the index information of the target digital human's facial markers is the accessor corresponding to the first accessor description module in the accessor list. Line n+19's "position":0 indicates that the accessor used to access the position coordinates of the target digital human's facial markers is the accessor corresponding to the second accessor description module in the accessor list.

[0142] The method for parsing scene description files in some embodiments of this application may further include the following steps 1 and 2:

[0143] Step 1: Obtain the target media description module from the media list ("media":[]) of the MPEG media ("MPEG_media":{}) of the scene description file. The target media description module is any media description module in the media list of the MPEG media of the scene description file.

[0144] Step 2: Obtain the description information of the target media file corresponding to the target media description module according to the target media description module.

[0145] In some embodiments, obtaining the description information of the target media file according to the target media description module may include at least one of the following steps 21 to 24:

[0146] Step 21: Obtain the name of the target media file based on the value of the media name syntax element ("name") in the target media description module.

[0147] For example, if the media name syntax element and its value in the target media description module are "name":"avatarThorax", then the name of the target media file can be determined to be: avatarThorax.

[0148] Step 22: Determine whether the target media file needs to be automatically played based on the value of the autoplay syntax element ("autoplay") in the target media description module.

[0149] In some embodiments, determining whether the target media file needs to be automatically played based on the value of the autoplay syntax element ("autoplay") in the target media description module may include: if the autoplay syntax element ("autoplay") in the target media description module and its value are "autoplay":true or "autoplay":1, then the target media file needs to be automatically played; and if the autoplay syntax element ("autoplay") in the target media description module and its value are "autoplay":false or "autoplay":0, then the target media file does not need to be automatically played.

[0150] Step 23: Determine whether the target media file needs to be played in a loop based on the value of the loop playback syntax element ("loop") in the target media description module.

[0151] In some embodiments, determining whether the target media file needs to be looped based on the value of the loop playback syntax element ("loop") in the target media description module may include: if the loop playback syntax element ("loop") in the target media description module and its value are "loop":true or "loop":1, then the target media file needs to be looped; and if the loop playback syntax element (loop) in the target media description module and its value are "loop":false or "loop":0, then the target media file does not need to be looped.

[0152] Step 24: Obtain the description information of each optional version of the target media file according to each optional description module in the optional list ("alternatives":[]) of the target media description module.

[0153] In some embodiments, step 24 above (obtaining description information of each optional version of the target media file according to each optional description module in the optional list of the target media description module) may include at least one of the following steps 241 to 244:

[0154] Step 241: Obtain the encapsulation format of the first optional version corresponding to the first optional description module based on the value of the media type syntax element ("mimeType") in the first optional description module.

[0155] Wherein, the first optional description module can be any optional description module in the optional list. Since the first optional description module can be any optional description module in the optional list, the description information of each optional version of the target media file can be obtained through the embodiments of this application.

[0156] For example, if the media type syntax element ("mimeType") in the optional description module corresponding to a certain optional version of the target media file has the value "mimeType":"application / mp4", then it can be determined that the container format of that optional version of the target media file is MP4.

[0157] Step 242: Obtain the first optional version of the Uniform Resource Identifier (URI) based on the value of the Uniform Resource Identifier syntax element ("uri") in the first optional description module.

[0158] For example, if the Uniform Resource Identifier syntax element and its value in a certain alternative description module in the alternative list ("alternatives":[]) of the target media description module is "uri":"https: / / www.example.com / avatarthorax", then the Uniform Resource Identifier of the alternative version corresponding to that alternative description module can be determined to be "https: / / www.example.com / avatarthorax".

[0159] Step 243: Obtain the track information of the first optional version based on the value of the first track index syntax element ("track") in the track array ("tracks":[]) of the first optional description module.

[0160] Step 244: Obtain the decoder type of the first optional version of the bitstream based on the value of the codecs element in the track array ("tracks":[]) of the first optional description module.

[0161] The scene description file parsing method in some embodiments of this application may include the following steps 3 and 4: Step 3: Obtain the target scene description module corresponding to the three-dimensional scene to be rendered from the scene list ("scenes":[]) of the scene description file. Step 4: Obtain the description information of the three-dimensional scene to be rendered according to the target scene description module.

[0162] In some embodiments, step 4 (obtaining the description information of the 3D scene to be rendered according to the target scene description module) may include: determining the index value of the node description module corresponding to each top-level node in the 3D scene to be rendered according to the index value declared in the node index list ("nodes":[]) of the target scene description module. For example, if the node index list of the target scene description module and its declared index value are "nodes":[0], it can be determined that the 3D scene to be rendered includes only one top-level node, and the node description module corresponding to the top-level node is the first node description module in the node list of the scene description file.

[0163] For example, if the node index list of the target scene description module and its declared index value is "nodes":[0,2], then it can be determined that the three-dimensional scene to be rendered includes two top-level nodes, and the node description modules corresponding to the two top-level nodes are the first node description module and the third node description module in the node list of the scene description file, respectively.

[0164] The scene description file parsing method of some embodiments of this application may include the following steps 5 and 6: Step 5: Obtain the target node description module from the node list ("nodes":[]) of the scene description file. The target node description module is any node description module in the node list. Step 6: Obtain the description information of the target node corresponding to the target node description module.

[0165] In some embodiments, step 4 above (obtaining the target node description module from the node list of the scene description file) may include at least one of the following steps 61 to 64:

[0166] Step 61: Obtain the name of the target node based on the value of the node name syntax element ("name") in the target node description module. For example, if the node name syntax element in the target node description module has the value "name":"avatarThorax", then the name of the target node can be obtained as avatarThorax based on the value of the node name syntax element in the target node description module.

[0167] Step 62: Obtain the index values ​​of the node description modules corresponding to each child node attached to the target node based on the index values ​​declared in the child node index list ("children":[]) of the target node description module. For example: if the child node index list in the target node description module and its declared index value is "children":[1,2], then based on the index values ​​declared in the child node index list of the target node description module, it can be determined that the target node has two child nodes, and the two child nodes attached to the target node are the nodes corresponding to the second and third node description modules in the node list of the scene description file, respectively.

[0168] Step 63: Based on the index value declared in the mesh index syntax element ("mesh") of the target node description module, obtain the index value of the mesh description module corresponding to each 3D mesh mounted on the target node.

[0169] For example, if the mesh index syntax element and its value in the target node description module are "mesh":0, then the 3D mesh corresponding to the first mesh description module in the mesh list ("meshes":[]) of the scene description file can be obtained according to the index value declared in the mesh index syntax element of the target node description module.

[0170] Step 64: Obtain the spatial position offset of the target node relative to its parent node based on the value of the position offset syntax element ("translation") in the target node description module. For example: if the position offset syntax element in a node description module is "translation":[0.0,0.0,20.0], then the spatial position offset of the node relative to its parent node can be obtained as [0.0,0.0,20.0] based on the value of the position offset syntax element in the node description module.

[0171] It should be noted that the spatial position offset of a node relative to its parent node is directly obtained from the value of the position offset syntax element in the node description module. This spatial position offset is not necessarily the spatial position offset of the node relative to the node representing the target digital person. The spatial position offset of the node relative to the node representing the target digital person is the sum of the spatial offsets of each node on the path from the node to the node representing the target digital person.

[0172] For example: Node A is a child node of the node representing the target digital human, and node B is a child node of node A. The position offset syntax element in the node description module corresponding to node A is "translation":[0.0,10.0,25.0], and the position offset syntax element in the node description module corresponding to node B is "translation":[0.0,10.0,20.0]. Then, we can first obtain the offset of node A relative to the node representing the target digital human as [0.0,10.0,25.0], and the offset of node B relative to node A as [0.0,00.0,20.0]. Then, we can sum the position offsets to obtain the offset of node B relative to the node representing the target digital human as [0.0,10.0,45.0].

[0173] The method for parsing scene description files in some embodiments of this application may further include: Step 7, obtaining a target mesh description module from the mesh list ("meshes":[]) of the scene description file. The target mesh description module is any mesh description module in the mesh list ("meshes":[]). Step 8, obtaining the description information of the target 3D mesh corresponding to the target mesh description module.

[0174] In some embodiments, step 8 (obtaining the description information of the target 3D mesh corresponding to the target mesh description module according to the target mesh description module) may include at least one of the following steps 81 to 83:

[0175] Step 81: Obtain the name of the target 3D mesh based on the value of the mesh name syntax element ("name") in the target mesh description module. For example, if the mesh name syntax element in a mesh description module in the scene description file has the value "name":"avatarNeck", then the name of the 3D mesh corresponding to that mesh description module can be obtained as: avatarNeck.

[0176] Step 82: Based on the value of the position syntax element ("position") in the attribute ("attributes":{}) of the primitive ("primitives"[]) of the target mesh description module, obtain the index value of the accessor description module corresponding to the accessor used to access the dynamic 3D model of the digital human component corresponding to the target 3D mesh. For example: if the position syntax element in the attribute of a primitive of a mesh description module in the scene description file is "position":2, then based on the value of the position syntax element in the attribute of the primitive of the mesh description module, it can be determined that the accessor used to access the dynamic 3D model of the digital human component corresponding to the 3D mesh of that mesh description module is the accessor corresponding to the third accessor description module in the scene description file.

[0177] Step 83: Obtain the type of the topology of the target 3D mesh based on the value of the mode syntax element ("mode") in the primitives ("primitives"[]) of the target mesh description module. For example, if the mode syntax element in the primitives of a mesh description module in the scene description file has a value of "mode":0, then the type of the topology of the 3D mesh corresponding to the mesh description module can be obtained as scattered points based on the value of the mode syntax element in the primitives of the mesh description module.

[0178] In some embodiments, the method for parsing scene description files provided in some embodiments of this application may further include the following steps 9 and 10:

[0179] Step 9: Obtain the target accessor description module from the accessor list ("accessors":[]) of the scene description file. The target accessor description module is any accessor description module in the accessor list ("accessors":[]) of the scene description file. Step 10: Obtain the description information of the target accessor corresponding to the target accessor description module.

[0180] In some embodiments, step 10 (obtaining the description information of the target accessor corresponding to the target accessor description module according to the target accessor description module) may include at least one of the following steps 101 to 108:

[0181] Step 101: Obtain the data type of the data accessed by the target accessor based on the value of the data type syntax element ("componentType") in the target accessor description module. For example, if the data type syntax element and its value in a certain accessor description module are: "componentType": 5126, then it can be determined that the data type accessed by the accessor corresponding to this accessor description module is a 32-bit floating-point number (float).

[0182] Step 102: Determine the type of the target accessor based on the value of the accessor type syntax element ("type") in the target accessor description module. For example, if the value of the accessor type syntax element in a certain accessor description module is "type":"VEC2", then the type of the accessor corresponding to that accessor description module can be determined to be a two-dimensional vector.

[0183] Step 103: Determine the number of data items accessed by the target accessor based on the value of the data count syntax element ("count") in the target accessor description module. For example, if the data count syntax element and its value in a certain accessor description module are "count": 1828, then it can be determined that the number of data items accessed by the accessor corresponding to that accessor description module is 1828.

[0184] Step 104: Determine the index value of the cache slice description module corresponding to the cache slice that caches the data accessed by the target accessor based on the value of the first cache slice index syntax element ("bufferViews") in the target accessor description module.

[0185] For example, if the first cache slice index syntax element of a certain accessor description module and its value is "bufferView":1, then it can be determined that the data accessed by the accessor corresponding to the accessor description module is cached in the cache slice corresponding to the second cache slice description module in the cache slice list ("bufferViews":[]).

[0186] Step 105: Determine whether the target accessor is a time-varying accessor based on MPEG extensions based on whether the target accessor description module contains an MPEG time-varying accessor ("MPEG_accessor_timed":{}). In some embodiments, determining whether the target accessor is a time-varying accessor based on MPEG extensions based on whether the target accessor description module contains an MPEG time-varying accessor ("MPEG_accessor_timed":{}) may include: if the target accessor description module contains an MPEG time-varying accessor, then determine that the target accessor is a time-varying accessor based on MPEG extensions; and if the target accessor description module does not contain an MPEG time-varying accessor, then determine that the target accessor is not a time-varying accessor based on MPEG extensions.

[0187] Step 106: Determine the cache slice description module corresponding to the cache slice that caches the time-varying parameters of the target accessor based on the value of the second cache slice index syntax element ("bufferView") in the MPEG time-varying accessor ("MPEG_accessor_timed":{}) of the target accessor description module. For example, if the second cache slice index syntax element in the MPEG time-varying accessor of a certain accessor description module is "bufferView":3, then it can be determined that the time-varying parameters of the accessor corresponding to this accessor description module are cached in the cache slice corresponding to the fourth cache slice description module in the cache slice list ("bufferViews":[]).

[0188] Step 107: Determine whether the value of the syntax element in the target accessor changes over time based on the value of the time-varying syntax element ("immutable") in the MPEG time-varying accessor ("MPEG_accessor_timed":{}) of the target accessor description module.

[0189] In some embodiments, determining whether the value of a syntax element in the target accessor changes over time based on the value of the time-varying syntax element ("immutable") in the MPEG time-varying accessor of the target accessor description module may include: if the time-varying syntax element in the MPEG time-varying accessor of the target accessor description module and its value are "immutable":true or "immutable":1, then it is determined that the value of the syntax element in the target accessor does not change over time; and if the time-varying syntax element in the MPEG time-varying accessor of the target accessor description module and its value are "immutable":false or "immutable":0, then it is determined that the value of the syntax element in the target accessor changes over time.

[0190] Step 108: Determine the name of the target accessor based on the value of the accessor name syntax element ("name") in the target accessor description module.

[0191] In some embodiments, the method for parsing scene description files provided in some embodiments of this application may further include the following steps 11 and 12: Step 11: Obtain a target buffer description module from the buffer list ("buffers":[]) of the scene description file. The target buffer description module is any buffer description module in the buffer list ("buffers":[]). Step 12: Obtain the description information of the target buffer corresponding to the target buffer description module.

[0192] In some embodiments, step 12 (obtaining the description information of the target cache corresponding to the target cache description module according to the target cache description module) may include at least one of the following steps 121 to 127:

[0193] Step 121: Obtain the name of the target cache based on the value of the cache name syntax element ("name") in the target cache description module.

[0194] Step 122: Obtain the capacity of the target buffer corresponding to the target buffer description module based on the value of the first byte length syntax element ("byteLength") in the target buffer description module. The target buffer description module is any buffer description module obtained from the buffer list in the scene description file. For example, if the first byte length syntax element of a buffer description module has a value of "byteLength":43972, then the capacity of the buffer corresponding to that buffer description module can be determined to be 43972 bytes.

[0195] Step 123: Determine whether the target buffer is a circular buffer based on MPEG extensions based on whether the target buffer description module contains an MPEG circular buffer ("MPEG_buffer_circular":{}).

[0196] In some embodiments, determining whether the target buffer is an MPEG-extended circular buffer based on whether the target buffer description module contains an MPEG circular buffer may include: if the target buffer description module contains an MPEG circular buffer, determining that the target buffer is an MPEG-extended circular buffer; and if the target buffer description module does not contain an MPEG circular buffer, determining that the target buffer is not an MPEG-extended circular buffer.

[0197] Step 124: Obtain the number of storage stages in the target buffer based on the value of the stage number syntax element ("count") in the MPEG circular buffer ("MPEG_buffer_circular":{}) of the target buffer description module.

[0198] For example, if the syntax element for the number of stages in the MPEG circular buffer of a certain buffer description module is "count":3, then it can be determined that the buffer corresponding to this buffer description module includes 3 storage stages.

[0199] Step 125: Based on the value of the media index syntax element ("media") in the MPEG circular buffer ("MPEG_buffer_circular":{}) of the target buffer description module, obtain the index value of the media description module corresponding to the media file to which the source data of the data cached by the target buffer belongs.

[0200] Step 126: Obtain the track index value of the source data of the data cached by the target buffer based on the value of the second track index syntax element ("tracks") in the MPEG circular buffer ("MPEG_buffer_circular":{}) of the target buffer description module.

[0201] Step 127: Obtain the data for reconstructing the dynamic 3D model of the corresponding digital human component based on the value of the Uniform Resource Identifier syntax element ("uri") in the target cache description module.

[0202] In some embodiments, the method for parsing a scene description file provided in the above embodiments may further include the following steps 13 and 14: Step 13: Obtain a target cache slice description module from the cache slice list ("bufferViews":[]) of the scene description file. The target cache slice description module is any cache slice description module in the cache slice list. Step 14: Obtain the description information of the target cache slice corresponding to the target cache slice description module.

[0203] In some embodiments, step 14 above (obtaining the description information of the target cache slice corresponding to the target cache slice description module according to the target cache slice description module) may include at least one of the following steps 141 to 143:

[0204] Step 141: Obtain the capacity of the target cache slice corresponding to the target cache slice description module based on the value of the second byte length syntax element ("byteLength") in the target cache slice description module. For example, if the second byte length syntax element and its value in a certain cache slice description module are "byteLength":21936, then the capacity of the cache slice corresponding to that cache slice description module can be determined to be 21936 bytes.

[0205] Step 142: Obtain the offset of the data cached by the target cache slice based on the value of the offset syntax element ("byteOffset") in the target cache slice description module. For example, if the offset syntax element and its value in a cache slice description module are "byteOffset":0, then it can be determined that the offset of the data cached by the cache slice corresponding to that cache slice description module is 0.

[0206] Step 143: Obtain the name of the target cache slice based on the value of the cache slice name syntax element ("name") in the target cache slice description module.

[0207] In some embodiments, the method for parsing the scene description file may further include: determining the version number of the scene description file based on the version syntax element ("version") and its value in the digital asset description module ("asset":{}) of the scene description file.

[0208] For example, the digital assets in the scene description file are shown below:

[0209] Therefore, it can be determined that the scene description file is written based on glTF version 2.0, and the version of the scene description file is the reference version of the scene description standard.

[0210] In some embodiments, the method for parsing the scene description file may further include: obtaining the extensions used by the scene description file according to the extensions used list ("extensionsUsed":[]) in the scene description file.

[0211] For example, the extension usage list ("extensionsUsed":[]) of the scene description file is shown below:

[0212] Then, the extended items used by the scene description file can be obtained, including: MPEG media, MPEG buffer circular, MPEG accessor timed, and MPEG node avatar.

[0213] In some embodiments, this application provides a scene description file parsing apparatus that executes the scene description file parsing method provided in the above embodiments. The principle and technical effect are similar, and will not be described in detail here to avoid redundancy.

Claims

1. A method for parsing a scene description file, comprising: The first node description module is obtained from the node list of the scene description file of the 3D scene to be rendered. The first node description module is the node description module corresponding to the node representing the target digital human. Obtain the facial marker point array from the digital human node array of the first node description module; Descriptive information of the facial markers of the target digital human is obtained based on the facial marker array.

2. The method according to claim 1, wherein the facial marker point array comprises: The level type syntax element and the value of the level type syntax element; The step of obtaining the description information of the facial marker points of the target digital human based on the facial marker point array includes: Based on the value of the level type syntax element, obtain the number and type of facial markers of the target digital human.

3. The method according to claim 1, wherein the facial marker point array comprises: Index syntax elements and the values ​​of those index syntax elements; The step of obtaining the description information of the facial marker points of the target digital human based on the facial marker point array includes: Based on the value of the index syntax element, the index value of the accessor description module corresponding to the first accessor in the accessor list of the scene description file is obtained. The first accessor is an accessor used to access the index information of the facial marker points of the target digital human.

4. The method according to claim 1, wherein the facial marker point array comprises: The location coordinate syntax element and the value of the location coordinate syntax element, wherein obtaining the description information of the facial marker points of the target digital human based on the facial marker point array includes: Based on the value of the location coordinate syntax element, the index value of the accessor description module corresponding to the second accessor in the accessor list of the scene description file is obtained. The second accessor is an accessor used to access the location coordinates of the facial marker points of the target digital human.

5. The method according to claim 1, further comprising: Obtain the digital human type syntax element and the value of the digital human type syntax element from the digital human node array of the first node description module; Based on the value of the digital human type syntax element, obtain the representation scheme of the target digital human.

6. The method according to claim 1, further comprising: Obtain the mapping list corresponding to the target number from the digital human node array of the first node description module; Based on the mapping list corresponding to the target digital human, obtain the digital human body parts represented by each digital human component of the target digital human and the node description modules in the node list corresponding to each digital human component of the target digital human.

7. The method according to claim 6, wherein the step of obtaining the digital human body parts represented by each digital human component of the target digital human and the node description module in the node list corresponding to each digital human component of the target digital human based on the mapping list corresponding to the target digital human includes: Obtain the mapping array corresponding to each digital human component of the target digital human from the mapping list corresponding to the target digital human; Based on the value of the part name syntax element in the mapping array corresponding to each digital human component of the target digital human, obtain the digital human body parts represented by each digital human component of the target digital human; Based on the value of the node index syntax element in the mapping array corresponding to each digital human component of the target digital human, obtain the node description module in the node list corresponding to each digital human component of the target digital human.

8. The method according to claim 1, further comprising: Obtain the active identifier syntax element and the value of the active identifier syntax element from the digital human node array of the first node description module; Based on the active identifier syntax element and its value, determine whether the target digital human is an active digital human.

9. The method according to any one of claims 1-8, further comprising: Obtain the target media description module from the media list of the Moving Picture Experts Group MPEG media in the scene description file; Obtain the description information of the target media file corresponding to the target media description module according to the target media description module; Wherein, the target media description module is any media description module in the media list, and the step of obtaining the description information of the target media file corresponding to the target media description module includes at least one of the following: The name of the target media file is obtained based on the value of the media name syntax element in the target media description module; or, whether the target media file needs to be automatically played is determined based on the value of the autoplay syntax element in the target media description module. Alternatively, determine whether the target media file needs to be played in a loop based on the value of the loop playback syntax element in the target media description module; or obtain the description information of each optional version of the target media file based on each optional description module in the optional list of the target media description module.

Images