Codec method, encoding device, decoding device, and storage medium

Abstract

An embodiment of the present application provides a codec method, an encoding device, a decoding device, and a storage medium, in which the decoding device decodes a code stream to determine a first instruction parameter, and if the first instruction parameter indicates that a patch data unit that does not contain geometry information exists, determines a second instruction parameter corresponding to the patch data unit, and if the second instruction parameter indicates that the patch data unit does not contain geometry information, performs a reconstruction process based on texture information corresponding to the patch data unit.

Description

[Technical Field]

[0001] The present application relates to the technical field of image processing, and in particular to a codec method, an encoding device, a decoding device, and a storage medium. [Background technology]

[0002] In the video codec field, in the process of rendering a non-diffuse reflective surface using an image-based rendering method, the colors of the surface in different observation directions need to be collected by a camera first, and then, during rendering, according to the position and observation direction of the rendering window, the colors of the surface collected by the cameras in different observation directions are weighted and fused to form the color of the surface at the pose of the rendering window.

[0003] Currently, it is permitted to represent the same area in 3D space using multiple texture patch data units (PDUs) and place them in a texture atlas, but at the same time, the geometry patches corresponding to these texture patches must also be placed in a geometry atlas for encoding and transmission.

[0004] As such, it can be seen that the currently supported methods for representing non-diffuse reflective surfaces in 3D space cannot take into account both the representation of non-diffuse reflective properties and compression performance. Therefore, how to improve codec efficiency and compression performance while being able to fully represent the non-diffuse reflective properties of surfaces is an important issue that needs to be resolved. Summary of the Invention

[0005] The embodiments of the present application provide a codec method, an encoding device, a decoding device, and a storage medium that can improve compression performance by reducing complexity while improving decoding accuracy and realizing a simple and efficient codec method.

[0006] The technical solutions of the embodiments of the present application can be realized as follows:

[0007] In a first aspect, an embodiment of the present application provides a decoding method applied to a decoding device, the method comprising: decoding the codestream to determine a first indication parameter; if the first indication parameter indicates that a patch data unit not containing geometry information exists, determining a second indication parameter corresponding to the patch data unit; If the second instruction parameter indicates that the patch data unit does not include geometry information, performing a reconstruction process based on texture information corresponding to the patch data unit.

[0008] In a second aspect, an embodiment of the present application provides an encoding method applied to an encoding device, the method comprising: If a patch data unit does not contain geometry information, encoding the patch data unit to generate a codestream, and simultaneously determining a first indication parameter and a second indication parameter of the patch data unit, wherein the first indication parameter is used to determine whether there is a patch data unit that does not contain geometry information, and the second indication parameter is used to determine whether the patch data unit contains geometry information; and writing the first instruction parameter and the second instruction parameter into the codestream.

[0009] In a third aspect, an embodiment of the present application provides a decoding device, the decoding device comprising: a decoding unit; a first determination unit; and a reconstruction unit; the decoder is configured to decode the codestream; the first determination unit is configured to determine a first indication parameter, and when the first indication parameter indicates that a patch data unit not including geometry information exists, determine a second indication parameter corresponding to the patch data unit; The reconstruction unit is configured to perform reconstruction processing based on texture information corresponding to the patch data unit when the second instruction parameter indicates that the patch data unit does not include geometry information.

[0010] In a fourth aspect, an embodiment of the present application provides a decoding device, the decoding device comprising a first processor and a first memory storing instructions executable by the first processor, the instructions causing the first processor to perform the above-mentioned decoding method.

[0011] In a fifth aspect, an embodiment of the present application provides a coding device, the coding device comprising: a coding unit; The encoding unit is configured to perform the steps of: if a patch data unit does not include geometry information, encoding the patch data unit to generate a code stream, and at the same time determining a first instruction parameter and a second instruction parameter of the patch data unit, wherein the first instruction parameter is used to determine whether a patch data unit does not include geometry information, and the second instruction parameter is used to determine whether the patch data unit includes geometry information; and writing the first instruction parameter and the second instruction parameter to the code stream.

[0012] In a sixth aspect, an embodiment of the present application provides an encoding device, the encoding device comprising a second processor and a second memory storing instructions executable by the second processor, the instructions causing the second processor to perform the encoding method described above.

[0013] In a seventh aspect, an embodiment of the present application provides a computer storage medium, the computer storage medium storing a computer program, the computer program causing a first processor to perform the decoding method described in the first aspect, or causing a second processor to perform the encoding method described in the second aspect.

[0014] The present invention provides a codec method, an encoding device, a decoding device, and a storage medium, in which the decoding device decodes a codestream to determine a first indication parameter, and if the first indication parameter indicates that a patch data unit containing no geometry information exists, determines a second indication parameter corresponding to the patch data unit, and if the second indication parameter indicates that the patch data unit contains no geometry information, performs reconstruction processing based on texture information corresponding to the patch data unit. If the patch data unit does not contain geometry information, the decoding device encodes the patch data unit to generate a codestream, and simultaneously determines the first indication parameter and the second indication parameter of the patch data unit, where the first indication parameter is used to determine whether a patch data unit containing no geometry information exists and the second indication parameter is used to determine whether the patch data unit contains geometry information, and writes the first indication parameter and the second indication parameter to the codestream. That is, in the embodiment of the present application, the first indication parameter can determine whether there is a patch data unit that does not contain geometry information, and the second indication parameter can determine whether the patch data unit does not contain geometry information, thereby supporting the transmission of different amounts of texture information and geometry information, effectively improving the rendering quality through sufficient texture information, while reducing the redundancy of geometry data, thereby fully expressing the non-diffuse reflection characteristics of the surface, and improving codec efficiency and compression performance. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a flowchart illustrating an implementation of a decoding method. [Figure 2] 2 is a flowchart illustrating an implementation of the decoding method. [Figure 3] 3 is a flowchart illustrating the implementation of the decoding method. [Figure 4] 1 is a flowchart illustrating an implementation of an encoding method. [Figure 5] 1 is a schematic diagram showing the configuration of a decoding device proposed in an embodiment of the present application. [Figure 6] 2 is a schematic diagram showing the configuration of a decoding device proposed in an embodiment of the present application. [Figure 7] 1 is a schematic diagram showing the configuration of an encoding device proposed in an embodiment of the present application. [Figure 8] 2 is a schematic diagram showing the configuration of an encoding device proposed in an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0016] The following clearly and completely describes the technical solutions of the embodiments of the present application with reference to the drawings in the embodiments of the present application. It should be noted that the specific embodiments described in this specification are for interpreting the relevant application and do not limit the present application. Furthermore, for the convenience of explanation, only parts related to the present application are shown in the drawings.

[0017] In real life, due to the influence of the material of the object, the same point on the same surface of the same object often appears in different colors when observed from different angles. This phenomenon in which the color of the object surface changes with the change in observation angle is called the non-diffuse reflection phenomenon of the object surface, and such a surface is also called a non-diffuse reflecting surface.

[0018] To reduce the transmission pixel rate and improve compression efficiency, MIV (MPEG Immersive Video) performs inter-view redundancy removal on the multi-view depth video input to the system for each frame. Finally, in a single time slice, a region in 3D space is represented using only one geometry patch (also called a subblock image) that describes the region's geometry information and one texture patch corresponding to the geometry patch. These patches are arranged on a canvas called an atlas (also called a combined image). Different types of patches are arranged on different atlases. An atlas with geometry patches is called a geometry atlas, i.e., a geometry combined image, and an atlas with texture patches is called a texture atlas, i.e., a texture combined image. These atlases are temporally stacked to form an atlas sequence, which is then encoded using a video codec to form a codestream for transmission.

[0019] In the existing MIV implementation, the Test Model of MPEG Immersive Video (TMIV), a process for removing redundancies between viewpoints, a region in 3D space is represented using only one geometry patch and one texture patch. This representation is feasible and efficient when used to represent non-diffuse reflective surfaces, because the colors of diffuse reflective surfaces in each observation direction are very similar and nearly identical. However, when used to represent non-diffuse reflective surfaces, this representation cannot realize non-diffuse reflective characteristics with different colors in different observation directions because only one texture patch is transmitted per region.

[0020] In the process of rendering a non-diffuse surface using an image-based rendering method, the colors of the surface at different viewing directions must first be collected by a camera. Then, during rendering, the colors of the surface collected by the cameras at different viewing directions are weighted and fused according to the position and viewing direction of the rendering window to form the color of the surface at the pose of the rendering window. That is, to realize the representation of a non-diffuse surface, multiple observed texture images of the surface at different viewing directions must be used. Therefore, an efficient representation of a non-diffuse surface is to represent it using one geometry patch and multiple texture patches observed from different directions.

[0021] The existing MIV standard allows the same area in 3D space to be represented using multiple texture patches and placed in a texture atlas, but at the same time, the current MIV standard requires that the geometry patches corresponding to these texture patches also be placed in a geometry atlas for encoding and transmission.

[0022] In other words, the methods of representing non-diffuse reflective surfaces in three-dimensional space supported by the current MIV standard mainly include a method of representing them using one geometry patch and one texture patch, or a method of representing them using multiple geometry patches and the same number of multiple texture patches.

[0023] However, a representation method using only one geometry patch and one texture patch cannot adequately represent the non-diffuse reflection characteristics of the surface, which affects the subsequent rendering quality.A representation method using multiple geometry patches and multiple texture patches can adequately represent the non-diffuse reflection characteristics of the surface, but multiple geometry patches repeatedly represent the same surface, which results in low representation efficiency of geometry information and reduced coding efficiency.

[0024] As described above, it can be seen that in the method for representing a non-diffuse reflection surface in a three-dimensional space supported by the current MIV standard, both the representation of non-diffuse reflection characteristics and compression performance cannot be considered. Therefore, how to improve the codec efficiency and compression performance while fully representing the non-diffuse reflection characteristics of the surface has become an important issue to be solved.

[0025] To solve the above problems, in the embodiments of the present application, the first indication parameter can be used to determine whether there is a patch data unit (PDU: Patch Data Unit) that does not include geometry information, and the second indication parameter can be used to determine whether the patch data unit does not include geometry information. Thereby, the transmission of different numbers of texture information and geometry information can be supported, and while effectively improving the rendering quality with sufficient texture information, the redundancy of geometry data can be reduced, and thereby, on the premise of fully representing the non-diffuse reflection characteristics of the surface, the codec efficiency and compression performance can be improved.

[0026] Furthermore, the codec method proposed in the present application can represent one region in a three-dimensional space using only at least one geometry patch data without repeatedly representing the same region in the three-dimensional space using a plurality of geometry patches. Therefore, in order to achieve efficient encoding and rendering of non-diffuse reflection objects, for one non-diffuse reflection surface, it is represented and transmitted using G geometry patches and T texture patches (both G and T are integers, and 1 ≤ G < T). In this way, while providing a plurality of texture information from different viewpoints for the same region, a sufficiently small amount of geometry information sufficient to represent the region can be transmitted. Compared with the existing solutions supported by the MIV standard, this solution can support a sufficient and efficient representation of the non-diffuse reflection surface, effectively improve the rendering quality of the surface, reduce the redundancy of geometry data information, and thereby improve the encoding efficiency of the system when compressing the non-diffuse reflection surface.

[0027] The codec method in the embodiments of the present application may be applied to a video encoding system, a video decoding system, or both a video encoding system and a video decoding system, and is not particularly limited in the embodiments of the present application.

[0028] In the following, the technical solutions of the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application.

[0029] One embodiment of the present application proposes a decoding method applied to a decoding device, and FIG. 1 is an implementation flowchart 1 of the decoding method. As shown in FIG. 1, the decoding method by the decoding device may include the following steps:

[0030] In step 101, the codestream is decoded to determine a first instruction parameter.

[0031] In an embodiment of the present application, the decoding device is capable of determining the first indication parameter by decoding the codestream.

[0032] Furthermore, in the embodiment of the present application, the first instruction parameter may be a sequence-level instruction parameter or an image-level instruction parameter.

[0033] In addition, in the implementation of the present application, the first instruction parameter can indicate whether there is at least one patch data unit that does not contain geometry information among all patch data units corresponding to the current atlas sequence.

[0034] It should be noted that in the implementation of the present application, the first indication parameter can indicate whether or not there is at least one patch data unit that does not include geometry information among all patch data units corresponding to the current atlas.

[0035] It can be understood that in the present embodiment, the first indication parameter can be understood as a flag bit indicating whether there is a patch data unit that does not include geometry information and only includes texture information. Specifically, the decoding device can decode the codestream to determine a variable as the first indication parameter, and then determine the first indication parameter according to the value of the variable. Here, the value of the first indication parameter can be a first value or a second value.

[0036] In addition, in an embodiment of the present application, if the value of the first instruction parameter is a first value, it can be determined that there is a patch data unit that does not contain geometry information, and if the value of the first instruction parameter is a second value, it can be determined that there is no patch data unit that does not contain geometry information, i.e., all patch data units contain geometry information.

[0037] It can be understood that in the embodiment of the present application, the first value may be set to 1 and the second value may be set to 0, or the first value may be set to true and the second value may be set to false, but this is not particularly limited in the present application.

[0038] Furthermore, in an embodiment of the present application, a flag asme_patch_geometry_absent_flag can be used to represent the first indication parameter. For example, in the present application, if the value of asme_patch_geometry_absent_flag is 1, it can indicate that there is a patch data unit that does not contain a geometry patch, that is, there is at least one patch data unit that does not contain geometry information and only contains texture information, and if the value of asme_patch_geometry_absent_flag is 0, it can indicate that there is no patch data unit that does not contain a geometry patch, that is, all patch data units contain texture information and geometry information.

[0039] Furthermore, in an embodiment of the present application, the current atlas sequence may be an atlas sequence to be currently decoded. Specifically, the atlas sequence may include geometry information or texture information corresponding to one or more patch data units, i.e., may include geometry patches or texture patches corresponding to one or more patch data units.

[0040] As can be understood, in embodiments of the present application, for any patch data unit in the current atlas sequence, the patch data unit may include both geometry information and texture information, or may include only texture information without including geometry information.

[0041] As can be understood, in embodiments of the present application, for any patch data unit in the current atlas, the patch data unit may include both geometry information and texture information, or may include only texture information without including geometry information.

[0042] Furthermore, in the embodiment of the present application, different methods can be selected to reconstruct a patch data unit depending on whether the patch data unit includes geometry information or does not include geometry information. Therefore, a first indication parameter can be first determined to indicate whether a patch data unit does not include geometry information.

[0043] In step 102, if the first indication parameter indicates that a patch data unit that does not contain geometry information exists, a second indication parameter corresponding to the patch data unit is determined.

[0044] In an embodiment of the present application, after determining the first indication parameter, if the first indication parameter indicates that there are patch data units that do not contain geometry information, the decoding device can further determine a second indication parameter corresponding to each patch data unit, and can determine the patch data units that do not contain geometry information based on the second indication parameter.

[0045] Furthermore, in an embodiment of the present application, the first instruction parameter may be a sequence-level instruction parameter or an image-level instruction parameter, so that if the first instruction parameter indicates that there is a patch data unit in the current atlas sequence that does not contain geometry information, the second instruction parameter corresponding to the patch data unit in the current atlas sequence is determined, or if the first instruction parameter indicates that there is a patch data unit in the current atlas that does not contain geometry information, the second instruction parameter corresponding to the patch data unit in the current atlas is determined.

[0046] It should be noted that in the implementation of the present application, the second indication parameter can indicate whether the corresponding patch data unit does not include geometry information.

[0047] It can be understood that in the present embodiment, the second instruction parameter can be understood as a flag bit indicating whether the codestream includes only texture information and no geometry information. Specifically, the decoding device can decode the codestream to determine a variable as the second instruction parameter, and then determine the second instruction parameter according to the value of the variable. Here, the value of the second instruction parameter can be a first value or a second value.

[0048] In addition, in an embodiment of the present application, if the value of the second instruction parameter is a first value, it can be determined that the corresponding patch data unit does not contain geometry information, and if the value of the second instruction parameter is a second value, it can be determined that the corresponding patch data unit contains geometry information.

[0049] It can be understood that in the embodiment of the present application, the first value may be set to 1 and the second value may be set to 0, or the first value may be set to true and the second value may be set to false, but this is not particularly limited in the present application.

[0050] Furthermore, in the embodiment of the present application, a flag pdu_geometry_absent_flag may be used to represent a second indication parameter. For example, in the present application, a value of pdu_geometry_absent_flag of 1 may indicate that the corresponding patch data unit does not include a geometry patch, i.e., the corresponding patch data unit does not include geometry information and only includes texture information, and a value of pdu_geometry_absent_flag of 0 may indicate that the corresponding patch data unit includes a geometry patch, i.e., the corresponding patch data unit includes texture information and a geometry patch.

[0051] In the present embodiment, the current atlas sequence may include at least one sub-atlas, where each sub-atlas includes at least one patch data unit. Correspondingly, a flag pdu_geometry_absent_flag[tileID][p] may be used to represent a second indication parameter, where tileID and p may be identifiers of the patch data unit. For example, in the present embodiment, a value of pdu_geometry_absent_flag[tileID][p] of 1 indicates that the p-th patch data unit in the sub-atlas of tileID does not include a geometry patch, i.e., the p-th patch data unit in the sub-atlas of tileID does not include geometry information and includes only texture information. A value of pdu_geometry_absent_flag[tileID][p] of 0 indicates that the p-th patch data unit in the sub-atlas of tileID includes a geometry patch, i.e., the p-th patch data unit in the sub-atlas of tileID includes texture information and geometry information.

[0052] In step 103, if the second instruction parameter indicates that the patch data unit does not contain geometry information, perform the reconstruction process based on the texture information corresponding to the patch data unit.

[0053] In an embodiment of the present application, when the first instruction parameter indicates that a patch data unit exists that does not contain geometry information, after determining the second instruction parameter corresponding to the patch data unit, if it is determined that the second instruction parameter indicates that the patch data unit does not contain geometry information, a reconstruction process can be further performed based on the texture information corresponding to the patch data unit.

[0054] In addition, in an embodiment of the present application, if it is determined based on the second instruction parameter that the corresponding patch data unit does not include geometry information and only includes texture information, the texture information corresponding to the patch data unit can be first determined, and then the patch data unit can be reconstructed using the texture information, and finally the reconstructed patch data unit corresponding to the patch data unit can be determined.

[0055] Furthermore, in an embodiment of the present application, when determining a reconstructed patch data unit corresponding to a patch data unit based on texture information corresponding to the patch data unit, the reconstruction process may be completed using only the texture information, or the geometry information of the patch data unit may be first reconstructed using the geometry information of a reference patch data unit, and then combined with the texture information of the patch data unit to complete the image reconstruction process.

[0056] It can be understood that, in the present application, according to one embodiment, if the second instruction parameter indicates that the patch data unit does not include geometry information, first determine texture information corresponding to the patch data unit, and then perform a reconstruction process on the patch data unit based on the texture information, where the reconstructed patch data unit only includes texture information and does not include geometry information.

[0057] In the embodiment of the present application, if the second instruction parameter indicates that the corresponding patch data unit only includes a texture patch (texture information) and does not include a geometry patch (geometry information), that is, if it can be determined based on the second instruction parameter that the corresponding patch data unit does not include geometry information, there is no need to perform a decoding process for the geometry information, and it is sufficient to only obtain the corresponding texture information by decoding. Correspondingly, in the subsequent image reconstruction process, there is no need to consider the geometry information of the patch data unit, and the corresponding reconstructed patch data unit is determined based only on the corresponding texture information.

[0058] In other words, in the decoding method proposed in this application, each time an Atlas sequence (current atlas sequence) is processed in the decoding process, the asme_patch_geometry_absent_flag (first indication parameter) corresponding to the Atlas sequence is first determined, and then, based on the asme_patch_geometry_absent_flag, it is possible to determine whether or not there is a patch data unit in the Atlas sequence that does not contain a geometry patch (geometry information).

[0059] If the value of asme_patch_geometry_absent_flag is 1, it can be determined that there are patch data units in the Atlas sequence that do not include a geometry patch. In this case, it is necessary to further determine pdu_geometry_absent_flag (second instruction parameter) corresponding to each patch data unit to determine which patch data units specifically do not include a geometry patch. If the value of asme_patch_geometry_absent_flag is 0, it can be determined that there are no patch data units in the Atlas sequence that do not include a geometry patch. In this case, it is possible to directly obtain the geometry patch (first geometry information) and texture patch (texture information) of each patch data unit by decoding without determining pdu_geometry_absent_flag. Thus, reconstruction processing is performed based on the geometry patch and texture patch.

[0060] Furthermore, in the embodiment of the present application, if it is determined that the Atlas sequence contains a patch data unit that does not contain a geometry patch, during the process of decoding all patch data units in the Atlas sequence one by one, it can further determine whether the corresponding patch data unit contains a geometry patch based on pdu_geometry_absent_flag. Specifically, if the value of pdu_geometry_absent_flag is 1, it can be determined that the corresponding patch data unit does not contain a geometry patch. In this case, during decoding, the geometry patch in the corresponding patch data unit is not obtained from the geometry Atlas, and only the texture patch in the corresponding patch data unit is obtained from the texture Atlas, and finally, the corresponding reconstructed patch data unit can be determined based on the texture patch. If the value of pdu_geometry_absent_flag is 0, it can be determined that the corresponding patch data unit contains a geometry patch. In this case, during decoding, the geometry patch in the corresponding patch data unit still needs to be obtained from the geometry Atlas, and at the same time, the texture patch in the corresponding patch data unit needs to be obtained from the texture Atlas, and finally, the corresponding reconstructed patch data unit can be determined based on the geometry patch and the texture patch.

[0061] It can be understood that in the present application, according to one embodiment, when the second indication parameter indicates that the patch data unit does not include geometry information, texture information corresponding to the patch data unit and identification information of the reference patch data unit can be first determined, and then first geometry information corresponding to the patch data unit can be determined based on the identification information, and finally, a reconstruction process can be performed on the patch data unit based on the texture information and the first geometry information, where the reconstructed patch includes both texture information and geometry information.

[0062] In the embodiment of the present application, if the second instruction parameter indicates that the corresponding patch data unit only includes a texture patch (texture information) and does not include a geometry patch (geometry information), that is, if it is determined based on the second instruction parameter that the corresponding patch data unit does not include geometry information, it is necessary to first perform a reconstruction process of the geometry information and simultaneously obtain the corresponding texture information by decoding. Correspondingly, in the subsequent image reconstruction process, the corresponding reconstructed patch data unit can be determined based on the texture information of the patch data unit and the reconstructed geometry information (first geometry information).

[0063] Furthermore, in an embodiment of the present application, identification information of a reference patch data unit corresponding to a patch data unit may be first determined, and then first geometry information corresponding to the patch data unit may be determined based on the identification information. Here, the identification information may be used to determine the reference patch data unit. For example, pdu_geometry_reference_patch_id[tileID][p] may be used to represent the identification information. Here, tileID and p may be indexes of the reference patch data unit, and the identification information pdu_geometry_absent_flag[tileID][p] may specify the pth patch data unit in the sub-atlas of tileID as the reference patch data unit. It can be understood that in the embodiments of the present application, after determining the corresponding reference patch data unit based on the identification information, second geometry information corresponding to the reference patch data unit can be further determined, and the second geometry information can be used to complete the reconstruction of the geometry information of the patch data unit and determine the first geometry information corresponding to the patch data unit.

[0064] That is, in an embodiment of the present application, if it is determined based on the second instruction parameter that the corresponding patch data unit does not contain geometry information, the second geometry information of the reference patch data unit can be used to reconstruct the first geometry information of the patch data unit, thereby determining the corresponding reconstructed patch data unit based on the texture information and the first geometry information of the patch data unit.

[0065] In other words, in the decoding method proposed in this application, each time an Atlas sequence (current atlas sequence) is processed in the decoding process, the asme_patch_geometry_absent_flag (first indication parameter) corresponding to the Atlas sequence is first determined, and then, based on the asme_patch_geometry_absent_flag, it is possible to determine whether or not there is a patch data unit in the Atlas sequence that does not contain a geometry patch (geometry information).

[0066] If the value of asme_patch_geometry_absent_flag is 1, it can be determined that there is a patch data unit in the Atlas sequence that does not contain a geometry patch. In this case, it is necessary to further determine pdu_geometry_absent_flag (second instruction parameter) corresponding to each patch data unit to determine which patch data units specifically do not contain a geometry patch. If the value of asme_patch_geometry_absent_flag is 0, it can be determined that there is no patch data unit in the Atlas sequence that does not contain a geometry patch. In this case, it is possible to directly obtain the geometry patch (first geometry information) and texture patch (texture information) of each patch data unit by decoding without determining pdu_geometry_absent_flag. Thus, reconstruction processing is performed based on the geometry patch and texture patch.

[0067] Furthermore, in an embodiment of the present application, if it is determined that the Atlas sequence contains a patch data unit that does not include a geometry patch, during the process of decoding all patch data units in the Atlas sequence one by one, it can further determine whether the corresponding patch data unit includes a geometry patch based on pdu_geometry_absent_flag. Specifically, if the value of pdu_geometry_absent_flag is 1, it can be determined that the corresponding patch data unit does not include a geometry patch. In this case, during decoding, the geometry patch in the corresponding patch data unit is not obtained from the geometry Atlas. It is necessary to further determine the index number pdu_geometry_reference_patch_id (identification information) of the geometry reconstruction reference patch data unit (reference patch data unit) of the corresponding patch data unit. Further, the geometry patch (second geometry information) of the reference patch data unit specified thereby is obtained based on pdu_geometry_reference_patch_id. Then, the geometry patch of the reference patch data unit can be used to reconstruct the geometry patch of the patch data unit. Finally, in combination with the texture patch (texture information) obtained from the texture Atlas, the reconstruction process can be completed and the corresponding reconstructed patch data unit can be determined. If the value of pdu_geometry_absent_flag is 0, it can be determined that the corresponding patch data unit contains a geometry patch. In this case, when decoding, it is still necessary to obtain the geometry patch in the corresponding patch data unit from the geometry atlas, and at the same time, it is necessary to obtain the texture patch in the corresponding patch data unit from the texture atlas. Finally, it is possible to determine the corresponding reconstructed patch data unit based on the geometry patch and the texture patch.

[0068] Furthermore, in an embodiment of the present application, FIG. 2 is an implementation flowchart 2 of the decoding method. As shown in FIG. 2, after decoding the codestream and determining the first indication parameter, i.e., after step 101, the decoding method by the decoding device may further include the following steps:

[0069] In step 104, if the first indication parameter indicates that all patch data units include geometry information, determine texture information and first geometry information corresponding to the patch data units.

[0070] In an embodiment of the present application, after determining the first indication parameter, the decoding device can further determine texture information and first geometry information corresponding to each patch data unit if the first indication parameter indicates that all patch data units include geometry information.

[0071] It can be understood that in the embodiment of the present application, if it is determined based on the first instruction parameter that each patch data unit includes a geometry patch (geometry information) and a texture patch (texture information), the texture information and first geometry information corresponding to the patch data unit can be directly obtained by decoding.

[0072] That is, in an embodiment of the present application, when the first instruction parameter indicates that all patch data units include geometry information, the texture information and first geometry information of the patch data units can be directly obtained without determining the second instruction parameter.

[0073] In step 105, a reconstruction process is performed on the patch data unit based on the texture information and the first geometry information.

[0074] In an embodiment of the present application, after determining the texture information and first geometry information corresponding to a patch data unit, a reconstruction process can be performed on the patch data unit based on the texture information and the first geometry information, and finally, a corresponding reconstructed patch data unit can be obtained.

[0075] It should be noted that in the embodiments of the present application, for patch data units that include both texture information and first geometry information, the corresponding texture information and first geometry information can be directly used to complete the determination of the reconstructed patch data unit.

[0076] Further, in an embodiment of the present application, Figure 3 is an implementation flowchart 3 of the decoding method. As shown in Figure 3, if the first indication parameter indicates that there is a patch data unit that does not contain geometry information, after determining the second indication parameter corresponding to the patch data unit, i.e., after step 102, the decoding method by the decoding device may further include the following steps:

[0077] In step 106, if the second indication parameter indicates that the patch data unit includes geometry information, determine texture information and first geometry information corresponding to the patch data unit.

[0078] In an embodiment of the present application, if the first instruction parameter indicates that there is a patch data unit that does not contain geometry information, after determining the second instruction parameter corresponding to the patch data unit, if the second instruction parameter indicates that the patch data unit contains geometry information, texture information and first geometry information corresponding to the patch data unit can be further determined.

[0079] It can be understood that in the embodiment of the present application, if it is determined based on the second instruction parameter that the corresponding patch data unit includes both a geometry patch (geometry information) and a texture patch (texture information), the texture information and the first geometry information corresponding to the patch data unit can be directly obtained by decoding.

[0080] That is, in the embodiment of the present application, when the first indication parameter indicates that there is a patch data unit that does not contain geometry information, the specific situation of whether each patch data unit contains geometry information can be further determined based on the determined second indication parameter. Here, when the second indication parameter indicates that the corresponding patch data unit contains geometry information, that is, for a patch data unit that contains both geometry information and texture information, the texture information and first geometry information of the patch data unit can be directly obtained.

[0081] In step 105, a reconstruction process is performed on the patch data unit based on the texture information and the first geometry information.

[0082] In an embodiment of the present application, after determining the texture information and first geometry information corresponding to a patch data unit, a reconstruction process can be performed on the patch data unit based on the texture information and the first geometry information, and finally, a corresponding reconstructed patch data unit can be obtained.

[0083] Furthermore, in an embodiment of the present application, if the second instruction parameter indicates that the patch data unit does not contain geometry information, after performing the reconstruction process based on the texture information corresponding to the patch data unit, i.e., after step 103, the method of rendering processing by the decoding device may include the following steps:

[0084] In step 107, a target window is constructed based on the patch data unit containing the geometry information.

[0085] In step 108, a shading process is performed on the target window based on the texture information of the patch data unit to obtain a rendered image.

[0086] In an embodiment of the present application, when the reconstruction process is completed using only texture information for a patch data unit that does not contain geometry information, a target window can be further constructed based on the patch data unit that contains geometry information, and then a shading process can be performed on the target window based on the texture information of the patch data unit to obtain a rendered image.

[0087] Furthermore, in the embodiment of the present application, after the decoding process is completed, a rendering process can be further performed using the image information obtained after decoding, where the rendering process can include two processes: a viewpoint view restoration process and a target window geometry reconstruction and pixel shading process.

[0088] In the viewpoint view restoration process, a general method is to sequentially restore the texture patches in all patch data units to their original positions in the texture view of the original viewpoint to which they belong, and sequentially restore the geometry patches in the patch data units to their original positions in the geometry view of the original viewpoint to which they belong. Meanwhile, in the process of scanning all patch data units to perform view restoration, in the present application, if the first instruction parameter indicates that all patch data units include both geometry information and texture information, or if the second instruction parameter indicates that the corresponding patch data unit includes both geometry information and texture information, viewpoint view restoration processing can be performed on the patch data units according to the general method described above. For patch data units that do not include geometry information and only include texture information, as indicated by the second instruction parameter (i.e., patch data units whose pdu_geometry_absent_flag has a value of 1), only the texture patches of the patch data units can be restored to their original positions in the texture view of the original viewpoint to which they belong, without performing a geometry patch restoration operation.

[0089] Furthermore, in the embodiment of the present application, when performing the geometry reconstruction and pixel shading process of the target window, first, the existing multi-view geometry image restored by the viewpoint view restoration process is used to reconstruct the geometry image of the target window, then each pixel point in the target window is scanned, and candidate pixels for each pixel are obtained from the multi-view texture image based on the position of the pixel point in the target window in three-dimensional space, and the pixel values ​​of each candidate pixel are weighted and fused based on the relative position and orientation relationship between each candidate pixel and the target window to obtain the pixel value of each pixel point in the target window.

[0090] In other words, in an embodiment of the present application, if there are cases where, for all patch data units, some patch data units contain both geometry information and texture information and the remaining patch data units contain only texture information, when rendering a target window, only some of the patch data units that contain geometry information can be used to reconstruct the target window geometry image, and the target window can be shaded using the texture information of all available patch data units.

[0091] Furthermore, in an embodiment of the present application, if the second instruction parameter indicates that the patch data unit does not contain geometry information, after performing the reconstruction process based on the texture information corresponding to the patch data unit, i.e., after step 103, the method of rendering processing by the decoding device may include the following steps:

[0092] In step 109, a geometry-missing patch data unit reconstruction process is performed on the patch data units that do not contain geometry information based on the reference patch data units.

[0093] In step 110, a target window is constructed based on the geometry information of the patch data unit.

[0094] In step 108, a shading process is performed on the target window based on the texture information of the patch data unit to obtain a rendered image.

[0095] In an embodiment of the present application, for a patch data unit that does not contain geometry information, the geometry information of the patch data unit is first reconstructed using the geometry information of the reference patch data unit, and then combined with the texture information of the patch data unit to complete the image reconstruction process.First, a geometry-missing patch data unit reconstruction process is performed on the patch data units that do not contain geometry information based on the reference patch data unit, so that all patch data units contain both geometry information and texture information.Furthermore, a target window is constructed based on the geometry information of the patch data unit, and then a shading process is performed on the target window based on the texture information of the patch data unit to obtain a rendered image.

[0096] Furthermore, in the embodiment of the present application, after the decoding process is completed, the image information obtained after decoding can be used to perform a rendering process, where the rendering process may include the processes of missing geometry patch reconstruction, viewpoint view restoration, target window geometry reconstruction and pixel shading.

[0097] In the missing geometry patch reconstruction process, all patch data units can be scanned. If the value of pdu_geometry_absent_flag of a patch data unit is 1, it is determined that the patch data unit is missing a geometry patch (geometry information), and the missing geometry patch reconstruction can be initiated. Specifically, the geometry patch of the patch data unit can be reconstructed using the geometry patch and viewpoint parameter information in the reference patch data unit whose index number is pdu_geometry_reference_patch_id. After the above process, all patch data units contain a texture patch and a geometry patch.

[0098] Furthermore, in an embodiment of the present application, after the missing geometry patch reconstruction is completed, for all patch data units, the texture patches in the patch data units may be sequentially restored to their original positions in the texture view of the original viewpoint to which they belong, and the geometry patches in the patch data units may be sequentially restored to their original positions in the geometry view of the original viewpoint to which they belong.

[0099] Furthermore, in the embodiment of the present application, when performing the geometry reconstruction and pixel shading process of the target window, first, the existing multi-view geometry image restored by the viewpoint view restoration process is used to reconstruct the geometry image of the target window, then each pixel point in the target window is scanned, and candidate pixels for each pixel are obtained from the multi-view texture image based on the position of the pixel point in the target window in three-dimensional space, and the pixel values ​​of each candidate pixel are weighted and fused based on the relative position and orientation relationship between each candidate pixel and the target window to obtain the pixel value of each pixel point in the target window.

[0100] In other words, in an embodiment of the present application, if there are cases where, for all patch data units, some patch data units include both geometry information and texture information and the remaining patch data units include only texture information, when rendering the target window, a geometry-missing patch data unit reconstruction process is first performed, so that all patch data units after the geometry-missing patch data unit reconstruction process include geometry patches and texture patches, and finally, the geometry information of all patch data units can be used to reconstruct the target window geometry image and the texture information of all patch data units can be used to shade the target window.

[0101] As described above, the decryption method proposed in the above steps 101 to 110 can support the decryption of patch data units that do not contain geometry information and only contain texture information. Thereby, for one non-diffuse reflection surface, it is possible to perform representation and transmission using G geometry patches and T texture patches (both G and T are integers, and 1 ≤ G < T), that is, for the same region, while providing multiple texture information from different viewpoints, it is possible to transmit a sufficiently small amount of geometry information to represent the region. Thereby, it is possible to support a sufficient and efficient representation of the non-diffuse reflection surface, effectively improve the rendering quality of the surface, reduce the redundancy of the geometry data information, and thereby improve the encoding efficiency when compressing the non-diffuse reflection surface.

[0102] Note that the decryption method proposed in the present application introduces a mechanism that does not require the patch data unit to contain geometry patches in order to enable the MIV standard to support efficient encoding of non-diffuse reflection content.

[0103] Correspondingly, the decryption method proposed in the present application sets syntax elements that support patch data units that do not contain geometry patches at the atlas sequence level and the patch data unit level based on the existing MIV standard syntax structure, so that patch data units that do not contain geometry patches can also be accurately decrypted. Here, the geometry information of the patch data unit that does not contain geometry patches is obtained by reconstructing a specific patch data unit that contains geometry patches.

[0104] Embodiments of the present application provide a decoding method. The decoding device decodes a code stream to determine a first instruction parameter. When the first instruction parameter indicates that there is a patch data unit that does not contain geometry information, it determines a second instruction parameter corresponding to the patch data unit. When the second instruction parameter indicates that the patch data unit does not contain geometry information, it performs a reconstruction process based on the texture information corresponding to the patch data unit. That is, in the embodiments of the present application, the first instruction parameter can be used to determine whether there is a patch data unit that does not contain geometry information, and the second instruction parameter can be used to determine whether the patch data unit does not contain geometry information, so that different numbers of texture information and geometry information can be supported for transmission, effectively improving the rendering quality with sufficient texture information while reducing the redundancy of geometry data, thereby being able to fully represent the non-diffuse reflection characteristics of the surface and improving the codec efficiency and compression performance.

[0105] Based on the above embodiments, the decoding method provided in another embodiment of the present application is used to achieve efficient encoding and rendering of non-diffuse reflection objects. Specifically, for one non-diffuse reflection surface, G geometry patches and T texture patches are used for representation and transmission (both G and T are integers, and 1 ≤ G < T). Thereby, while providing multiple texture information from different viewpoints for the same area, a sufficient amount of geometry information can be transmitted to represent the area.

[0106] Furthermore, in the embodiments of the present application, on the decoding side, the decoding device can enable accurate decoding of patch data units that do not contain geometry patches. Specifically, based on the current MIV standard, by modifying the related syntax structure, the modified MIV standard will support representing and transmitting G geometry patches and T texture patches for one non-diffuse reflection surface (both G and T are integers, and 1 ≤ G < T).

[0107] Note that in the present embodiment, the syntax can be modified to declare that there are no geometry patches in the current patch data unit.

[0108] As can be understood, in the embodiment of the present application, one syntax element asme_patch_geometry_absent_flag is added to the syntax structure "Atlas sequence parameter set MIV extension syntax" of the MIV standard to indicate that there is a patch data unit in the current atlas sequence that does not include a geometry patch and includes only texture patches. Here, Table 1 is the current syntax structure 1, and Table 2 is the modified syntax structure 1. Based on Table 1, Table 2 adds a syntax element asme_patch_geometry_absent_flag to indicate whether there is a patch data unit in the current atlas sequence that does not include a geometry patch and includes only texture patches.

[0109] [Table 1]

[0110] [Table 2]

[0111] It should be noted that in the present embodiment, after decoding the codestream based on Table 2 above, if it is determined that asme_patch_geometry_absent_flag is equal to 1, it can indicate that the pdu_geometry_absent_flag syntax element is present in the patch_data_unit() syntax structure, that is, it can indicate that there is a patch data unit in the current atlas sequence that does not contain geometry information and only contains texture information. If it is determined that asme_patch_geometry_absent_flag is equal to 0, it can indicate that the pdu_geometry_absent_flag syntax element is not present in the patch_data_unit() syntax structure, that is, it can indicate that all patch data units in the current atlas sequence contain texture information and geometry information.

[0112] Furthermore, in an embodiment of the present application, if it is determined that there are patch data units in the current atlas sequence that do not contain geometry information and only contain texture information, the decoding can further determine the pdu_geometry_absent_flag syntax element corresponding to each patch data unit.

[0113] As can be understood, in the embodiment of the present application, one syntax element, pdu_geometry_absent_flag[tileID][p], is added to the syntax structure "Patch data unit MIV extension syntax" of the MIV standard to indicate whether the current patch data unit includes a texture patch as well as a geometry patch, thereby allowing a decoding device to recognize a patch data unit that does not include a geometry patch, and therefore does not need to consider the geometry information of the patch data unit in subsequent operations. Here, Table 3 is the current syntax structure 2, and Table 4 is the modified syntax structure 2. Based on Table 3, Table 4 adds a syntax element, pdu_geometry_absent_flag[tileID][p], indicating whether the current patch data unit does not include a geometry patch and only includes a texture patch.

[0114] [Table 3]

[0115] [Table 4]

[0116] In the embodiment of the present application, based on the above Table 4, if it is determined that pdu_geometry_absent_flag[tileID][p] is equal to 1, it can indicate that valid geometry information does not exist in the geometry patch data (Patch), that is, it can indicate that the patch data unit does not contain geometry information. If it is determined that pdu_geometry_absent_flag[tileID][p] is equal to 0, it can indicate that valid geometry information exists in the geometry patch data (Patch), that is, it can indicate that the patch data unit contains geometry information.

[0117] Furthermore, in an embodiment of the present application, if it is determined that there are patch data units in the current atlas sequence that do not contain geometry information and only contain texture information, the pdu_geometry_absent_flag syntax element corresponding to each patch data unit can be further determined by decoding.

[0118] Thus, in this embodiment, each time an Atlas sequence is processed during the decoding process, the atlas sequence is checked for patch data units that do not contain geometry patches based on the asme_patch_geometry_absent_flag. If the asme_patch_geometry_absent_flag is 1, this indicates that the atlas sequence contains patch data units that do not contain geometry patches. If the asme_patch_geometry_absent_flag is 0, this indicates that the atlas sequence does not contain patch data units that do not contain geometry patches. Next, in the process of decoding all patch data units in the atlas sequence one by one, the pdu_geometry_absent_flag can be used to determine whether the current patch data unit contains a geometry patch. If the pdu_geometry_absent_flag is 1, this indicates that the current patch data unit does not contain a geometry patch. In this case, the geometry patch in the current patch data unit is not obtained from the geometry atlas during decoding. If the pdu_geometry_absent_flag is 0, this indicates that the current patch data unit contains a geometry patch.

[0119] In the present embodiment, in the current rendering process, for all patch data units, the texture patches in the patch data units are sequentially restored to their original positions in the texture view of the original viewpoint to which they belong, and the geometry patches in the patch data units are restored to their original positions in the geometry view of the original viewpoint to which they belong. Meanwhile, in the process of scanning all patch data units to perform view restoration, if the pdu_geometry_absent_flag of a patch data unit is 1, only the texture patches of the block image unit are restored to their original positions in the texture view of the viewpoint to which they belong, without performing the restoration operation of the geometry patches.

[0120] Furthermore, in the embodiment of the present application, when performing the geometry reconstruction and pixel shading processing of the target window, the existing multi-view geometry image restored in the previous step is used to reconstruct the geometry image of the target window, and then each pixel point in the target window is scanned, and candidate pixels for each pixel are obtained from the restored multi-view texture image based on the position of the pixel point in the target window in three-dimensional space, and the pixel values ​​of each candidate pixel are weighted and fused based on the relative position and orientation relationship between each candidate pixel and the target window to obtain the pixel value of each pixel point in the target window.

[0121] That is, in an embodiment of the present application, only some patch data units in the patch data unit list contain both geometry information and texture information, and the remaining patch data units contain only texture information. When rendering a target window, only some patch data units in the patch data unit list that contain geometry information are used to reconstruct a target window geometry image, and texture information from all available patch data units is used to shade the target window.

[0122] Embodiments of the present application provide a decoding method. The decoding apparatus decodes a code stream to determine a first instruction parameter. When the first instruction parameter indicates that there is a patch data unit that does not contain geometry information, it determines a second instruction parameter corresponding to the patch data unit. When the second instruction parameter indicates that the patch data unit does not contain geometry information, it performs a reconstruction process based on the texture information corresponding to the patch data unit. That is, in the embodiments of the present application, the first instruction parameter can be used to determine whether there is a patch data unit that does not contain geometry information, and the second instruction parameter can be used to determine whether the patch data unit does not contain geometry information. Thereby, it is possible to support the transmission of different numbers of texture information and geometry information, effectively improve the rendering quality with sufficient texture information, reduce the redundancy of geometry data, and thereby fully represent the non-diffuse reflection characteristics of the surface, and improve the codec efficiency and compression performance.

[0123] Based on the above embodiments, the decoding method provided in another embodiment of the present application is used to achieve efficient encoding and rendering of non-diffuse reflection objects. Specifically, for one non-diffuse reflection surface, G geometry patches and T texture patches are used for representation and transmission (both G and T are integers, and 1 ≤ G < T). Thereby, while providing multiple texture information from different viewpoints for the same region, it is possible to transmit a sufficient amount of geometry information to represent the region.

[0124] Furthermore, in the embodiments of the present application, on the decoding side, the decoding apparatus can enable accurate decoding of patch data units that do not contain geometry patches. Specifically, based on the current MIV standard, by modifying the related syntax structure, the modified MIV standard supports representing and transmitting G geometry patches and T texture patches for one non-diffuse reflection surface (both G and T are integers, and 1 ≤ G < T).

[0125] Note that in the present embodiment, the syntax can be modified to declare that there is no geometry patch in the current patch data unit, while at the same time indicating that the geometry information of the current patch data unit is obtained by reconstruction of a specific patch data unit that contains the geometry patch, and giving the index of the patch data unit that provides the geometry reconstruction information.

[0126] As can be understood, in the embodiment of the present application, one syntax element asme_patch_geometry_absent_flag is added to the syntax structure "Atlas sequence parameter set MIV extension syntax" of the MIV standard to indicate that there is a patch data unit in the current atlas sequence that does not include a geometry patch and only includes a texture patch. As shown in Table 2 above, the syntax element asme_patch_geometry_absent_flag is added to indicate whether there is a patch data unit in the current atlas sequence that does not include a geometry patch and only includes a texture patch.

[0127] It should be noted that in the embodiment of the present application, after decoding the codestream based on Table 2 above, if it is determined that asme_patch_geometry_absent_flag is equal to 1, it can indicate that the pdu_geometry_absent_flag syntax element is present in the patch_data_unit() syntax structure, that is, it can indicate that there is a patch data unit in the current atlas sequence that does not contain geometry information and only contains texture information. If it is determined that asme_patch_geometry_absent_flag is equal to 0, it can indicate that the pdu_geometry_absent_flag syntax element is not present in the patch_data_unit() syntax structure, that is, it can indicate that all patch data units contain texture information and geometry information.

[0128] Furthermore, in an embodiment of the present application, if it is determined that there are patch data units in the current atlas sequence that do not contain geometry information and only contain texture information, the pdu_geometry_absent_flag syntax element corresponding to each patch data unit can be further determined by decoding.

[0129] As can be understood, in the embodiment of the present application, one syntax element, pdu_geometry_absent_flag[tileID][p], is added to the syntax structure "Patch data unit MIV extension syntax" of the MIV standard to indicate whether the current patch data unit includes a texture patch as well as a geometry patch, so that the decoding device can recognize a patch data unit that does not include a geometry patch, and therefore does not need to consider the geometry information of the patch data unit in subsequent operations. Here, Table 5 is a modified syntax structure 3, which is based on Table 3 and adds a syntax element, pdu_geometry_absent_flag[tileID][p], to indicate whether the current patch data unit does not include a geometry patch and only includes a texture patch.

[0130] [Table 5]

[0131] In the present embodiment, based on Table 5 above, if pdu_geometry_absent_flag[tileID][p] is determined to be equal to 1, it can indicate that valid geometry information is not present in the geometry patch data (Patch), that is, it can indicate that the patch data unit does not contain geometry information. If pdu_geometry_absent_flag[tileID][p] is determined to be equal to 0, it can indicate that valid geometry information is present in the geometry patch data (Patch), that is, it can indicate that the patch data unit contains geometry information.

[0132] As can be understood, in the embodiment of the present application, after it is determined that the current patch data unit does not contain a geometry patch in the syntax structure "Patch data unit MIV extension syntax" of the MIV standard, i.e., it is determined that pdu_geometry_absent_flag[tileID][p] is equal to 1, one syntax element pdu_geometry_reference_patch_id[tileID][p] is further added to provide the index number of the geometry reconstruction reference patch data unit. The geometry patch in the patch data unit specified by the index number can be used to reconstruct the geometry information missing from the current patch data unit.

[0133] Thus, in this embodiment, each time an Atlas sequence is processed during the decoding process, the atlas sequence is checked for the presence of a patch data unit that does not contain a geometry patch based on the asme_patch_geometry_absent_flag. If the asme_patch_geometry_absent_flag is 1, this means that the atlas sequence contains a patch data unit that does not contain a geometry patch. If the asme_patch_geometry_absent_flag is 0, this means that the atlas sequence does not contain a patch data unit that does not contain a geometry patch. Next, in the process of decoding all patch data units in the atlas sequence one by one, the atlas sequence is checked for the presence of a geometry patch based on the pdu_geometry_absent_flag. If the pdu_geometry_absent_flag is 1, this means that the current patch data unit does not contain a geometry patch. In this case, instead of obtaining the geometry patch in the current patch data unit from the geometry atlas, the index number pdu_geometry_reference_patch_id of the geometry reconstruction reference patch data unit for the current patch data unit is determined during decoding. At this time, the geometry patch of the reference patch data unit specified by the index number is obtained. If pdu_geometry_absent_flag is 0, it means that the current patch data unit includes a geometry patch.

[0134] In this embodiment, all patch data units are scanned, and if the pdu_geometry_absent_flag of the current patch data unit is 1, it means that a geometry patch is missing from the current patch data unit, and the missing geometry patch reconstruction is initiated. That is, the geometry patch and viewpoint parameter information in the patch data unit with the index number pdu_geometry_reference_patch_id are used to reconstruct the geometry patch of the current patch data unit. After this process, all patch data units contain a texture patch and a geometry patch.

[0135] Furthermore, in an embodiment of the present application, after the geometry-missing patch data unit reconstruction process, for all patch data units, the texture patches in the patch data unit are sequentially restored to their original positions in the texture view of the original viewpoint to which they belong, and the geometry patches in the patch data unit are restored to their original positions in the geometry view of the original viewpoint to which they belong.

[0136] Furthermore, in the embodiment of the present application, when performing the geometry reconstruction and pixel shading processing of the target window, the existing multi-view geometry image restored in the previous step is used to reconstruct the geometry image of the target window, and then each pixel point in the target window is scanned, and candidate pixels for each pixel are obtained from the restored multi-view texture image based on the position of the pixel point in the target window in three-dimensional space, and the pixel values ​​of each candidate pixel are weighted and fused based on the relative position and orientation relationship between each candidate pixel and the target window to obtain the pixel value of each pixel point in the target window.

[0137] That is, in the embodiments of the present application, all patch data units in the patch data unit list after the geometry missing patch data unit reconstruction process include geometry patches and texture patches. When rendering the target window, the geometry information in all available patch data units is used to reconstruct the target window geometry image, and the texture information in all available patch data units is used to shade the target window.

[0138] The embodiments of the present application provide a decoding method. The decoding device decodes a code stream to determine a first indication parameter. When the first indication parameter indicates that there is a patch data unit that does not include geometry information, it determines a second indication parameter corresponding to the patch data unit. When the second indication parameter indicates that the patch data unit does not include geometry information, it performs a reconstruction process based on the texture information corresponding to the patch data unit. That is, in the embodiments of the present application, the first indication parameter can be used to determine whether there is a patch data unit that does not include geometry information, and the second indication parameter can be used to determine whether the patch data unit does not include geometry information, thereby supporting the transmission of different numbers of texture information and geometry information, effectively improving the rendering quality with sufficient texture information, reducing the redundancy of geometry data, and thereby being able to fully express the non-diffuse reflection characteristics of the surface and improving the codec efficiency and compression performance.

[0139] Based on the above embodiments, the encoding method proposed in the present application performs representation and transmission (G and T are both integers, and 1 ≤ G < T) using G geometry patches and T texture patches for one non-diffuse reflection surface during encoding. This means that there is a patch data unit that does not include geometry patches during encoding.

[0140] One embodiment of the present application proposes an encoding method applied to an encoding device, and FIG. 4 is an implementation flowchart of the encoding method. As shown in FIG. 4, the encoding method by the encoding device may include the following steps:

[0141] In step 201, if the patch data unit does not contain geometry information, encode the patch data unit to generate a codestream, and simultaneously determine a first indication parameter and a second indication parameter of the patch data unit, where the first indication parameter is used to determine whether there is a patch data unit that does not contain geometry information, and the second indication parameter is used to determine whether the patch data unit contains geometry information.

[0142] In step 202, the first instruction parameter and the second instruction parameter are written to the codestream.

[0143] In an embodiment of the present application, if there is a patch data unit that does not contain geometry information, the encoding device encodes the patch data unit to generate a code stream, and at the same time, the encoding device can determine a first instruction parameter and a second instruction parameter for each patch data unit, and then write the first instruction parameter and the second instruction parameter into the code stream.

[0144] Furthermore, in the embodiment of the present application, the first instruction parameter may be a sequence-level instruction parameter or an image-level instruction parameter.

[0145] In the implementation of the present application, the first indication parameter can indicate whether there is at least one patch data unit that does not include geometry information among all patch data units corresponding to the current atlas sequence, that is, the first indication parameter can be used to determine whether there is a patch data unit that does not include geometry information in the current atlas sequence.

[0146] In the implementation of the present application, the first indication parameter can indicate whether there is at least one patch data unit that does not include geometry information among all patch data units corresponding to the current atlas, that is, the first indication parameter can be used to determine whether there is a patch data unit that does not include geometry information in the current atlas.

[0147] It can be understood that in the present embodiment, the first indication parameter can be understood as a flag bit indicating whether there is a patch data unit that does not include geometry information and only includes texture information. Specifically, the encoding device can determine a variable as the first indication parameter, and thereby determine the first indication parameter according to the value of the variable. Here, the value of the first indication parameter can be a first value or a second value.

[0148] In addition, in an embodiment of the present application, if it is determined that there is a patch data unit in the current atlas sequence that does not contain geometry information, the value of the first instruction parameter can be determined to be a first value, and if it is determined that there is no patch data unit in the current atlas sequence that does not contain geometry information, i.e., if it is determined that all patch data units in the current atlas sequence contain geometry information, the value of the first instruction parameter can be determined to be a second value.

[0149] In addition, in an embodiment of the present application, if it is determined that there are no patch data units in the current atlas that do not contain geometry information, the value of the first instruction parameter can be determined to be a first value, and if it is determined that there are no patch data units in the current atlas that do not contain geometry information, i.e., if it is determined that all patch data units in the current atlas contain geometry information, the value of the first instruction parameter can be determined to be a second value.

[0150] It can be understood that in the embodiment of the present application, the first value may be set to 1 and the second value may be set to 0, or the first value may be set to true and the second value may be set to false, but this is not particularly limited in the present application.

[0151] Furthermore, in an embodiment of the present application, a flag asme_patch_geometry_absent_flag can be used to represent a first indication parameter. For example, in the present application, if there is a patch data unit that does not include a geometry patch, i.e., if there is at least one patch data unit that does not include geometry information and includes only texture information, the value of asme_patch_geometry_absent_flag can be set to 1; if there is no patch data unit that does not include a geometry patch, i.e., if all patch data units include texture information and geometry information, the value of asme_patch_geometry_absent_flag can be set to 0.

[0152] Furthermore, in an embodiment of the present application, the current atlas sequence may be an atlas sequence currently to be encoded. Specifically, the atlas sequence may include geometry information or texture information corresponding to one or more patch data units, i.e., may include geometry patches or texture patches corresponding to one or more patch data units.

[0153] As can be understood, in embodiments of the present application, for any patch data unit in the current atlas sequence, the patch data unit may include both geometry information and texture information, or may include only texture information without including geometry information.

[0154] As can be understood, in embodiments of the present application, for any patch data unit in the current atlas, the patch data unit may include both geometry information and texture information, or may include only texture information without including geometry information.

[0155] Furthermore, in the embodiment of the present application, different methods can be selected to encode a patch data unit depending on whether the patch data unit contains geometry information or does not contain geometry information, so it is necessary to determine a first indication parameter to indicate whether there is a patch data unit that does not contain geometry information.

[0156] In the implementation of the present application, the second indication parameter can indicate whether the corresponding patch data unit does not include geometry information, that is, the second indication parameter can be used to determine whether the patch data unit includes geometry information, where each patch data unit corresponds to one second indication parameter.

[0157] It can be understood that in the present embodiment, the second instruction parameter can be understood as a flag bit indicating whether the image does not include geometry information and only includes texture information. Specifically, the encoding device can determine a variable as the second instruction parameter, and can determine the second instruction parameter according to the value of the variable. Here, the value of the second instruction parameter can be a first value or a second value.

[0158] In addition, in an embodiment of the present application, if it is determined that the corresponding patch data unit does not contain geometry information, it can be determined that the value of the second instruction parameter is a first value, and if it is determined that the corresponding patch data unit does contain geometry information, it can be determined that the value of the second instruction parameter is a second value.

[0159] It can be understood that in the embodiment of the present application, the first value may be set to 1 and the second value may be set to 0, or the first value may be set to true and the second value may be set to false, but this is not particularly limited in the present application.

[0160] Furthermore, in the embodiment of the present application, a flag pdu_geometry_absent_flag may be used to represent a second indication parameter. For example, in the present application, if the corresponding patch data unit does not include a geometry patch, i.e., the corresponding patch data unit does not include geometry information and only includes texture information, the value of pdu_geometry_absent_flag may be set to 1; if the corresponding patch data unit includes a geometry patch, i.e., the corresponding patch data unit includes texture information and a geometry patch, the value of pdu_geometry_absent_flag may be set to 0.

[0161] In the present embodiment, the current atlas sequence may include at least one sub-atlas, where each sub-atlas includes at least one patch data unit. Correspondingly, a flag pdu_geometry_absent_flag[tileID][p] may be used to represent a second indication parameter, where tileID and p may be identifiers of the patch data unit. For example, in the present embodiment, a value of pdu_geometry_absent_flag[tileID][p] of 1 indicates that the p-th patch data unit in the sub-atlas of tileID does not include a geometry patch, i.e., the p-th patch data unit in the sub-atlas of tileID does not include geometry information and includes only texture information. A value of pdu_geometry_absent_flag[tileID][p] of 0 indicates that the p-th patch data unit in the sub-atlas of tileID includes a geometry patch, i.e., the p-th patch data unit in the sub-atlas of tileID includes texture information and geometry information.

[0162] Furthermore, in an embodiment of the present application, if there is a patch data unit that does not include geometry information, it is possible to further perform encoding processing based on texture information corresponding to the patch data unit to generate a codestream.

[0163] In addition, in an embodiment of the present application, if it is determined that a patch data unit does not contain geometry information and only contains texture information, the texture information corresponding to the patch data unit is first determined, and then an encoding process is performed on the patch data unit using the texture information to generate a code stream.

[0164] Furthermore, in embodiments of the present application, when determining an encoded patch data unit corresponding to a patch data unit based on texture information corresponding to the patch data unit, the encoding process may be completed using only the texture information, or the geometry information of the patch data unit may be first determined using the geometry information of the reference patch data unit, and then combined with the texture information of the patch data unit to complete the image encoding process.

[0165] It can be understood that, in the present application, according to one embodiment, if a patch data unit does not include geometry information, texture information corresponding to the patch data unit can be first determined, and then an encoding process can be performed on the patch data unit based on the texture information to generate a codestream, where during the encoding process, only the texture information included in the patch data unit is encoded.

[0166] In the embodiment of the present application, if a patch data unit includes only a texture patch (texture information) and does not include a geometry patch (geometry information), that is, if it is determined that the patch data unit does not include geometry information, there is no need to acquire geometry information, and only the corresponding texture information is acquired. Correspondingly, in the subsequent image encoding process, there is no need to consider the geometry information of the patch data unit, and encoding process is performed based only on the corresponding texture information.

[0167] It can be understood that in the present application, according to one embodiment, if a patch data unit does not include geometry information, texture information and reference patch data corresponding to the patch data unit can be first determined, and then first geometry information corresponding to the patch data unit can be determined based on the reference patch, and finally, an encoding process can be performed on the patch data unit based on the texture information and the first geometry information to generate a codestream, where during the encoding process, the encoding process is performed on the texture information and geometry information included in the patch data unit.

[0168] In the embodiment of the present application, if a patch data unit includes only a texture patch (texture information) and does not include a geometry patch (geometry information), that is, if it is determined that the patch data unit does not include geometry information, it is necessary to first determine the geometry information based on the reference patch data unit, and at the same time, determine the corresponding texture information. Correspondingly, in the subsequent image encoding process, the texture information of the patch data unit and the determined geometry information (first geometry information) can be combined to perform the encoding process.

[0169] Furthermore, in an embodiment of the present application, a reference patch data unit corresponding to the patch data unit can be first determined, and then, based on the reference patch data unit, first geometry information corresponding to the patch data unit can be determined.

[0170] In addition, in an embodiment of the present application, the encoding device may determine identification information of the reference patch data unit and write the identification information to the codestream. Here, the identification information may represent an index number of the reference patch data unit. For example, the identification information may be represented using pdu_geometry_reference_patch_id[tileID][p], where tileID and p may be indexes of the reference patch data unit, and the identification information pdu_geometry_absent_flag[tileID][p] may specify the pth patch data unit in the sub-atlas of tileID as the reference patch data unit.

[0171] It can be understood that in an embodiment of the present application, after determining the corresponding reference patch data unit, second geometry information corresponding to the reference patch data unit can be further determined, and the second geometry information can be used to determine the geometry information of the patch data unit, i.e., the first geometry information corresponding to the patch data unit.

[0172] In other words, in an embodiment of the present application, if it is determined that a patch data unit does not contain geometry information, the second geometry information of the reference patch data unit can be used to determine the first geometry information of the patch data unit, thereby performing an encoding process and generating a code stream based on the texture information and first geometry information of the patch data unit.

[0173] As can be understood, in an embodiment of the present application, the encoding device can write a first indication parameter into the code stream, and if the first indication parameter indicates that a patch data unit that does not contain geometry information exists, the encoding device can further write a second indication parameter into the code stream, and if the second indication parameter indicates that the patch data unit does not contain geometry information, the encoding device can further write identification information into the code stream.

[0174] Furthermore, in the embodiment of the present application, the encoding method by the encoding device may further include the following steps.

[0175] In step 204, if all patch data units contain geometry information, the texture information and first geometry information corresponding to the patch data units are determined.

[0176] In an embodiment of the present application, if all patch data units include geometry information, texture information and first geometry information corresponding to each patch data unit can be further determined.

[0177] As can be understood, in the embodiments of the present application, if it is determined that each patch data unit includes a geometry patch (geometry information) and a texture patch (texture information), the texture information and first geometry information corresponding to the patch data unit can be directly determined.

[0178] That is, in the embodiment of the present application, if all patch data units include geometry information, the texture information and first geometry information of the patch data units can be obtained directly.

[0179] In step 205, an encoding process is performed on the patch data unit based on the texture information and the first geometry information to generate a codestream.

[0180] In an embodiment of the present application, after determining texture information and first geometry information corresponding to a patch data unit, an encoding process can be performed on the patch data unit based on the texture information and first geometry information to generate a code stream.

[0181] It should be noted that in the embodiment of the present application, for a patch data unit that includes both texture information and first geometry information, the encoding process can be completed by directly using the corresponding texture information and first geometry information.

[0182] Furthermore, in the embodiment of the present application, the encoding method by the encoding device may further include the following steps.

[0183] In step 206, if the patch data unit includes geometry information, the texture information and first geometry information corresponding to the patch data unit are determined.

[0184] In an embodiment of the present application, if it is determined that the patch data unit includes geometry information, texture information and first geometry information corresponding to the patch data unit may be further determined.

[0185] As can be understood, in an embodiment of the present application, if it is determined that a patch data unit includes both a geometry patch (geometry information) and a texture patch (texture information), the texture information and first geometry information corresponding to the patch data unit can be directly determined.

[0186] That is, in the embodiment of the present application, when there is a patch data unit that does not contain geometry information, the specific situation of whether each patch data unit contains geometry information can be further determined. Here, when a patch data unit contains geometry information, that is, when the patch data unit contains both geometry information and texture information, the texture information and first geometry information of the patch data unit can be directly obtained.

[0187] In step 205, an encoding process is performed on the patch data unit based on the texture information and the first geometry information to generate a codestream.

[0188] In an embodiment of the present application, after determining texture information and first geometry information corresponding to a patch data unit, an encoding process can be performed on the patch data unit based on the texture information and first geometry information to generate a code stream.

[0189] Furthermore, in an embodiment of the present application, the method of rendering processing by the encoding device may include the following steps.

[0190] In step 207, a target window is constructed based on the patch data unit containing the geometry information.

[0191] In step 208, a shading process is performed on the target window based on the texture information of the patch data unit to obtain a rendered image.

[0192] In an embodiment of the present application, when the encoding process is completed using only texture information for a patch data unit that does not contain geometry information, a target window can be further constructed based on the patch data unit that contains geometry information, and then a shading process can be performed on the target window based on the texture information of the patch data unit to obtain a rendered image.

[0193] Furthermore, in the embodiment of the present application, after the encoding process is completed, a rendering process can be further performed using the image information obtained after encoding, where the rendering process can include two processes: a viewpoint view restoration process and a target window geometry reconstruction and pixel shading process.

[0194] In the viewpoint view restoration process, a general method is to sequentially restore the texture patches in all patch data units to their original positions in the texture view of the original viewpoint to which they belong, and sequentially restore the geometry patches in the patch data units to their original positions in the geometry view of the original viewpoint to which they belong. Meanwhile, in the process of scanning all patch data units to perform view restoration, in the present application, if the first instruction parameter indicates that all patch data units include both geometry information and texture information, or if the second instruction parameter indicates that the corresponding patch data unit includes both geometry information and texture information, viewpoint view restoration processing can be performed on the patch data units according to the general method described above. For patch data units that do not include geometry information and only include texture information, as indicated by the second instruction parameter (i.e., patch data units whose pdu_geometry_absent_flag has a value of 1), only the texture patches of the patch data units can be restored to their original positions in the texture view of the original viewpoint to which they belong, without performing a geometry patch restoration operation.

[0195] Furthermore, in the embodiment of the present application, when performing the geometry reconstruction and pixel shading process of the target window, first, the existing multi-view geometry image restored by the viewpoint view restoration process is used to reconstruct the geometry image of the target window, then each pixel point in the target window is scanned, and candidate pixels for each pixel are obtained from the multi-view texture image based on the position of the pixel point in the target window in three-dimensional space, and the pixel values ​​of each candidate pixel are weighted and fused based on the relative position and orientation relationship between each candidate pixel and the target window to obtain the pixel value of each pixel point in the target window.

[0196] In other words, in an embodiment of the present application, if there are cases where, for all patch data units, some patch data units contain both geometry information and texture information and the remaining patch data units contain only texture information, when rendering a target window, only some of the patch data units that contain geometry information can be used to reconstruct the target window geometry image, and the target window can be shaded using the texture information of all available patch data units.

[0197] Furthermore, in an embodiment of the present application, the method of rendering processing by the encoding device may include the following steps.

[0198] In step 209, a geometry-missing patch data unit reconstruction process is performed on the patch data units that do not contain geometry information based on the reference patch data units.

[0199] In step 210, a target window is constructed based on the geometry information of the patch data unit.

[0200] In step 208, a shading process is performed on the target window based on the texture information of the patch data unit to obtain a rendered image.

[0201] In an embodiment of the present application, for a patch data unit that does not contain geometry information, the geometry information of the patch data unit is first restored using the geometry information of the reference patch data unit, and then combined with the texture information of the patch data unit to complete the image encoding process. First, a geometry patch missing restoration process is performed on the patch data unit that does not contain geometry information based on the reference patch data unit, so that all patch data units contain both geometry information and texture information. Furthermore, a target window is constructed based on the geometry information of the patch data unit, and then a shading process is performed on the target window based on the texture information of the patch data unit to obtain a rendered image.

[0202] Furthermore, in the embodiment of the present application, after the encoding process is completed, a rendering process can be further performed using the image information obtained after encoding, where the rendering process may include the processes of missing geometry patch reconstruction, viewpoint view restoration, target window geometry reconstruction and pixel shading.

[0203] In the missing geometry patch reconstruction process, all patch data units can be scanned. If it is determined that a patch data unit is missing a geometry patch (geometry information), restoration of the missing geometry patch begins. Specifically, the geometry patch and viewpoint parameter information in the reference patch data unit can be used to restore the geometry patch of the patch data unit. After the above process, all patch data units include a texture patch and a geometry patch.

[0204] Furthermore, in an embodiment of the present application, after the determination of missing geometry patches is completed, for all patch data units, the texture patches in the patch data units may be sequentially restored to their original positions in the texture view of the original viewpoint to which they belong, and the geometry patches in the patch data units may be sequentially restored to their original positions in the geometry view of the original viewpoint to which they belong.

[0205] Furthermore, in the embodiment of the present application, when performing the geometry reconstruction and pixel shading process of the target window, first, the existing multi-view geometry image restored by the viewpoint view restoration process is used to reconstruct the geometry image of the target window, then each pixel point in the target window is scanned, and candidate pixels for each pixel are obtained from the multi-view texture image based on the position of the pixel point in the target window in three-dimensional space, and the pixel values ​​of each candidate pixel are weighted and fused based on the relative position and orientation relationship between each candidate pixel and the target window to obtain the pixel value of each pixel point in the target window.

[0206] In other words, in an embodiment of the present application, if there are cases where, for all patch data units, some patch data units include both geometry information and texture information and the remaining patch data units include only texture information, when rendering the target window, a geometry-missing patch data unit reconstruction process is first performed, so that all patch data units after the geometry-missing patch data unit reconstruction process include geometry patches and texture patches, and finally, the geometry information of all patch data units can be used to reconstruct the target window geometry image and the texture information of all patch data units can be used to shade the target window.

[0207] As described above, according to the encoding method proposed in the above steps 201 to 210, during encoding, for one non-diffuse reflection surface, G geometric patches and T texture patches are used for representation and transmission (both G and T are integers, and 1 ≤ G < T). This means that there may be patch data units that do not contain geometric patches during encoding. That is, while providing multiple texture information from different viewpoints for the same region, a sufficiently small amount of geometric information can be transmitted to represent the region, thereby supporting a sufficient and efficient representation of the non-diffuse reflection surface, effectively improving the rendering quality of the surface, reducing the redundancy of geometric data information, and thereby improving the encoding efficiency when compressing the non-diffuse reflection surface.

[0208] In addition, the encoding method proposed in this application introduces a mechanism that the patch data unit may not contain geometric patches in order to enable the MIV standard to support efficient encoding of non-diffuse reflection content.

[0209] Correspondingly, the encoding method proposed in this application sets syntax elements that support patch data units without geometric patches at the atlas sequence level and the patch data unit level based on the existing MIV standard syntax structure, so that patch data units without geometric patches can also be accurately decoded.

[0210] According to an encoding method of an embodiment of the present application, if a patch data unit does not contain geometry information, the patch data unit is encoded to generate a code stream, and a first indication parameter and a second indication parameter of the patch data unit are determined, where the first indication parameter is used to determine whether a patch data unit does not contain geometry information, and the second indication parameter is used to determine whether the patch data unit contains geometry information, and the first indication parameter and the second indication parameter are written into the code stream. That is, in this embodiment of the present application, the first indication parameter can determine whether a patch data unit does not contain geometry information, and the second indication parameter can determine whether the patch data unit does not contain geometry information, thereby supporting the transmission of different amounts of texture information and geometry information, effectively improving rendering quality by providing sufficient texture information, while reducing redundancy in geometry data, thereby fully representing the non-diffuse reflection characteristics of a surface, and improving codec efficiency and compression performance.

[0211] Based on the above embodiment, according to another embodiment of the present application, FIG. 5 is a schematic diagram 1 showing the configuration of a decoding device proposed in the embodiment of the present application. As shown in FIG. 5, the decoding device 300 proposed in the embodiment of the present application may include a decoding unit 301, a first determination unit 302, and a reconstruction unit 303.

[0212] the decoding unit 301 is configured to decode a codestream; the first determination unit 302 is configured to determine a first indication parameter, and, if the first indication parameter indicates that a patch data unit not including geometry information exists, to determine a second indication parameter corresponding to the patch data unit; The reconstruction unit 303 is configured to perform reconstruction processing based on texture information corresponding to the patch data unit when the second instruction parameter indicates that the patch data unit does not include geometry information.

[0213] FIG. 6 is a schematic diagram 2 showing the configuration of a decoding device proposed in an embodiment of the present application. As shown in FIG. 6, the decoding device 300 proposed in an embodiment of the present application may include a first processor 304, a first memory 305 that stores instructions executable by the first processor 304, a first communication interface 306, and a first bus 307 for connecting the first processor 304, the first memory 305, and the first communication interface 306.

[0214] Further, in an embodiment of the present application, the first processor 304 is configured to decode the codestream to determine a first instruction parameter, and if the first instruction parameter indicates that a patch data unit (PDU) that does not contain geometry information is present, determine a second instruction parameter corresponding to the patch data unit, and if the second instruction parameter indicates that the patch data unit does not contain geometry information, perform a reconstruction process based on texture information corresponding to the patch data unit.

[0215] FIG. 7 is a schematic diagram 1 showing the configuration of the encoding device proposed in the embodiment of the present application. As shown in FIG. 7, the encoding device 400 proposed in the embodiment of the present application may include an encoding unit 401.

[0216] The encoding unit 401 is configured to perform the following steps: if a patch data unit does not include geometry information, encode the patch data unit to generate a code stream, and at the same time determine a first instruction parameter and a second instruction parameter of the patch data unit, where the first instruction parameter is used to determine whether a patch data unit does not include geometry information, and the second instruction parameter is used to determine whether the patch data unit includes geometry information; and write the first instruction parameter and the second instruction parameter to the code stream.

[0217] Figure 8 is a schematic diagram 2 showing the configuration of the encoding device proposed in the embodiments of the present application. As shown in Figure 8, the encoding device 400 proposed in the embodiments of the present application may include a second processor 402, a second memory 403 that stores instructions executable by the second processor 402, a second communication interface 404, and a second bus 405 for connecting the second processor 402, the second memory 403, and the second communication interface 404.

[0218] Further, in an embodiment of the present application, the second processor 402 is configured to, if a patch data unit does not include geometry information, encode the patch data unit to generate a code stream, and at the same time, determine a first instruction parameter and a second instruction parameter of the patch data unit, where the first instruction parameter is used to determine whether a patch data unit does not include geometry information and the second instruction parameter is used to determine whether the patch data unit includes geometry information, and write the first instruction parameter and the second instruction parameter to the code stream.

[0219] Furthermore, each functional module in this embodiment may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated unit may be realized in the form of hardware or in the form of a software functional module.

[0220] If an integrated unit is implemented in the form of a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, an essential part of the technical solution of this embodiment, or a part of the contribution to the prior art, or all or part of the solution of the technology, can be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) or a processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0221] An embodiment of the present application provides an encoding device and a decoding device, which can determine whether there is a patch data unit that does not contain geometry information through a first indication parameter, and can determine whether the patch data unit does not contain geometry information through a second indication parameter, thereby supporting the transmission of different amounts of texture information and geometry information, effectively improving rendering quality through sufficient texture information, while reducing redundancy in geometry data, thereby fully expressing the non-diffuse reflection characteristics of the surface, and improving codec efficiency and compression performance.

[0222] An embodiment of the present application provides a computer-readable storage medium having a program stored thereon, the program causing a processor to execute the method described in the above embodiment.

[0223] Specifically, the program instructions corresponding to the decryption method in this embodiment can be stored in a storage medium such as an optical disk, a hard disk, or a U disk. When the program instructions corresponding to the decryption method in the storage medium are read or executed by an electronic device, the electronic device can: decoding the codestream to determine a first indication parameter; if the first indication parameter indicates that a patch data unit not containing geometry information exists, determining a second indication parameter corresponding to the patch data unit; If the second instruction parameter indicates that the patch data unit does not include geometry information, a step of performing a reconstruction process based on texture information corresponding to the patch data unit is executed.

[0224] When the program instructions corresponding to the encoding method in the storage medium are read or executed by an electronic device, the electronic device: If a patch data unit does not contain geometry information, encoding the patch data unit to generate a codestream, and simultaneously determining a first indication parameter and a second indication parameter of the patch data unit, wherein the first indication parameter is used to determine whether there is a patch data unit that does not contain geometry information, and the second indication parameter is used to determine whether the patch data unit contains geometry information; and writing the first instruction parameter and the second instruction parameter into the codestream.

[0225] It should be understood by those skilled in the art that embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied in one or more computer-usable storage media (including, but not limited to, disk memory, optical memory, etc.) containing computer-usable program code.

[0226] The present application will be described with reference to implementation flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. Note that each process and / or block in the implementation flowcharts and / or block diagrams, and combinations of processes and / or blocks in the implementation flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine. The instructions, executed by the processor of the computer or other programmable data processing device, thereby produce an apparatus for implementing the functionality specified in one or more processes in the implementation flowcharts and / or one or more blocks in the block diagrams.

[0227] These computer program instructions may further be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory can produce an article of manufacture that includes an instruction apparatus that implements the functions specified in one or more processes of the implementation flowcharts and / or one or more blocks of the block diagrams.

[0228] These computer program instructions may further be loaded into a computer or other programmable data processing apparatus, thereby generating a computer-implemented process by executing a series of operational steps on the computer or other programmable data processing apparatus, whereby the instructions executed on the computer or other programmable data processing apparatus provide steps for implementing the functions specified in one or more processes of the implementation flowcharts and / or one or more blocks of the block diagrams.

[0229] The above description is only a preferred embodiment of the present application, and is not intended to limit the protection scope of the present application. [Industrial Applicability]

[0230] The present invention provides a codec method, an encoding device, a decoding device, and a storage medium, in which the decoding device decodes a codestream to determine a first indication parameter, and if the first indication parameter indicates that a patch data unit containing no geometry information exists, determines a second indication parameter corresponding to the patch data unit, and if the second indication parameter indicates that the patch data unit contains no geometry information, performs reconstruction processing based on texture information corresponding to the patch data unit. If the patch data unit does not contain geometry information, the decoding device encodes the patch data unit to generate a codestream, and simultaneously determines the first indication parameter and the second indication parameter of the patch data unit, where the first indication parameter is used to determine whether a patch data unit containing no geometry information exists and the second indication parameter is used to determine whether the patch data unit contains geometry information, and writes the first indication parameter and the second indication parameter to the codestream. That is, in the embodiment of the present application, the first indication parameter can determine whether there is a patch data unit that does not contain geometry information, and the second indication parameter can determine whether the patch data unit does not contain geometry information, thereby supporting the transmission of different amounts of texture information and geometry information, effectively improving the rendering quality through sufficient texture information, while reducing the redundancy of geometry data, thereby fully expressing the non-diffuse reflection characteristics of the surface, and improving codec efficiency and compression performance.

Claims

1. A decoding method applied to a decoding device, comprising: decoding the codestream to determine a first indication parameter; determining a second instruction parameter corresponding to one patch data unit of the plurality of patch data units when the first instruction parameter indicates that there is at least one patch data unit that does not include geometry information among the plurality of patch data units; If the second instruction parameter indicates that the patch data unit does not include geometry information, performing a reconstruction process based on texture information corresponding to the patch data unit.

2. If the first indication parameter indicates that there is at least one patch data unit in the current atlas sequence that does not contain geometry information among a plurality of patch data units, determining a second indication parameter corresponding to one patch data unit in the plurality of patch data units in the current atlas sequence; or determining a second instruction parameter corresponding to one patch data unit of the plurality of patch data units of the current atlas when the first instruction parameter indicates that at least one patch data unit does not include geometry information among the plurality of patch data units in the current atlas; The decoding method of claim 1 .

3. When the second instruction parameter indicates that the patch data unit does not include geometry information, the step of performing reconstruction processing based on texture information corresponding to the patch data unit includes: if the second indication parameter indicates that the patch data unit does not contain geometry information, determining texture information corresponding to the patch data unit and identification information of a reference patch data unit; determining first geometry information corresponding to the patch data unit based on the identification information; performing a reconstruction process on the patch data unit based on the texture information and the first geometry information; The decoding method according to claim 2 .

4. The decoding method comprises: If the first indication parameter indicates that all patch data units include geometry information, determining texture information and first geometry information corresponding to the patch data units; performing a reconstruction process on the patch data unit based on the texture information and the first geometry information; or The decoding method comprises: If the second indication parameter indicates that the patch data unit includes geometry information, determining texture information and first geometry information corresponding to the patch data unit; performing a reconstruction process on the patch data unit based on the texture information and the first geometry information; The decoding method according to claim 2.

5. The decoding method comprises: constructing a target window based on the patch data unit including geometry information; performing shading processing on the target window based on texture information of the patch data unit to obtain a rendered image; 5. The decoding method according to claim 4.

6. The decoding method comprises: performing a geometry-missing patch data unit reconstruction process on a patch data unit that does not contain geometry information based on the reference patch data unit; constructing a target window based on geometry information of the patch data unit; performing shading processing on the target window based on texture information of the patch data unit to obtain a rendered image; The decoding method according to claim 3.

7. When the value of the first instruction parameter is a first value, the first instruction parameter indicates that there is at least one patch data unit among the plurality of patch data units that does not include geometry information; when the value of the first instruction parameter is a second value, all patch data units indicated by the first instruction parameter include geometry information; When the value of the second instruction parameter is the first value, the second instruction parameter indicates that the patch data unit does not include geometry information; When the value of the second indication parameter is the second value, the second indication parameter indicates that the patch data unit includes geometry information. The decoding method of claim 1 .

8. An encoding method applied to an encoding device, comprising: if at least one patch data unit of the plurality of patch data units does not include geometry information, determining a first indication parameter and a second indication parameter of one patch data unit of the plurality of patch data units, wherein the first indication parameter is used to determine whether there is at least one patch data unit of the plurality of patch data units that does not include geometry information, and the second indication parameter is used to determine whether the patch data unit includes geometry information; and writing said first indication parameter and said second indication parameter into a codestream.

9. The encoding method comprises: if the patch data unit does not contain geometry information, determining texture information corresponding to the patch data unit; and performing an encoding process on the patch data unit based on the texture information to generate the codestream. The encoding method according to claim 8.

10. The encoding method comprises: if the patch data unit does not contain geometry information, determining texture information and a reference patch data unit corresponding to the patch data unit; determining first geometry information corresponding to the patch data unit based on the reference patch data unit; performing an encoding process on the patch data unit based on the texture information and the first geometry information to generate the codestream; and writing an identification of the reference patch data unit into the codestream. The encoding method according to claim 8.

11. The encoding method comprises: If all of the patch data units include geometry information, determining texture information and first geometry information corresponding to the patch data units; further comprising performing an encoding process on the patch data unit based on the texture information and the first geometry information to generate the codestream; or The encoding method comprises: If the patch data unit includes geometry information, the method further includes determining texture information and first geometry information corresponding to the patch data unit, and performing an encoding process on the patch data unit based on the texture information and the first geometry information to generate the codestream. The encoding method according to any one of claims 8 to 10.

12. The encoding method comprises: constructing a target window based on the patch data unit including geometry information; performing shading processing on the target window based on texture information of the patch data unit to obtain a rendered image; The encoding method according to claim 9.

13. The encoding method comprises: performing a geometry-missing patch data unit reconstruction process on a patch data unit that does not contain geometry information based on the reference patch data unit; constructing a target window based on geometry information of the patch data unit; performing shading processing on the target window based on texture information of the patch data unit to obtain a rendered image; The encoding method of claim 10.

14. A decoding device comprising: a first processor; and a first memory storing instructions executable by the first processor, the instructions causing the first processor to perform a method according to any one of claims 1 to 7.

15. 14. An encoding device comprising: a second processor; and a second memory storing instructions executable by the second processor, the instructions causing the second processor to perform the method of any one of claims 8 to 13.

Images