Point group symbolization / decryption method, symbolizer, decryptor, and computer storage medium

Abstract

A point cloud encoding / decoding method, an encoder, a decoder, and a computer storage medium are provided. The decoder identifies filtering flag information corresponding to an initial point cloud by decoding a bit stream, and the filtering flag information is used to identify whether or not to perform a filtering process on a reconstructed point cloud corresponding to the initial point cloud (101). If the filtering flag information indicates that a filtering process is to be performed on the reconstructed point cloud, the decoder identifies a filtering coefficient (102). The decoder uses the filtering coefficient to obtain a filtered point cloud corresponding to the reconstructed point cloud (103). The decoder updates the reconstructed point cloud using the filtered point cloud (104). The encoder identifies a reconstructed value of attribute information of a point in the initial point cloud, and constructs a reconstructed point cloud corresponding to the initial point cloud based on the reconstructed value (201). The encoder identifies a filtering coefficient based on the initial point cloud and the reconstructed point cloud (202). The encoder uses the filtering coefficient to obtain a filtered point cloud corresponding to the reconstructed point cloud (203). The encoder identifies filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud (204). If the filtering flag information indicates that a filtering process is to be performed on the reconstructed point cloud, the encoder writes the filtering flag information and filtering coefficients to the bitstream (205).

Description

Technical Field

[0001] Embodiments of the present application relate to the field of video encoding and decoding technologies, and in particular, to a point cloud encoding and decoding method, an encoder, a decoder, and a computer storage medium.

Background Art

[0002] Currently, in the encoder framework of geometry-based point cloud compression (G-PCC), the encoding of the attribute information of the point cloud mainly targets the encoding of color information. First, the color information is converted from the RGB color space to the YUV color space. Then, by recoloring the point cloud using the reconstructed geometric information, the unencoded attribute information is made to correspond to the reconstructed geometric information. In the encoding of color information, operations are mainly performed by using three color attribute transformation encoding methods, namely predicting transform, lifting transform, and RAHT (regional adaptive hierarchical transform), to finally generate a binary bitstream.

[0003] However, in general point cloud encoding and decoding methods, there is a problem that the prediction is not sufficiently accurate. As a result, the difference between the reconstructed point cloud and the original point cloud is relatively large, which affects the quality of the entire point cloud, so that the encoding and decoding efficiency decreases.

Summary of the Invention

[0004] In embodiments of the present application, a point cloud encoding and decoding method, an encoder, a decoder, and a computer storage medium are provided. Thereby, the quality of the point cloud can be improved, and the encoding and decoding efficiency can be greatly improved.

[0005] The technical solution of the embodiments of the present application can be realized as follows.

[0006] In a first aspect, in an embodiment of the present application, a point cloud decoding method is provided. The method is applied to a decoder and includes the following. By decoding a bitstream, filtering flag information corresponding to an initial point cloud is identified. The filtering flag information is used to identify whether to perform filtering processing on a reconstructed point cloud corresponding to the initial point cloud. When the filtering flag information instructs to perform filtering processing on the reconstructed point cloud, a filtering coefficient is identified. Using the filtering coefficient, a filtered point cloud corresponding to the reconstructed point cloud is obtained. The reconstructed point cloud is updated using the filtered point cloud.

[0007] In a second aspect, in an embodiment of the present application, a point cloud encoding method is provided. The method is applied to an encoder and includes the following. A reconstruction value of the attribute information of the points in the initial point cloud is identified, and a reconstructed point cloud corresponding to the initial point cloud is constructed based on the reconstruction value. A filtering coefficient is identified based on the initial point cloud and the reconstructed point cloud. Using the filtering coefficient, a filtered point cloud corresponding to the reconstructed point cloud is obtained. Filtering flag information corresponding to the initial point cloud is identified based on the reconstructed point cloud and the filtered point cloud. The filtering flag information is used to identify whether to perform filtering processing on the reconstructed point cloud. When the filtering flag information instructs to perform filtering processing on the reconstructed point cloud, the filtering flag information and the filtering coefficient are written into the bitstream.

[0008] In a third aspect, in an embodiment of the present application, an encoder is provided. The encoder includes a first identification unit, a construction unit, and an encoding unit. The first identification unit is configured to identify a reconstruction value of the attribute information of the points in the initial point cloud. The construction unit is configured to construct a reconstructed point cloud corresponding to the initial point cloud based on the reconstruction value. The first specifying unit is further configured to specify a filtering coefficient based on the initial point cloud and the reconstructed point cloud, obtain a filtered point cloud corresponding to the reconstructed point cloud using the filtering coefficient, and specify filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud. The filtering flag information is used to specify whether to perform filtering processing on the reconstructed point cloud. When the filtering flag information instructs to perform filtering processing on the reconstructed point cloud, the encoding unit is configured to write the filtering flag information and the filtering coefficient into the bitstream.

[0009] In a fourth aspect, in an embodiment of the present application, an encoder is provided. The encoder includes a first processor and a first memory storing instructions executable by the first processor. When the instructions are executed, the first processor executes the above-described point cloud encoding method.

[0010] In a fifth aspect, in an embodiment of the present application, a decoder is provided. The decoder includes a decoding unit, a second specifying unit, and an updating unit. The decoding unit is configured to decode the bitstream. The second specifying unit is configured to specify filtering flag information corresponding to the initial point cloud. The filtering flag information is used to specify whether to perform filtering processing on the reconstructed point cloud corresponding to the initial point cloud. When the filtering flag information instructs to perform filtering processing on the reconstructed point cloud, the second specifying unit is further configured to specify a filtering coefficient and obtain a filtered point cloud corresponding to the reconstructed point cloud using the filtering coefficient. The updating unit is configured to update the reconstructed point cloud using the filtered point cloud.

[0011] In a sixth aspect, in an embodiment of the present application, a decoder is provided. The decoder includes a second processor and a second memory storing instructions executable by the second processor. When the instructions are executed, the second processor executes the point cloud decoding method.

[0012] In a seventh aspect, in an embodiment of the present application, a computer storage medium is provided. A computer program is stored in the computer storage medium. When the computer program is executed by a first processor, the point cloud encoding method is executed, or when the computer program is executed by a second processor, the point cloud decoding method is executed.

[0013] In an embodiment of the present application, a point cloud encoding / decoding method, an encoder, a decoder, and a computer storage medium are disclosed. The decoder identifies filtering flag information corresponding to an initial point cloud by decoding a bitstream. The filtering flag information is used to identify whether to perform filtering processing on a reconstructed point cloud corresponding to the initial point cloud. When the filtering flag information indicates that filtering processing is to be performed on the reconstructed point cloud, the decoder identifies a filtering coefficient. The decoder obtains a filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient. The decoder updates the reconstructed point cloud by using the filtered point cloud. The encoder identifies a reconstruction value of attribute information of points in the initial point cloud and constructs a reconstructed point cloud corresponding to the initial point cloud based on the reconstruction value. The encoder identifies a filtering coefficient based on the initial point cloud and the reconstructed point cloud. The encoder obtains a filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient. The encoder identifies filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud. The filtering flag information is used to identify whether to perform filtering processing on the reconstructed point cloud. When the filtering flag information indicates that filtering processing is to be performed on the reconstructed point cloud, the encoder writes the filtering flag information and the filtering coefficient into the bitstream. In other words, in the point cloud encoding / decoding method according to the present application, Symbolization On one side, an initial point cloud and a reconstructed point cloud are used to calculate a filtering coefficient for performing filtering processing, and after it is determined that filtering processing is to be performed on the reconstructed point cloud, the filtering coefficient is transmitted to the decoding side. Accordingly, the decoding side can directly decode to obtain the filtering coefficient and use the filtering coefficient to perform filtering processing on the reconstructed point cloud. Thereby, the optimization of the reconstructed point cloud can be achieved, the quality of the point cloud can be improved, and the encoding / decoding efficiency can be greatly improved.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Embodiments for Carrying Out the Invention

[0015] To better understand the features and technical content of the embodiments of this application, the implementation of the embodiments of this application will be described in detail below with reference to the drawings. However, the attached drawings are only used for reference and do not limit the embodiments of this application.

[0016] In the embodiment of this application, in the encoder framework of G-PCC, after dividing the point cloud of the input 3D image model into slices, each slice is encoded independently.

[0017] FIG. 1 is a block diagram showing the process of G-PCC encoding. The G-PCC encoding process shown in FIG. 1 is applied to a point cloud coder. For the point cloud data waiting to be encoded, first, the point cloud data is divided into a plurality of slices by slice division. In each slice, the geometric information of the point cloud and the attribute information corresponding to each point cloud are encoded separately. In the process of geometric encoding, coordinate transformation is performed on the geometric information so that all the point clouds are included in one bounding box. Then, quantization is performed, which mainly plays the role of scaling. Due to the rounding process of quantization, the geometric information of some point clouds becomes the same, so it is determined whether to remove duplicate points based on parameters. The process of quantization and removal of duplicate points is also called the voxelization process. Next, octree division is performed on the bounding box. In the encoding process of geometric information based on the octree, the bounding box is divided into eight equal sub-cubes, and the non-empty sub-cubes (including the points in the point cloud) are continuously divided into eight equal parts until the leaf nodes obtained by the division become unit cubes of 1×1×1. Arithmetic encoding is performed on the points in the leaf nodes to generate a binary geometric bitstream, that is, a geometric code stream. In the encoding process of geometric information based on triangle soup (trisoup), first, octree division is also performed. Different from the encoding process of geometric information based on the octree, in the encoding process of geometric information based on the trisoup, it is not necessary to gradually divide the point cloud into unit cubes with a side length of 1×1×1. After dividing the point cloud into blocks with a side length of W, the division is stopped. Based on the surface formed by the distribution of the point cloud in each block, up to 12 intersections (vertices) generated by the surface and the 12 sides of the block are obtained. Arithmetic encoding (surface fitting based on the intersections) is performed on the intersections to generate a binary geometric bitstream, that is, a geometric code stream. The intersections are also used to realize geometric reconstruction.The reconstructed geometric information is used when encoding the attributes of the point cloud.

[0018] In the process of attribute encoding, after the geometric encoding is completed and the geometric information is reconstructed, color conversion is performed. That is, the color information (i.e., attribute information) is converted from the RGB color space to the YUV color space. Then, the point cloud is recolored using the reconstructed geometric information, and the unencoded attribute information is associated with the reconstructed geometric information. Attribute encoding mainly targets color information. There are mainly two conversion methods in the encoding process of color information. One is a distance-based lifting transform that depends on level of detail (LOD) division. The other is RAHT (regional adaptive hierarchical transform). Through these two methods, the color information is converted from the spatial domain to the frequency domain, high-frequency coefficients and low-frequency coefficients are obtained by the conversion, and finally the coefficients are quantized (i.e., quantization coefficients). Finally, the geometric encoding data after octree division and surface fitting and the quantization coefficient processed attribute encoding data are slice-synthesized, and the intersection coordinates of each block are sequentially encoded (i.e., arithmetic encoding) to generate a binary attribute bitstream, that is, an attribute code stream.

[0019] Figure 2 is a block diagram showing the process of G-PCC decoding. The G-PCC decoding process shown in Figure 2 is applied to a point cloud decoder. For the obtained binary bitstream, first, the geometric bitstream and the attribute bitstream in the binary bitstream are decoded independently. When decoding the geometric bitstream, the geometric information of the point cloud is obtained through arithmetic decoding - octree synthesis - surface fitting - geometric reconstruction - inverse coordinate transformation. When decoding the attribute bitstream, the attribute information of the point cloud is obtained through arithmetic decoding - inverse quantization - lifting inverse transform based on LOD or inverse transform based on RAHT - inverse color conversion. Based on the geometric information and the attribute information, the three-dimensional image model of the point cloud data waiting to be encoded is restored.

[0020] In the block diagram of the G-PCC encoding process shown in FIG. 1 above, the LOD division is mainly used in two ways: predictive transformation and lifting transformation in the attribute transformation of the point cloud.

[0021] After the geometric reconstruction of the point cloud, the LOD division process is performed. At this time, the geometric coordinate information of the point cloud can be directly obtained. The point cloud is divided into multiple LODs according to the Euclidean distance between the points in the point cloud, the color of the points in the LOD is sequentially decoded, the value of the number of zeros (zero_cnt) in the Zero-Run Length encoding technology is calculated, and the residual is decoded.

[0022] FIG. 3 is a schematic diagram showing zero-run length encoding. As shown in FIG. 3, a decoding operation is performed based on the zero-run length encoding method. First, the value of the first zero-cnt in the bitstream is analyzed. If zero-cnt is greater than 0, zero_cnt--. In this case, it indicates that the residual is 0. If zero-cnt is equal to 0, it indicates that the attribute residual of this point is not 0. Next, the corresponding residual value is decoded (decoder.decode(values)), the decoded residual is inverse quantized, added to the predictedColor of the current point to obtain the reconstructed value of the current point, and this operation is continuously executed until the decoding of all points in the point cloud is completed.

[0023] That is, the color reconstruction value of the current point is reconstructedColor, and it needs to be calculated based on the predicted color value (predictedColor) in the current prediction mode and the inverse quantization residual value (residual) of the color in the current prediction mode. That is, reconstructedColor = predictedColor + residual.

[0024] Furthermore, the current point is set as the nearest neighbor point of the subsequent points in the LOD, and attribute prediction is performed on the subsequent points using the color reconstruction value of the current point.

[0025] However, in the existing encoding / decoding framework, only basic reconstruction is performed on the point cloud sequence, and after reconstruction, no certain processing is performed to further improve the quality of the color attributes of the reconstructed point cloud. As a result, the difference between the reconstructed point cloud and the original point cloud is relatively large, the distortion is large, which affects the quality of the entire point cloud.

[0026] To solve the above problems, in the embodiments of the present application, a point cloud encoding / decoding method is provided. Symbolization On one side, using the initial point cloud and the reconstructed point cloud, calculate the filtering coefficient for performing filtering processing, and after specifying that filtering processing is to be performed on the reconstructed point cloud, transmit the filtering coefficient to the decoding side. Accordingly, the decoding side can directly decode to obtain the filtering coefficient, and use the filtering coefficient to perform filtering processing on the reconstructed point cloud. Thereby, the optimization of the reconstructed point cloud can be achieved, the quality of the point cloud can be improved, and the encoding / decoding efficiency can be greatly improved.

[0027] It should be noted that the point cloud encoding / decoding method according to the embodiments of the present application can affect the arithmetic encoding and the subsequent part in the point cloud encoding framework, and can affect the subsequent part after attribute reconstruction in the decoding framework.

[0028] That is, the point cloud encoding method according to the embodiments of the present application is applicable to the arithmetic encoding and the subsequent part as shown in FIG. 1, and accordingly, the point cloud decoding method according to the embodiments of the present application can also be applied to the subsequent part after attribute reconstruction as shown in FIG. 2. That is, the point cloud encoding / decoding method in the embodiments of the present application may be applied to a video encoding system, may be applied to a video decoding system, and may further be applied to both a video encoding system and a video decoding system, but is not specifically limited in the embodiments of the present application.

[0029] Hereinafter, the technical solution according to the embodiments of the present application will be clearly and completely described together with the drawings in the embodiments of the present application.

[0030] In one embodiment of the present application, a point cloud decoding method applied to a point cloud decoder is provided. FIG. 4 is a first flowchart showing the realization of point cloud decoding. As shown in FIG. 4, in the embodiment of the present application, the point cloud decoding method executed by the decoder can include the following steps.

[0031] Step 101: By decoding the bitstream, identify the filtering flag information corresponding to the initial point cloud. The filtering flag information is used to identify whether to perform filtering processing on the reconstructed point cloud corresponding to the initial point cloud.

[0032] In the embodiment of the present application, the decoder can first decode the bitstream, thereby identifying the filtering flag information corresponding to the initial point cloud.

[0033] Note that in the embodiment of the present application, the filtering flag information can be used to identify whether to perform filtering processing on the reconstructed point cloud corresponding to the initial point cloud. Specifically, the filtering flag information can further be used to identify for which color component in the reconstructed point cloud the filtering processing is to be performed.

[0034] Note that in the embodiment of the present application, for one point in the initial point cloud, when decoding the one point, the one point can be set as a point waiting to be decoded in the initial point cloud, and there are a plurality of decoded points around the one point.

[0035] Furthermore, in the embodiment of the present application, one point in the initial point cloud corresponds to one piece of geometric information and one piece of attribute information. The geometric information represents the spatial position of the one point, and the attribute information represents the reconstructed attribute value of the one point.

[0036] In addition, in the embodiments of the present application, the attribute information can include color information. Specifically, the attribute information may be color information in any color space. For example, the attribute information may be color information in the RGB space, color information in the YUV space, or color information in the YCbCr space, and is not specifically limited in the present application.

[0037] Furthermore, in the embodiments of the present application, when the attribute information is color information in the RGB space, the color components include the R component, the G component, and the B component. When the attribute information is color information in the YUV space, the color components include the Y component, the U component, and the V component. When the attribute information is color information in the YCbCr space, the color components include the Y component, the Cb component, and the Cr component.

[0038] As can be understood, in the embodiments of the present application, for one point in the initial point cloud, the attribute information of the one point may be the reconstructed attribute value after decoding. The attribute information may be color information, reflectivity, or other attributes, and is not limited thereto in the present application.

[0039] In addition, in the embodiments of the present application, the decoder can further identify the residual value of the attribute information of the points in the initial point cloud by decoding the bitstream.

[0040] Furthermore, in the embodiments of the present application, for one point in the initial point cloud, first, the residual value and the predicted value corresponding to the attribute information of the one point can be identified, and then, by calculating using the residual value and the predicted value, the reconstructed value of the attribute information of the one point can be obtained.

[0041] Specifically, in the embodiments of the present application, when specifying a predicted value corresponding to the attribute information of one point in the initial point cloud, by using the geometric information and attribute information of a plurality of target adjacent points of the one point and the geometric information of the one point to predict the attribute information of the current point, the corresponding predicted value can be obtained.

[0042] As can be understood, in the embodiments of the present application, for one point in the initial point cloud, after specifying the reconstruction value of the attribute information of the one point, the one point can be used as the nearest neighbor point of the subsequent point in the LOD, and thereby, continue to perform attribute prediction of the subsequent point by using the reconstruction value of the attribute information of the one point.

[0043] Furthermore, in the embodiments of the present application, after specifying the reconstruction value of the attribute information of the points in the initial point cloud, it is possible to specify the corresponding reconstructed point cloud by using the reconstruction values of the attribute information of each point in the initial point cloud.

[0044] Note that in the present application, the initial point cloud can be directly obtained by an encoding / decoding program called a point cloud reading function. The reconstructed point cloud corresponding to the initial point cloud is obtained after attribute decoding, attribute reconstruction, and geometric offset.

[0045] That is, in the embodiments of the present application, first, the reconstruction value of the attribute information of the points in the initial point cloud is specified, and then, based on the reconstruction value, the reconstructed point cloud corresponding to the initial point cloud can be constructed. Specifically, by decoding the bit stream, the residual value of the attribute information of the points in the initial point cloud is specified, and then, by using the residual value and predicted value of the points, the reconstruction value of the corresponding attribute information is specified, and further, the reconstructed point cloud corresponding to the initial point cloud can be constructed.

[0046] In the embodiments of the present application, the filtering flag information can include identification information of color components. The identification information of one color component can be used to indicate whether to perform filtering processing on the one color component.

[0047] As can be seen from the above, since the filtering flag information is specified based on the identification information corresponding to each color component, it can be used not only to specify whether to perform filtering processing on the reconstructed point cloud, but also specifically to specify for which color component the filtering processing is to be performed.

[0048] In the present application, when the value of the identification information is the first value, it indicates not to perform filtering processing on the color component. When the value of the identification information is the second value, it indicates to perform filtering processing on the color component.

[0049] Furthermore, in the present application, when the identification information of all color components is the first value, that is, when it indicates not to filter each of all color components, it can be specified that the filtering flag information indicates not to perform filtering processing on the reconstructed point cloud. Accordingly, when the identification information of all color components is not the first value, that is, when it indicates to filter at least one color component, it can be specified that the filtering flag information indicates to perform filtering processing on the reconstructed point cloud.

[0050] Exemplarily, in the present application, the first value is 0 and the second value is 1. Or, the first value is set to false and the second value is set to true.

[0051] Step 102: When the filtering flag information indicates to perform filtering processing on the reconstructed point cloud, specify the filtering coefficient.

[0052] In an embodiment of the present application, the decoder can further identify a filtering coefficient for performing filtering processing on the initial point cloud after identifying filtering flag information corresponding to the initial point cloud by decoding a bit stream, when the filtering flag information instructs to perform filtering processing on the reconstructed point cloud.

[0053] Note that, in an embodiment of the present application, the filtering coefficient can be used for Wiener filtering, that is, the filtering coefficient is the coefficient of a Wiener filter. The filtering coefficient is the coefficient of an optimal filter used for filtering processing.

[0054] Specifically, the Wiener filter is a linear filter with least squares as the optimal criterion. Under certain constraint conditions, the sum of squares of the difference between its output and a certain predetermined function (usually called the desired output) is minimized, and finally it can be transformed into obtaining one Toeplitz equation through mathematical operations. The Wiener filter is also called a least squares filter or a least mean square filter.

[0055] As can be understood, in an embodiment of the present application, when identifying the filtering coefficient, the decoder can obtain the filtering coefficient by decoding the bit stream.

[0056] Furthermore, in an embodiment of the present application, after the decoder identifies the filtering flag information corresponding to the initial point cloud by decoding the bit stream, when the filtering flag information instructs not to perform filtering processing on the reconstructed point cloud, the decoder does not need to identify the filtering coefficient.

[0057] Step 103: Obtain a filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient.

[0058] In an embodiment of the present application, when filtering flag information instructs to perform filtering processing on a reconstructed point cloud, after specifying a filtering coefficient corresponding to the initial point cloud, further, filtering processing can be performed on the reconstructed point cloud by using the filtering coefficient, whereby a filtered point cloud corresponding to the reconstructed point cloud can be obtained.

[0059] Note that in an embodiment of the present application, filtering can be performed individually on each color component of the point cloud. For example, for the color information in the YUV space, a filtering coefficient vector for the Y component, a filtering coefficient vector for the U component, and a filtering coefficient vector for the V component can be specified respectively. These filtering coefficient vectors together constitute the filtering coefficient of the point cloud.

[0060] Furthermore, in an embodiment of the present application, when performing filtering processing on a reconstructed point cloud by using a filtering coefficient, first, according to the reconstructed component value of the color component of the color information of the point in the reconstructed point cloud and the filter type, a second attribute parameter corresponding to the color component can be specified. Next, based on the filtering coefficient vector corresponding to the color component and the second attribute parameter, a filtering value of the attribute information corresponding to the color component can be specified. Finally, based on the filtering value of the attribute information corresponding to the color component, a filtered point cloud is constructed.

[0061] Note that in the present application, if the attribute information is the color information in any space, for example, if the attribute information is the color information in the YUV space, first, based on the initial component value and the reconstructed component value of the color component (for example, the Y component, the U component, the V component) and the filter type, the first attribute parameter and the second attribute parameter of the color component can be specified.

[0062] In the embodiments of the present application, the first attribute parameter is used to specify the attribute value of the color component of the points in the initial point cloud, and the second attribute parameter is used to specify the attribute value of the color component of the points in the reconstructed point cloud. Specifically, the first attribute parameter represents the color attribute value of a point in the initial point cloud in one color component, and the second attribute parameter represents the color attribute values of a point in the reconstructed point cloud and its k adjacent points in one color component.

[0063] Furthermore, in the embodiments of the present application, the filter type can be used to indicate the filter order, and / or the filter shape, and / or the filter dimension. The filter shape includes a rhombus, a rectangle, etc., and the filter dimension includes one dimension and multiple dimensions.

[0064] That is, in the present application, different filter types may correspond to filters with different orders. For example, they may correspond to filters with orders k of 16, 32, and 128. Different filter types may also correspond to filters with different dimensions. For example, they may correspond to one-dimensional filters, two-dimensional filters, and multi-dimensional filters. Also, in the case of two-dimensional filters and multi-dimensional filters, different filter types may indicate the adoption of different filter shapes. For example, a rhombus filter, a rectangle filter, etc. may be adopted.

[0065] In the present application, the second attribute parameter corresponding to one color component is calculated. Next, the filtering value of the attribute information corresponding to the one color component is specified by using the second attribute parameter and the filtering coefficient vector corresponding to the one color component. After traversing all the color components and obtaining the filtering values of the attribute information corresponding to each color component, the filtered point cloud can be finally specified based on the filtering values of the attribute information corresponding to each color component.

[0066] To make it understandable, in the embodiments of the present application, when performing filtering processing on the reconstructed point cloud using a filter, both the reconstructed point cloud and the filtering coefficient can be input into the filter, that is, the input of the filter is the filtering coefficient and the reconstructed point cloud. Thereby, finally, the filtering processing on the reconstructed point cloud can be completed based on the filtering coefficient, and the corresponding filtered point cloud can be obtained.

[0067] Note that in the present application, since the filtering parameter is obtained based on the original point cloud and the reconstructed point cloud, by using the filtering coefficient for the reconstructed point cloud, the original point cloud can be restored to the maximum extent.

[0068] Specifically, in the present application, when filtering the reconstructed point cloud based on the filtering coefficient, for one color component in the color information, the filtering value of the attribute information corresponding to the one color component can be specified based on the filtering coefficient vector corresponding to the one color component of the filtering coefficient and the second attribute parameter corresponding to the one color component.

[0069] Exemplarily, in the present application, taking the color information in the YUV space in the point cloud sequence as an example, it is assumed that the filter order indicated by the filter type is k. It is assumed that the point cloud sequence is n. The color attribute values of the points in the reconstructed point cloud and the k adjacent points of the points in the same color component (for example, the Y component) are represented by the matrix P(n,k). That is, P(n,k) is the reconstructed component value of the Y component of the color information of the points in the reconstructed point cloud and the second attribute parameter specified by the filter type.

[0070] Exemplarily, in the present application, The followingBased on the number 13, by using the filtering coefficient vector H(k) in the Y component for the corresponding second attribute parameter P(n,k), the filtering value R(n) of the attribute information in the Y component can be obtained.

[0071] Next, the U component and the V component are traversed by the above method, and the filtering value of the attribute information in the U component and the filtering value of the attribute information in the V component can be finally determined. Furthermore, by using the filtering values of the attribute information in all color components, a filtered point cloud corresponding to the reconstructed point cloud can be constructed.

[0072] Step 104: Update the reconstructed point cloud by using the filtered point cloud.

[0073] In the embodiments of the present application, after the decoder obtains a filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient, the decoder can further update the reconstructed point cloud by using the filtered point cloud.

[0074] Note that in the embodiments of the present application, compared with the reconstructed point cloud, the quality of the filtered point cloud obtained through the filtering process is significantly improved. Therefore, after obtaining the filtered point cloud, the original reconstructed point cloud can be updated by using the filtered point cloud. Specifically, the original reconstructed point cloud can be covered by using the filtered point cloud, and overall encoding, decoding, and quality improvement can be realized.

[0075] Furthermore, in the embodiments of the present application, FIG. 5 is a second flowchart showing the realization of point cloud decoding. As shown in FIG. 5, before obtaining a filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient, that is, before step 103, the point cloud decoding method executed by the decoder can further include the following steps.

[0076] Step 105: Identify the filtering index parameters corresponding to the initial point cloud by decrypting the bit stream.

[0077] Step 106: Identify the filtering coefficients when the filtering index parameters instruct to perform filtering processing on the reconstructed point cloud.

[0078] In the embodiments of the present application, the decoder can identify the filtering index parameters corresponding to the initial point cloud by decrypting the bit stream. When the filtering index parameters instruct to perform filtering processing on the reconstructed point cloud, the decoder can further decrypt to obtain the corresponding filtering coefficients.

[0079] In the embodiments of the present application, the filtering index parameters can be used to identify whether to perform filtering processing on the reconstructed point cloud. Specifically, the filtering flag information can further be used to identify the filter used for the filtering process.

[0080] Exemplarily, in the embodiments of the present application, when the value of the filtering index parameter is 0, it can be identified that no filtering process is performed on the reconstructed point cloud. When the value of the filtering index parameter is not 0 (for example, 1, 2, 3, etc.), it can be identified that a filtering process is performed on the reconstructed point cloud.

[0081] Furthermore, in the embodiments of the present application, when identifying a filter using a filtering index parameter, if the value of the filtering index parameter is 1, it can be determined that filtering processing is performed on the reconstructed point cloud using a one-dimensional filter. If the value of the filtering index parameter is 2, it is determined that filtering processing is performed on the reconstructed point cloud using a two-dimensional filter. The two-dimensional filter is either a rhombic filter or a rectangular filter. Or, if the value of the filtering index parameter is 2, it is determined that filtering processing is performed on the reconstructed point cloud using a multi-dimensional filter. That is, when the value of the filtering index parameter is 0, it indicates that no filtering processing is performed on the reconstructed point cloud. When the value of the filtering index parameter is not 0, different values indicate that filtering processing is performed on the reconstructed point cloud using different filters.

[0082] As can be seen from the above, in the embodiments of the present application, to determine whether to perform filtering processing on the reconstructed point cloud, a flag (for example, filtering flag information) may be used, or an index (for example, a filtering index parameter) may be used. In the present application, the specific form for determining whether to perform filtering processing is not limited.

[0083] Furthermore, in the embodiments of the present application, after obtaining filtering flag information corresponding to the initial point cloud by decrypting the bitstream, if the filtering flag information instructs not to perform filtering processing on the reconstructed point cloud, the process of specifying the filtering coefficient may not be executed, that is, the filtering coefficient is not decrypted. Or, after obtaining the filtering index parameter corresponding to the initial point cloud by decrypting the bitstream, if the value of the filtering index parameter instructs not to perform filtering processing on the reconstructed point cloud, the process of specifying the filtering coefficient may not be executed, that is, the filtering coefficient is not decrypted.

[0084] As can be understood, in the present application, when the filtering flag information instructs not to perform filtering processing on the reconstructed point cloud, or when the value of the filtering index parameter instructs not to perform filtering processing on the reconstructed point cloud, the decoder can skip the filtering processing process and perform point cloud reconstruction according to the original program, that is, the update process of the reconstructed point cloud is not executed.

[0085] Furthermore, in the embodiments of the present application, when determining whether to perform filtering processing on the reconstructed point cloud, an element of quantization parameter may be introduced, that is, based on the quantization parameter, the filtering flag information, and / or the filtering index parameter, it is possible to further specify whether to perform filtering processing.

[0086] Specifically, in the present application, first, the quantization parameter of the initial point cloud can be specified, and then, based on the quantization parameter, the filtering flag information, and / or the filtering index parameter, it is possible to determine whether to perform filtering processing on the reconstructed point cloud. The quantization parameter can represent the quantization step size.

[0087] As can be understood, in the present application, when determining whether to perform filtering processing on the reconstructed point group based on the quantization parameter, filtering flag information, and / or filtering index parameter, it is possible to specify whether to perform filtering processing based on each of the quantization parameter, filtering flag information, and / or filtering index parameter. If any of the quantization parameter, filtering flag information, and / or filtering index parameter instructs to perform filtering processing, it is determined that filtering processing is to be performed on the reconstructed point group, and the filtering parameter can be written into the bitstream. Or, if at least one of the quantization parameter, filtering flag information, and / or filtering index parameter instructs to perform filtering processing, it is determined that filtering processing is to be performed on the reconstructed point group, and the filtering parameter can be written into the bitstream.

[0088] Optionally, in the embodiments of the present application, the quantization parameter can be used not only to specify whether to perform filtering processing, but also to further specify for which color component of the color information filtering processing is to be performed.

[0089] Specifically, in the present application, when the quantization parameter is greater than the quantization threshold, it can be specified that filtering processing is to be performed on the first color component among all the color components of the points in the reconstructed point group. When the quantization parameter is less than or equal to the quantization threshold, it can be specified that filtering processing is to be performed on the second color component among all the color components of the points in the reconstructed point group. The first color component and the second color component are different.

[0090] Exemplarily, in the present application, when the quantization step size is large, only the Y component with large variability can be filtered, and when the quantization step size is small, only the UV components can be filtered.

[0091] Furthermore, in an embodiment of the present application, the decoder can further identify, by decoding the bitstream, information for indicating an optimal filter type and filtering coefficients corresponding to the optimal filter type. The optimal filter type can be one of at least one filter type supported by the decoding.

[0092] Accordingly, in an embodiment of the present application, on the decoding side, two or more filters with different orders can be provided. For example, two filters with orders k1 = 16 and k2 = 128 can be mentioned. Next, by decoding the bitstream, one filter corresponding to the optimal filter type used for the filtering process and the filtering coefficients corresponding to the optimal filter type are identified. Thereby, it becomes possible to perform a filtering process on the reconstructed point cloud using the filter and the filtering coefficients, and a higher filtering performance can be realized.

[0093] That is, Symbolization On one side, a plurality of different filters can be used to separately perform identification of the filtering coefficients and the filtering process. Finally, the filter with the best filtering effect can be selected, and the information corresponding to the filter and the corresponding filtering coefficients can be written into the bitstream. Thereby, on the decoding side, by decoding the bitstream, the information of the filter with the best filtering effect and the filtering coefficients can be directly obtained, and further, the filtering process can be performed using the filter, and the filtering performance and the encoding / decoding efficiency can be improved.

[0094] Furthermore, in the embodiments of the present application, since the output of the color transform is an integer and there is information loss during the transform, on the decoding side, instead of performing the final YUV-to-RGB conversion process, the point cloud sequence of YUV attributes can be directly output, improving the performance and effect of decoding.

[0095] As described above, according to the point cloud decoding method proposed from step 101 to step 106, when filtering the color components (for example, Y component, U component, V component) of the color reconstruction value of the point cloud sequence, the reconstructed point cloud output from the decoding side can be selectively improved in quality, with a small increase in the number of bits of the bitstream, and the compression performance can be improved.

[0096] Note that the point cloud decoding method according to the present application is applicable to any point cloud sequence, and in particular, it has a more prominent optimization effect on sequences with dense point distributions and low bitrates.

[0097] Furthermore, the point cloud decoding method according to the present application is operable for all of the decoding methods of three types of color attribute transforms (prediction transform, lifting transform, RAHT), and has versatility. Also, the filtering process does not affect the point cloud reconstruction process and has no other adverse effects.

[0098] Note that the filter according to the embodiments of the present application can be used within the prediction loop, that is, used as an in-loop filter and can be used as a reference for decoding subsequent points in the point cloud. Also, the filter according to the embodiments of the present application can be used outside the prediction loop, that is, used as a post filter and is not used as a reference for decoding subsequent points in the point cloud. It is not specifically limited in the present application.

[0099] In the present application, if the filter according to the embodiment of the present application is an in-loop filter, after determining that filtering processing is to be performed, it is necessary to write parameter information (for example, filtering flag information and / or filtering index parameter) for instructing to perform the filtering processing into the bitstream, and it is also necessary to write the filtering coefficient into the bitstream. Accordingly, after determining that the filtering processing is not to be performed, it is possible to select not to write the parameter information (for example, filtering flag information and / or filtering index parameter) for instructing to perform the filtering processing into the bitstream, and also not to write the filtering coefficient into the bitstream. Not performed In this case, it is possible to select not to write the parameter information (for example, filtering flag information and / or filtering index parameter) for instructing to perform the filtering processing into the bitstream, and also not to write the filtering coefficient into the bitstream.

[0100] In the present application, if the filter according to the embodiment of the present application is a post-processing filter, in one case, when the filtering coefficient corresponding to the filter is in one single auxiliary information data unit (for example, Supplemental enhancement information (SEI)), it is possible to select not to write the parameter information (for example, filtering flag information and / or filtering index parameter) for instructing to perform the filtering processing into the bitstream, and also not to write the filtering coefficient into the bitstream. Accordingly, when the decoder has not acquired the SEI, the filtering processing is not performed on the reconstructed point cloud. In another case, when the filtering coefficient corresponding to the filter and other information are in one auxiliary information data unit, after determining that the filtering processing is to be performed, it is necessary to write the parameter information (for example, filtering flag information and / or filtering index parameter) for instructing to perform the filtering processing into the bitstream, and it is also necessary to write the filtering coefficient into the bitstream. Accordingly, after determining that the filtering processing is not to be performed, the filtering processing is Not performed In this case, it is possible to select not to write the parameter information (for example, filtering flag information and / or filtering index parameter) for instructing to perform the filtering processing into the bitstream, and also not to write the filtering coefficient into the bitstream. Not performedParameter information (e.g., filtering flag information and / or filtering index parameter) that indicates this can be selected such that it is not written to the bit stream, and also the filtering coefficients are not written to the bit stream.

[0101] Furthermore, in the point cloud encoding / decoding method according to an embodiment of the present application, it can be selected to perform filtering processing on one or more parts in the reconstructed point cloud, that is, the filtering flag information and / or the filtering index parameter may be applied to the entire reconstructed point cloud or may be applied to a certain part in the reconstructed point cloud.

[0102] In an embodiment of the present application, a point cloud encoding / decoding method is disclosed. The decoder identifies filtering flag information corresponding to the initial point cloud by decoding the bit stream. The filtering flag information is used to identify whether to perform filtering processing on the reconstructed point cloud corresponding to the initial point cloud. When the filtering flag information indicates to perform filtering processing on the reconstructed point cloud, the decoder identifies the filtering coefficients. The decoder obtains the filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficients. The decoder updates the reconstructed point cloud by using the filtered point cloud. In other words, in the point cloud encoding / decoding method according to the present application, Symbolization On one side, the initial point cloud and the reconstructed point cloud are used to calculate the filtering coefficients for performing filtering processing, and after it is determined to perform filtering processing on the reconstructed point cloud, the filtering coefficients are transmitted to the decoding side. Accordingly, the decoding side can directly decode to obtain the filtering coefficients and perform filtering processing on the reconstructed point cloud by using the filtering coefficients. Thereby, the optimization of the reconstructed point cloud can be achieved, the quality of the point cloud can be improved, and the encoding / decoding efficiency can be greatly improved.

[0103] In one embodiment of the present application, a point cloud encoding method applied to a point cloud encoder is provided. FIG. 6 is a first flowchart showing the realization of point cloud encoding. As shown in FIG. 6, in the embodiment of the present application, the point cloud encoding method executed by the encoder may include the following steps.

[0104] Step 201: Identify the reconstruction value of the attribute information of the points in the initial point cloud, and construct a reconstructed point cloud corresponding to the initial point cloud based on the reconstruction value.

[0105] In the embodiment of the present application, first, the reconstruction value of the attribute information of the points in the initial point cloud can be identified, and then, a reconstructed point cloud corresponding to the initial point cloud can be constructed based on the reconstruction value of the attribute information of the points.

[0106] Note that in the embodiment of the present application, for one point in the initial point cloud, when encoding the one point, the one point can be set as a point waiting to be encoded in the initial point cloud, and there are a plurality of encoded points around the one point.

[0107] Furthermore, in the embodiment of the present application, one point in the initial point cloud corresponds to one piece of geometric information and one piece of attribute information. The geometric information represents the spatial position of the one point, and the attribute information represents the reconstruction attribute value of the one point.

[0108] Note that in the embodiment of the present application, the attribute information may include color information. Specifically, the attribute information may be color information in any color space. For example, the attribute information may be color information in the RGB space, color information in the YUV space, or color information in the YCbCr space, and is not specifically limited in the present application.

[0109] Furthermore, in the embodiments of the present application, when the attribute information is color information in the RGB space, the color components include the R component, the G component, and the B component. When the attribute information is color information in the YUV space, the color components include the Y component, the U component, and the V component. When the attribute information is color information in the YCbCr space, the color components include the Y component, the Cb component, and the Cr component.

[0110] As can be understood, in the embodiments of the present application, for one point in the initial point cloud, the attribute information of the one point may be an encoded and reconstructed attribute value. The attribute information may be color information, may be reflectivity or other attributes, and is not limited thereto in the present application.

[0111] Furthermore, in the embodiments of the present application, for one point in the initial point cloud, first, a residual value and a predicted value corresponding to the attribute information of the one point can be determined, and then, by calculating using the residual value and the predicted value, a reconstructed value of the attribute information of the one point can be obtained.

[0112] Specifically, in the embodiments of the present application, when determining the predicted value corresponding to the attribute information of one point in the initial point cloud, by using the geometric information and attribute information of a plurality of target adjacent points of the one point and the geometric information of the one point, the attribute information of the current point is predicted to obtain the corresponding predicted value.

[0113] As can be understood, in the embodiments of the present application, after determining the reconstructed value of the attribute information of one point in the initial point cloud, the one point can be used as the nearest neighbor point of the subsequent point in the LOD, and thereby, the subsequent point attribute prediction is continuously performed by using the reconstructed value of the attribute information of the one point.

[0114] Furthermore, in the embodiments of the present application, after identifying the reconstruction value of the attribute information of the points in the initial point cloud, it is possible to identify the corresponding reconstructed point cloud by using the reconstruction value of the attribute information of each point in the initial point cloud.

[0115] Note that in the present application, the initial point cloud can be directly obtained by an encoding / decoding program called a point cloud reading function. The reconstructed point cloud corresponding to the initial point cloud is obtained after attribute encoding, attribute reconstruction, and geometric offset.

[0116] Step 202: Identify the filtering coefficient based on the initial point cloud and the reconstructed point cloud.

[0117] In the embodiments of the present application, after identifying the reconstruction value of the attribute information of the points in the initial point cloud and constructing the reconstructed point cloud corresponding to the initial point cloud based on the reconstruction value, it is further possible to identify the filtering coefficient used in the filtering process based on the initial point cloud and the corresponding reconstructed point cloud. The filtering coefficient can be the filtering coefficient of the filtering process.

[0118] Note that in the embodiments of the present application, the filtering coefficient can be used for Wiener filtering, that is, the filtering coefficient is the coefficient of the Wiener filter.

[0119] Specifically, the Wiener filter is a linear filter with the least squares as the optimal criterion. Under certain constraint conditions, the square of the difference between its output and a certain predetermined function (usually called the desired output) is minimized, and it can finally be transformed into the calculation of a single Toeplitz formula through mathematical operations. The Wiener filter is also called the least squares filter or the least-squares filter.

[0120] Wiener filtering is a method of filtering a signal mixed with noise by utilizing the correlation characteristics and spectral characteristics of a stationary random process, and has conventionally been one of the basic filtering methods. The specific algorithm of Wiener filtering is as follows in Equation 1.

Equation

[0121] Converting Equation 1 into the form of a formula results in Equation 2 as follows.

Equation

[0122] If the desired signal is d(n), the error e(n) between the known signal and the desired signal can be calculated as follows.

Equation

[0123] Since the Wiener filter uses the minimum mean square error as the objective function, the objective function is set as Equation 4 below.

Equation

[0124] When the filtering coefficient is optimal, the derivative of the objective function with respect to the coefficient becomes 0. That is, it is as follows.

Equation

[0125] That is, it is as follows.

Equation

Equation

[0126] Furthermore, it can be expressed as in Equation 8.

Equation

[0127] Rxd is the correlation matrix of the input signal and the desired signal, and Rxx is the autocorrelation matrix of the input signal. The filtering coefficient H can be obtained as follows by the Wiener-Hough equation.

Equation

[0128] Furthermore, in the present application, if the attribute information is the color information of any space, for example, if the attribute information is the color information in the YUV space, when specifying the filtering coefficient based on the initial point cloud and the reconstructed point cloud, first, based on the initial component value and the reconstructed component value of the color component (for example, the Y component, the U component, the V component) and the filter type, the first attribute parameter and the second attribute parameter of the color component can be specified. Furthermore, based on the first attribute parameter and the second attribute parameter, the filtering coefficient for performing the filtering process can be further specified.

[0129] Note that in the embodiments of the present application, the first attribute parameter is used to specify the attribute value of the color component of the point in the initial point cloud, and the second attribute parameter is used to specify the attribute value of the color component of the point in the reconstructed point cloud. Specifically, the first attribute parameter represents the color attribute value of the point in the initial point cloud in one color component, and the second attribute parameter represents the color attribute value of the point and its k adjacent points in the reconstructed point cloud in one color component.

[0130] Furthermore, in the embodiments of the present application, the filter type can be used to indicate the filter order, and / or the filter shape, and / or the filter dimensionality. The filter shape includes a rhombus, a rectangle, etc., and the filter dimensionality includes one-dimensional and multi-dimensional.

[0131] That is, in the present application, different filter types may correspond to filters with different orders. For example, they may correspond to filters with orders k being 16 32, 128. Different filter types may also correspond to filters with different dimensionalities. For example, they may correspond to one-dimensional filters, two-dimensional filters, and multi-dimensional filters. Also, in the case of two-dimensional filters and multi-dimensional filters, different filter types may indicate the adoption of different filter shapes. For example, a rhombus filter, a rectangle filter, etc. may be adopted.

[0132] Specifically, in the embodiments of the present application, when specifying the filtering coefficient based on the initial point cloud and the reconstructed point cloud, first, based on the initial component attribute value of the color component of the color information of the points in the initial point cloud, the first attribute parameter of the color component is specified. At the same time, according to the reconstructed component value of the color component of the color information of the points in the reconstructed point cloud, the filter order, and the filter type, the second attribute parameter of the attribute information matrix corresponding to the color component is specified. Finally, the filtering coefficient can be specified based on the first attribute parameter and the second attribute parameter.

[0133] In the present application, the calculation of the corresponding first attribute parameter and second attribute parameter for one color component is performed, and the filtering coefficient vector corresponding to the one color component is specified using the first attribute parameter and the second attribute parameter. After traversing all the color components and obtaining the filtering coefficient vectors of each color component, the filtering coefficient can be finally specified based on the filtering coefficient vectors of each color component.

[0134] Specifically, in the embodiments of the present application, when determining the filtering coefficient based on the initial point cloud and the reconstructed point cloud, first, based on the first attribute parameter and the second attribute parameter corresponding to the color component, the cross-correlation parameter corresponding to the color component is determined. At the same time, based on the second attribute parameter, the autocorrelation parameter corresponding to the color component is determined. Next, based on the cross-correlation parameter and the autocorrelation parameter, the filtering coefficient vector corresponding to the color component is determined. Finally, all color components are traversed, and the filtering coefficient can be determined by using the filtering coefficient vectors corresponding to all color components.

[0135] Exemplarily, in the present application, taking the color information in the YUV space in the point cloud sequence as an example, assuming that the filter order indicated by the filter type is k, that is, the filtering coefficient is calculated using k adjacent points of one point in the point cloud. Assume that the point cloud sequence is n. The color attribute value of a point in the initial point cloud in one color component (for example, the Y component) is represented by the vector S(n). That is, S(n) is the first attribute parameter including the initial component value of the Y component of the color information of the points in the initial point cloud. The color attribute values of the points in the reconstructed point cloud and their k adjacent points in the same color component (for example, the Y component) are represented by the matrix P(n,k). That is, P(n,k) is the second attribute parameter specified by the reconstructed component value of the Y component of the color information of the points in the reconstructed point cloud and the filter type.

[0136] Furthermore, according to the following formula 10, the cross-correlation parameter B(k) can be calculated and obtained based on the first attribute parameter S(n) and the second attribute parameter P(n,k).

Formula

[0137] According to the following formula 11, the autocorrelation parameter A(k, k) can be calculated and obtained based on the second attribute parameter P(n, k).

Equation

[0138] According to the Wiener - Hopf equation, the filter coefficient in the Y component, that is, the filter coefficient vector H(k) of the k - th filter in the Y component is as follows.

Equation

[0139] Next, traverse the U component and the V component by the above method, and finally the filter coefficient vector in the U component and the filter coefficient vector in the V component can be determined. Furthermore, the filter coefficient can be obtained by using the filter coefficient vectors in all color components.

[0140] As can be seen from the above, in this application, when determining the filter coefficient by using the initial point cloud and the corresponding reconstructed point cloud, first, based on the color information of the points in the initial point cloud and the reconstructed point cloud, the first attribute parameter and the second attribute parameter corresponding to each color component can be determined respectively. Furthermore, the filter coefficient vector corresponding to each color component can be determined, and finally, the filter coefficient can be obtained based on the filter coefficient vectors of all color components.

[0141] Step 203: Use the filter coefficient to obtain the filtered point cloud corresponding to the reconstructed point cloud.

[0142] In an embodiment of the present application, after the coder identifies the filtering coefficient based on the initial point cloud and the reconstructed point cloud, it can further use the filtering coefficient to obtain a filtered point cloud corresponding to the reconstructed point cloud.

[0143] Furthermore, in an embodiment of the present application, when performing a filtering process on the reconstructed point cloud using the filtering coefficient, based on the filtering coefficient vector corresponding to the color component and the second attribute parameter, the filtering value of the attribute information corresponding to the color component is identified, and then, based on the filtering value of the attribute information corresponding to the color component, the filtered point cloud can be constructed.

[0144] As can be understood, in an embodiment of the present application, when performing a filtering process on the reconstructed point cloud using a filter, a noisy signal and a desired signal are required. In the point cloud encoding / decoding framework, the reconstructed point cloud can be used as the noisy signal, and the initial point cloud can be used as the desired signal. Therefore, both the initial point cloud and the reconstructed point cloud can be input into the filter, that is, the input of the filter is the initial point cloud and the reconstructed point cloud. After calculating the filtering coefficient of the filter, that is, the filtering coefficient, further, based on the filtering coefficient, the filtering process for the reconstructed point cloud is completed, and the corresponding filtered point cloud can be obtained.

[0145] Note that in the present application, since the filtering parameter is obtained based on the original point cloud and the reconstructed point cloud, by using the filtering coefficient for the reconstructed point cloud, the original point cloud can be restored to the maximum extent.

[0146] Specifically, in the present application, when filtering the reconstructed point cloud based on the filtering coefficient, for one color component in the color information, based on the filtering coefficient vector of the filtering coefficient corresponding to the one color component and the second attribute parameter corresponding to the one color component, the filtering value of the attribute information corresponding to the one color component can be specified.

[0147] Exemplarily, in the present application, by using the filtering coefficient vector H(k) in the Y component for the corresponding second attribute parameter P(n,k), the filtering value R(n) of the attribute information in the Y component can be obtained.

Number

[0148] Next, the U component and the V component are traversed by the above method, and the filtering value of the attribute information in the U component and the filtering value of the attribute information in the V component can be finally specified. Furthermore, using the filtering values of the attribute information in all color components, a filtered point cloud corresponding to the reconstructed point cloud can be constructed.

[0149] Step 204: Specify the filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud. The filtering flag information is used to specify whether to perform filtering processing on the reconstructed point cloud.

[0150] In an embodiment of the present application, after obtaining the filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient, the encoder can further specify the filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud.

[0151] In addition, in the embodiments of the present application, the filtering flag information can be used to identify whether to perform filtering processing on the reconstructed point cloud. Specifically, the filtering flag information can further be used to identify for which color component in the reconstructed point cloud the filtering processing is to be performed.

[0152] Furthermore, in the embodiments of the present application, when identifying the filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud, first, identify the first performance parameter of the color component of the point in the reconstructed point cloud and the second performance parameter of the color component of the point in the filtered point cloud, then, based on the first performance parameter and the second performance parameter, identify the identification information corresponding to the color component, and finally, based on the identification information corresponding to the color component, the filtering flag information can be identified.

[0153] As can be understood, in the embodiments of the present application, the first performance parameter and the second performance parameter can respectively represent the encoding / decoding performance of the reconstructed point cloud and the encoding / decoding performance of the filtered point cloud for the same color component.

[0154] Exemplarily, in the present application, the first performance parameter can be the peak signal-to-noise ratio (PSNR) of one color component of the point in the reconstructed point cloud, and the second performance parameter can be the PSNR of one color component of the point in the filtered point cloud.

[0155] Specifically, in the embodiments of the present application, taking the PSNR in the Y component as an example, when identifying the identification information corresponding to the color component based on the first performance parameter and the second performance parameter, if the PSNR1 corresponding to the reconstructed point cloud is greater than the PSNR2 corresponding to the filtered point cloud, that is, if the PSNR of the filtered Y component decreases, it is considered that the filtering effect is not good. Therefore, the value of the identification information corresponding to the Y component can be set to indicate not to perform filtering processing on the Y component. Correspondingly, if the PSNR1 corresponding to the reconstructed point cloud is smaller than the PSNR2 corresponding to the filtered point cloud, that is, if the PSNR of the filtered Y component improves, it is considered that the filtering effect is good. Therefore, the value of the identification information corresponding to the Y component can be set to indicate to perform filtering processing on the Y component.

[0156] Furthermore, in the embodiments of the present application, the U component and the V component can be traversed by the above method, and the identification information corresponding to the U component and the identification information corresponding to the V component can be finally identified. Furthermore, the filtering flag information can be identified by using the identification information corresponding to all color components.

[0157] As can be seen from the above, since the filtering flag information is identified based on the identification information corresponding to each color component, it can be used not only to identify whether to perform filtering processing on the reconstructed point cloud, but specifically, it can also be used to identify for which color component the filtering processing is to be performed.

[0158] Note that in the present application, when the value of the identification information is the first value, it indicates not to perform filtering processing on the color component. When the value of the identification information is the second value, it indicates to perform filtering processing on the color component.

[0159] Furthermore, in the present application, when the identification information of all color components is the first value, that is, when it is instructed not to filter each of all color components, it can be specified that the filtering flag information instructs not to perform a filtering process on the reconstructed point cloud. Accordingly, when the identification information of all color components is not the first value, that is, when it is instructed to filter at least one color component, it can be specified that the filtering flag information instructs to perform a filtering process on the reconstructed point cloud.

[0160] Exemplarily, in the present application, the first value is 0 and the second value is 1. Alternatively, the first value is set to false and the second value is set to true.

[0161] Step 205: When the filtering flag information instructs to perform a filtering process on the reconstructed point cloud, write the filtering flag information and the filtering coefficient into the bit stream.

[0162] In an embodiment of the present application, after the coder specifies the filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud, when the filtering flag information instructs to perform a filtering process on the reconstructed point cloud, the filtering coefficient can be written into the bit stream, and the filtering flag information can also be written into the bit stream.

[0163] As can be understood, in the present application, when the filtering flag information instructs to perform a filtering process on the reconstructed point cloud, it is necessary to write the filtering flag information for instructing whether to perform a filtering process on the reconstructed point cloud and the filtering parameter for performing a filtering process on the reconstructed point cloud into the bit stream and transmit them to the decoding side.

[0164] In addition, in the embodiments of the present application, the coder can further write the residual value of the attribute information of the points in the initial point cloud into the bitstream and transmit it to the decoding side.

[0165] Furthermore, in the embodiments of the present application, FIG. 7 is a second flowchart showing the realization of point cloud encoding. As shown in FIG. 7, after obtaining the filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient, that is, after step 203, the point cloud encoding method executed by the coder can further include the following steps.

[0166] Step 206: Identify the filtering index parameter corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud.

[0167] Step 207: When the value of the filtering index parameter instructs to perform filtering processing on the reconstructed point cloud, write the filtering index parameter and the filtering coefficient into the bitstream.

[0168] In the embodiments of the present application, after obtaining the filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient, the coder further identifies the filtering index parameter corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud. When the value of the filtering index parameter instructs to perform filtering processing on the reconstructed point cloud, the filtering coefficient can be written into the bitstream and transmitted to the decoding side, and the filtering index parameter can also be written into the bitstream.

[0169] In the embodiments of the present application, the filtering index parameter can be used to determine whether to perform filtering processing on the reconstructed point cloud. Specifically, the filtering flag information can further be used to identify the filter used for the filtering process.

[0170] As can be understood, in the embodiments of the present application, based on the reconstructed point cloud and the filtered point cloud, the value of the filtering index parameter can be set, whereby it is possible to determine whether to perform filtering processing on the reconstructed point cloud by using the filtering index parameter.

[0171] Exemplarily, in the embodiments of the present application, when the value of the filtering index parameter is 0, it can be determined that no filtering process is performed on the reconstructed point cloud. When the value of the filtering index parameter is not 0 (for example, 1, 2, 3, etc.), it can be determined that filtering processing is performed on the reconstructed point cloud.

[0172] Furthermore, in the embodiments of the present application, when identifying a filter using a filtering index parameter, if the value of the filtering index parameter is 1, it can be determined that filtering processing is performed on the reconstructed point cloud using a one-dimensional filter. If the value of the filtering index parameter is 2, it is determined that filtering processing is performed on the reconstructed point cloud using a two-dimensional filter. The two-dimensional filter is either a rhombus filter or a rectangular filter. Or, if the value of the filtering index parameter is 2, it is determined that filtering processing is performed on the reconstructed point cloud using a multi-dimensional filter. That is, when the value of the filtering index parameter is 0, it indicates that no filtering processing is performed on the reconstructed point cloud. When the value of the filtering index parameter is not 0, different values indicate that filtering processing is performed on the reconstructed point cloud using different filters.

[0173] As can be seen from the above, in the embodiments of the present application, to determine whether to perform filtering processing on the reconstructed point cloud, a flag (for example, filtering flag information) may be used, or an index (for example, a filtering index parameter) may be used. In the present application, the specific form for determining whether to perform filtering processing is not limited.

[0174] Furthermore, in the embodiments of the present application, after identifying the filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud, if the filtering flag information indicates that no filtering processing is performed on the reconstructed point cloud, the filtering coefficient may not be written into the bit stream. Or, after identifying the filtering index parameter corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud, if the value of the filtering index parameter indicates that no filtering processing is performed on the reconstructed point cloud, the filtering coefficient may not be written into the bit stream.

[0175] For better understanding, in this application, when the filtering flag information instructs not to perform filtering processing on the reconstructed point cloud, the filtering flag information may not be written into the bitstream. Or, when the value of the filtering index parameter instructs not to perform filtering processing on the reconstructed point cloud, the filtering index parameter may not be written into the bitstream.

[0176] Furthermore, in the embodiments of this application, when determining whether to perform filtering processing on the reconstructed point cloud, an element called quantization parameter may be introduced, that is, using the quantization parameter, the filtering flag information and / or the filtering index parameter, it is possible to further specify whether to perform filtering processing.

[0177] Specifically, in this application, first, the quantization parameter of the initial point cloud can be specified, and then, based on the quantization parameter, the filtering flag information and / or the filtering index parameter, it is possible to determine whether to perform filtering processing on the reconstructed point cloud. The quantization parameter can represent the quantization step size.

[0178] As can be understood, in the present application, when determining whether to perform filtering processing on the reconstructed point group based on the quantization parameter, filtering flag information, and / or filtering index parameter, it is possible to specify whether to perform filtering processing based on each of the quantization parameter, filtering flag information, and / or filtering index parameter. When any of the quantization parameter, filtering flag information, and / or filtering index parameter instructs to perform filtering processing, it is determined that filtering processing is to be performed on the reconstructed point group, and the filtering parameter can be written into the bit stream. Or, when at least one of the quantization parameter, filtering flag information, and / or filtering index parameter instructs to perform filtering processing, it is determined that filtering processing is to be performed on the reconstructed point group, and the filtering parameter can be written into the bit stream.

[0179] Optionally, in the embodiments of the present application, the quantization parameter can be used not only to specify whether to perform filtering processing, but also to further specify for which color component of the color information filtering processing is to be performed.

[0180] Specifically, in the present application, when the quantization parameter is greater than the quantization threshold, it can be specified that filtering processing is to be performed on the first color component among all the color components of the points in the reconstructed point group. When the quantization parameter is less than or equal to the quantization threshold, it can be specified that filtering processing is to be performed on the second color component among all the color components of the points in the reconstructed point group. The first color component and the second color component are different.

[0181] Exemplarily, in the present application, when the quantization step size is large, only the Y component with large variability can be filtered, and when the quantization step size is small, only the UV components can be filtered.

[0182] Furthermore, in an embodiment of the present application, on the encoding side, two or more filters with different orders can be provided. For example, two filters with orders k1 = 16 and k2 = 128 can be mentioned. Next, filtering processing is performed on the reconstruction point group with these two filters respectively. Finally, considering the overall quality improvement, operation time, and size of the number of bits, one filter with better performance is selected. The order and filtering coefficients corresponding to the one filter are written into the bit stream, and furthermore, it can be instructed to perform filtering processing using the one filter on the decoding side.

[0183] Specifically, in the present application, when specifying the filtering coefficients, at least one second attribute parameter corresponding to the color component is constructed based on the reconstructed component value of the color component of the point color information in the reconstruction point group and at least one filter type. That is, based on the reconstructed component values of the same color component, the second attribute parameters corresponding to each filter type are specified. Furthermore, in combination with the first attribute parameter corresponding to the color component, the filtering coefficient vector corresponding to each filter type for the color component can be specified. Also, after traversing all color components, the filtering coefficients corresponding to each filter type can be obtained, that is, at least one set of filtering coefficients corresponding to at least one filter type is specified. One filter type corresponds to one set of filtering coefficients.

[0184] Accordingly, in the embodiments of the present application, after identifying at least one set of filtering coefficients corresponding to at least one filter type, based on the at least one filter type, at least one set of filtering coefficients is used to perform filtering processing on the reconstructed point cloud respectively, that is, filtering processing can be performed on the reconstructed point cloud respectively by using different filters and the corresponding filtering coefficients. Thereby, at least one filtered point cloud corresponding to at least one filter type can be obtained. One filter type corresponds to one filtered point cloud.

[0185] Furthermore, in the present application, based on at least one filtered point cloud, an optimal filter type is selected from at least one filter type, and then, information for indicating the optimal filter type and the filtering coefficients corresponding to the optimal filter type can be written into the bit stream.

[0186] That is, on the encoding side, a plurality of different filters can be used to separately perform identification of filtering coefficients and filtering processing. Finally, the filter with the best filtering effect is selected, and the information corresponding to the filter and the corresponding filtering coefficients can be written into the bit stream. Thereby, on the decoding side, by decoding the bit stream, the information of the filter with the best filtering effect and the filtering coefficients can be directly obtained. Furthermore, filtering processing can be performed by using the filter, and the filtering performance and the encoding / decoding efficiency can be improved.

[0187] As described above, according to the point cloud encoding method proposed from step 201 to step 207, when filtering the color components (for example, Y component, U component, V component) of the color reconstruction value of the point cloud sequence, the reconstructed point cloud can be selectively improved in quality, the increase in the number of bits of the bit stream is small, and the compression performance can be improved.

[0188] Note that the point cloud encoding method according to the present application is applicable to any point cloud sequence. In particular, it has a more prominent optimization effect on sequences with dense point distributions and sequences with low bit rates.

[0189] Furthermore, the point cloud encoding method according to the present application is operable for all of the encoding methods of three types of color attribute conversions (prediction conversion, lifting conversion, RAHT), and has versatility. Also, the filtering process does not affect the point cloud reconstruction process and has no other adverse effects.

[0190] Note that the filter according to the embodiment of the present application can be used within the prediction loop, that is, used as an in-loop filter and can be used as a reference for encoding subsequent points in the point cloud. Also, the filter according to the embodiment of the present application can be used outside the prediction loop, that is, used as a post-filter and is not used as a reference for encoding subsequent points in the point cloud. Regarding this, it is not specifically limited in the present application.

[0191] Note that in the present application, if the filter according to the embodiment of the present application is an in-loop filter, after it is determined to perform the filtering process, it is necessary to write parameter information (for example, filtering flag information and / or filtering index parameter) indicating to perform the filtering process into the bitstream, and it is also necessary to write the filtering coefficient into the bitstream. Accordingly, after it is determined not to perform the filtering process, it is possible to select not to write the parameter information (for example, filtering flag information and / or filtering index parameter) indicating to perform the filtering process into the bitstream, and also not to write the filtering coefficient into the bitstream. Not performed Furthermore, after it is determined not to perform the filtering process, it is possible to select not to write the parameter information (for example, filtering flag information and / or filtering index parameter) indicating to perform the filtering process into the bitstream, and also not to write the filtering coefficient into the bitstream.

[0192] In the present application, if the filter according to the embodiment of the present application is a post-processing filter, in one case, when the filtering coefficient corresponding to the filter is in one single auxiliary information data unit (for example, supplementary enhancement information (SEI)), the filtering process is Not performed It is possible to select not to write the parameter information (for example, filtering flag information and / or filtering index parameter) indicating this to the bit stream, and also not to write the filtering coefficient to the bit stream. Accordingly, if the decoder has not acquired the SEI, the filtering process is not performed on the reconstructed point cloud. In another case, when the filtering coefficient corresponding to the filter and other information are in one auxiliary information data unit, after it is specified to perform the filtering process, it is necessary to write the parameter information (for example, filtering flag information and / or filtering index parameter) indicating to perform the filtering process to the bit stream, and it is also necessary to write the filtering coefficient to the bit stream. Accordingly, after it is specified not to perform the filtering process, the filtering process is Not performed It is possible to select not to write the parameter information (for example, filtering flag information and / or filtering index parameter) indicating this to the bit stream, and also not to write the filtering coefficient to the bit stream.

[0193] Furthermore, in the point cloud encoding / decoding method according to the embodiment of the present application, it is possible to select to perform the filtering process on one or more parts in the reconstructed point cloud, that is, the filtering flag information and / or the filtering index parameter may be applied to the entire reconstructed point cloud or may be applied to a certain part in the reconstructed point cloud.

[0194] In an embodiment of the present application, a point cloud encoding method is disclosed. The encoder identifies a reconstruction value of the attribute information of the points in the initial point cloud, and constructs a reconstructed point cloud corresponding to the initial point cloud based on the reconstruction value. The encoder identifies a filtering coefficient based on the initial point cloud and the reconstructed point cloud. The encoder uses the filtering coefficient to obtain a filtered point cloud corresponding to the reconstructed point cloud. The encoder identifies filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud. The filtering flag information is used to identify whether to perform a filtering process on the reconstructed point cloud. When the filtering flag information indicates that a filtering process is to be performed on the reconstructed point cloud, the encoder writes the filtering flag information and the filtering coefficient into the bit stream. In other words, in the point cloud encoding / decoding method according to the present application, Symbolization On one side, the initial point cloud and the reconstructed point cloud are used to calculate a filtering coefficient for performing a filtering process, and after it is determined that a filtering process is to be performed on the reconstructed point cloud, the filtering coefficient is transmitted to the decoding side. Accordingly, the decoding side can directly decode to obtain the filtering coefficient and use the filtering coefficient to perform a filtering process on the reconstructed point cloud. Thereby, the optimization of the reconstructed point cloud can be achieved, the quality of the point cloud can be improved, and the encoding / decoding efficiency can be greatly improved.

[0195] Based on the above embodiment, in another embodiment of the present application, when the attribute information is the color information in the YUV space and taking the filtering process using a Wiener filter as an example, FIG. 8 is a flowchart showing point cloud encoding / decoding. As shown in FIG. 8, on the encoding side, the encoder can first obtain the input of the Wiener filter, that is, the initial point cloud and the corresponding reconstructed point cloud. The initial point cloud can be directly obtained by an encoding / decoding program called a point cloud reading function. The reconstructed point cloud is obtained after attribute encoding, attribute reconstruction, and geometric offset based on the initial point cloud.

[0196] As can be understood, in the embodiments of the present application, during the encoding of the initial point cloud, the residual value of the attribute information of the points in the initial point cloud can be determined, and the encoder can write the residual value into the bitstream and transmit it to the decoding side.

[0197] Furthermore, after completing the point cloud reconstruction and obtaining the reconstructed point cloud, Wiener filtering based on the Wiener filter is started. On the encoding side, the main role of the Wiener filter is to calculate the optimal filtering coefficient, that is, to calculate the filtering coefficient, and at the same time complete the filtering process for the reconstructed point cloud, and also determine whether the quality of the point cloud has improved after Wiener filtering.

[0198] Note that in the present application, the filter order k (i.e., the number k of neighboring points selected during filtering) indicated by the filter type, and the number of color components that require filtering, etc., affect the quality of the filtered point cloud. Therefore, in order to balance performance and efficiency, it can be selected that the filter order k = 32.

[0199] Specifically, in the present application, the filtering coefficient of the Wiener filter can be calculated by applying the principle of the Wiener filtering algorithm. At the same time, on the encoding side, based on the obtained filtering coefficient, each of the three color components Y, U, and V can be filtered to obtain the filtering performance in the three color components Y, U, and V, and based on the filtered attribute values corresponding to the three color components Y, U, and V, the corresponding filtered point cloud can be constructed.

[0200] FIG. 9 is a flowchart showing the filtering process on the encoding side. As shown in FIG. 9, the initial point cloud and the reconstructed point cloud are input to the Wiener filter, and alignment of position information and conversion of attribute information are performed for the points in the initial point cloud and the points in the reconstructed point cloud respectively (for example, converting the color information in the attribute information from the RGB space to the YUV space). The filtering coefficient can be calculated using the color information of the points in the initial point cloud and the color information of the points in the reconstructed point cloud. The autocorrelation parameter can be calculated based on the color information of the points in the reconstructed point cloud, and the cross-correlation parameter can be calculated based on the color information of the points in the initial point cloud and the color information of the points in the reconstructed point cloud. Finally, the optimal filtering coefficient, that is, the corresponding filtering coefficient, can be specified using the autocorrelation parameter and the cross-correlation parameter.

[0201] Note that in the present application, on the encoding side, it is not necessary to cover the reconstructed point cloud using the filtered point cloud.

[0202] Furthermore, in the embodiment of the present application, the decoder can calculate the PSNR value (performance parameter) of each of the three color components Y, U, and V of the color information of the filtered point cloud and the reconstructed point cloud, and thereby specify whether to perform filtering processing on the reconstructed point cloud based on the PSNR values of all the color components, and can specify the filtering flag information.

[0203] Optionally, in the present application, when the PSNR value of one or more color components is improved, for example, when the PSNR values of two color components U and V are improved but the PSNR value of the Y component is decreased, the decoder can specify that no filtering processing is performed on the Y component and filtering processing is only performed on the U and V components, that is, the filtering flag information is provided to indicate that filtering processing is performed on the U and V components.

[0204] Optionally, in this application, when the PSNR values of the three color components Y, U, and V all decrease, the encoder can determine not to perform filtering processing on the three color components Y, U, and V, that is, the filtering flag information is provided to instruct not to perform filtering processing on the reconstructed point cloud. Correspondingly, when the PSNR values of the three color components Y, U, and V all improve, the encoder can determine to perform filtering processing on the three color components Y, U, and V, that is, the filtering flag information is provided to instruct to perform filtering processing on the reconstructed point cloud.

[0205] In addition, in the embodiments of this application, the filtering flag information can be a specific array (determination array) with a size of 1×3, which is used to determine whether to perform filtering processing on the reconstructed point cloud, and can also be used to identify the color components that need to perform filtering processing on the decoding side. The filtering flag information can include three pieces of identification information corresponding to the three color components Y, U, and V. A value of 0 for the identification information indicates that no filtering processing is performed on the corresponding color component, and a value of 1 for the identification information can indicate that filtering processing is performed on the corresponding color component.

[0206] As can be understood, in this application, when all three pieces of identification information corresponding to the three color components are 0, the filtering flag information instructs not to perform filtering processing on the reconstructed point cloud. When it is not the case that all three pieces of identification information corresponding to the three color components are 0, the filtering flag information instructs to perform filtering processing on the reconstructed point cloud.

[0207] For example, the filtering flag information can be represented as (0, 1, 1). The value of the identification information corresponding to the Y component is 0, and the value of the identification information corresponding to the UV component is 1. That is, the filtering flag information instructs to perform filtering processing on the reconstructed point cloud and also instructs to perform filtering processing only on the UV component.

[0208] Furthermore, in this application, when the PSNR values of the three color components Y, U, and V are not improved, it is considered that the filtering effect is not good and the quality has deteriorated. Therefore, Encoder it is not necessary to write the filtering coefficient into the bitstream, and at the same time, it is not necessary to write the filtering flag information into the bitstream, thereby ensuring that the Wiener filtering process is not performed on the decoding side either.

[0209] On the decoding side, after the decoder decodes the residual value of the attribute information, it can identify the filtering flag information by decoding the bitstream. For example, it can obtain a 1×3 specific array by decoding.

[0210] As can be understood, in this application, when the filtering flag information instructs to perform filtering processing on the reconstructed point cloud, the quality of the reconstructed point cloud can be improved by Wiener filtering, and it can be considered that there is transmission of the filtering coefficient on the encoding side. The decoder can continue to decode to obtain the corresponding filtering coefficient. When the filtering flag information instructs not to perform filtering processing on the reconstructed point cloud, the quality of the reconstructed point cloud cannot be improved by Wiener filtering, and it can be considered that there is no transmission of the filtering coefficient on the encoding side. The decoder does not continue to decode the filtering coefficient, skips the Wiener filtering, and performs point cloud reconstruction according to the original program.

[0211] Furthermore, in the embodiments of the present application, after decrypting the filtering coefficient, the decryptor can transmit the filtering coefficient, and after the point cloud reconstruction is completed, perform filtering processing on the reconstructed point cloud by using the filtering coefficient.

[0212] FIG. 10 is a flowchart showing the filtering process on the decryption side. As shown in FIG. 10, during the Wiener filtering process, the decryptor uses the reconstructed point cloud and the filtering coefficient as the input to the Wiener filter on the decryption side. After completing the conversion of the attribute information of the points in the reconstructed point cloud (for example, converting the color information in the attribute information from the RGB space to the YUV space), perform filtering processing by using the filtering coefficient, and obtain a filtered point cloud with improved quality that has been filtered by calculation. After completing the conversion of the attribute information of the points in the filtered point cloud (conversion from the YUV space to the RGB space), output the filtered point cloud.

[0213] As can be understood, in the present application, after obtaining the filtered point cloud, the original reconstructed point cloud can be covered by using the filtered point cloud, and the overall encoding, decryption, and quality improvement can be realized.

[0214] Furthermore, in the embodiments of the present application, in the process of determining whether to perform filtering processing on the reconstructed point cloud, a specific element called the quantization step size can be further added. For example, when the quantization step size is large, only the Y component with large variability is filtered, and when the quantization step size is small, only the UV components are filtered.

[0215] In the embodiments of the present application, on the encoding side, two or more different filters (such as different orders and different shapes) can be provided. For example, two filters with orders k1 = 16 and k2 = 128 can be mentioned. The encoder uses each of these two filters to filter the reconstructed point cloud. Considering the overall quality improvement, running time, and bit number size, one filter with better performance is selected. Next, information for instructing the filter and corresponding filtering parameters are written into the bitstream and transmitted to the decoding side. Accordingly, on the decoding side, different corresponding filters are also provided. The decoder obtains information for instructing the filter and corresponding filtering parameters by decoding the bitstream. Thereby, a filter with better performance can be selected and filtering can be performed on the decoding side.

[0216] Furthermore, in the embodiments of the present application, since the output of the color attribute conversion is an integer and there is information loss during the conversion, on the decoding side, the conversion from YUV to RGB is not performed finally, and the point cloud sequence with YUV attributes is directly output.

[0217] In addition, in the embodiments of the present application, FIG. 11 is a schematic diagram showing the test results under CY test conditions. FIG. 12 is a schematic diagram showing the test results under C1 test conditions. As shown in FIGS. 11 and 12, after the point cloud encoding / decoding method according to the embodiments of the present application was realized on the G-PCC reference software TMC13 V12.0, some test sequences (cat1-A&cat1-B) required by MPEG (Moving Picture Expert Group) were tested under the CTC CY and C1 test conditions. The CY condition is an encoding method of lossless geometry and lossy attribute, and the C1 condition is an encoding method of lossless geometry and nearly lossless attribute. End-to-End BD-AttrRate indicates the BD-Rate of the end-to-end attribute value, and BD-Rate reflects the difference in the psnr curves in two cases (with or without filtering). When the BD-Rate decreases, it indicates that when the psnr is equal, the bit rate decreases and the performance improves, and conversely, the performance decreases. That is, the lower the BD-Rate, the better the compression effect. Cat1-A average and Cat1-B average respectively represent the average values of the test effects of each point cloud sequence of the two data sets. Luma is the Y component, Chroma Cb is the U component, Chroma Cr is the V component, and Overall average is the average value of the test effects of all sequences.

[0218] As can be seen from the above, the point cloud encoding and decoding method according to the embodiment of the present application includes a technique of performing Wiener filtering on the YUV components of the color reconstruction values in the point cloud sequence, and can selectively improve the quality of the reconstructed point cloud output from the decoding side. In particular, for sequences with dense point distributions and low bitrates, a more prominent optimization effect is achieved. Moreover, the compression performance can be improved without significantly increasing the number of bits in the bitstream. At the same time, the technique is operable for all of the encoding methods of three types of color attribute conversions (prediction conversion, lifting conversion, RAHT), and has versatility. In addition, the technique does not affect the reconstruction process on the decoding side and has no other adverse effects.

[0219] That is, for general technical problems, in the embodiment of the present application, a technique is proposed to perform Wiener filtering processing on the YUV components of the color reconstruction values of encoding and decoding to improve the quality of the point cloud. One (or more) Wiener filters are provided on each of the encoding and decoding sides. The encoding side obtains the Wiener filtering coefficient by calculation, and the decoding side reads the filtering coefficient, whereby post-processing can be performed on the reconstruction value of the point cloud, and a more excellent optimization effect is achieved.

[0220] In the embodiment of the present application, a point cloud decoding method is provided. Symbolization On one side, using the initial point cloud and the reconstructed point cloud, calculate the filtering coefficient for performing filtering processing, and after specifying to perform filtering processing on the reconstructed point cloud, transmit the filtering coefficient to the decoding side. Accordingly, the decoding side can directly decode to obtain the filtering coefficient, and use the filtering coefficient to perform filtering processing on the reconstructed point cloud. Thereby, the optimization of the reconstructed point cloud can be achieved, the quality of the point cloud can be improved, and the encoding and decoding efficiency can be greatly improved.

[0221] Based on the above embodiments, in another embodiment of the present application, FIG. 13 is a first schematic diagram showing the structure of an encoder. As shown in FIG. 13, the encoder 300 according to the embodiment of the present application can include a first specifying unit 301, a constructing unit 302, and an encoding unit 303. The first specifying unit 301 is configured to specify a reconstruction value of the attribute information of the points in the initial point cloud. The constructing unit 302 is configured to construct a reconstructed point cloud corresponding to the initial point cloud based on the reconstruction value. The first specifying unit 301 is further configured to specify a filtering coefficient based on the initial point cloud and the reconstructed point cloud, use the filtering coefficient to obtain a filtered point cloud corresponding to the reconstructed point cloud, and specify filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud. The filtering flag information is used to specify whether to perform a filtering process on the reconstructed point cloud. When the filtering flag information instructs to perform a filtering process on the reconstructed point cloud, the encoding unit 303 is configured to write the filtering flag information and the filtering coefficient into the bitstream.

[0222] FIG. 14 is a second schematic diagram showing the structure of an encoder. As shown in FIG. 14, the encoder 300 according to the embodiment of the present application can further include a first processor 304, a first memory 305, a first communication interface 306, and a first bus 307. Instructions executable by the first processor 304 are stored in the first memory 305, and the first bus 307 is used to connect the first processor 304, the first memory 305, and the first communication interface 306.

[0223] Furthermore, in an embodiment of the present application, the first processor 304 identifies a reconstructed value of the attribute information of the points in the initial point cloud, constructs a reconstructed point cloud corresponding to the initial point cloud based on the reconstructed value, identifies a filtering coefficient based on the initial point cloud and the reconstructed point cloud, uses the filtering coefficient to obtain a filtered point cloud corresponding to the reconstructed point cloud, and is configured to identify filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud. The filtering flag information is used to identify whether to perform a filtering process on the reconstructed point cloud. The first processor 304 is further configured to write the filtering flag information and the filtering coefficient into a bit stream when the filtering flag information instructs to perform a filtering process on the reconstructed point cloud.

[0224] FIG. 15 is a first schematic diagram showing the structure of a decoder. As shown in FIG. 15, a decoder 400 according to an embodiment of the present application may include a decoding unit 401, a second identifying unit 402, and an updating unit 403. The decoding unit 401 is configured to decode a bit stream. The second identifying unit 402 is configured to identify filtering flag information corresponding to the initial point cloud. The filtering flag information is used to identify whether to perform a filtering process on the reconstructed point cloud corresponding to the initial point cloud. The second identifying unit 402 is further configured to identify a filtering coefficient and use the filtering coefficient to obtain a filtered point cloud corresponding to the reconstructed point cloud when the filtering flag information instructs to perform a filtering process on the reconstructed point cloud. The updating unit 403 is configured to update the reconstructed point cloud by using the filtered point cloud.

[0225] FIG. 16 is a second schematic diagram showing the structure of the decoder. As shown in FIG. 16, the decoder 400 according to the embodiment of the present application may further include a second processor 404, a second memory 405, a second communication interface 406, and a second bus 407. Instructions executable by the second processor 404 are stored in the second memory 405, and the second bus 407 is used to connect the second processor 404, the second memory 405, and the second communication interface 406.

[0226] Furthermore, in the embodiment of the present application, the second processor 404 is configured to identify filtering flag information corresponding to the initial point cloud by decoding the bitstream. The filtering flag information is used to identify whether to perform filtering processing on the reconstructed point cloud corresponding to the initial point cloud. When the filtering flag information instructs to perform filtering processing on the reconstructed point cloud, the second processor 404 further identifies a filtering coefficient, obtains a filtered point cloud corresponding to the reconstructed point cloud using the filtering coefficient, and is configured to update the reconstructed point cloud using the filtered point cloud.

[0227] When the integrated unit is not sold or used as an independent product but implemented as a software functional module, it may be stored in a computer-readable recording medium. According to this understanding, for the technical solution of this application, the essential part, or the part that can contribute to the prior art, or all or part of the technical solution can be expressed as a software product. This computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method of this embodiment. The storage medium includes various types of media capable of storing program codes, such as a universal serial bus (USB) flash drive, a mobile hard disk, a read only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0228] In an embodiment of the present application, an encoder and a decoder are provided. The decoder identifies filtering flag information corresponding to an initial point cloud by decoding a bitstream. The filtering flag information is used to identify whether to perform filtering processing on a reconstructed point cloud corresponding to the initial point cloud. When the filtering flag information indicates that filtering processing is to be performed on the reconstructed point cloud, the decoder identifies a filtering coefficient. The decoder obtains a filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient. The decoder updates the reconstructed point cloud by using the filtered point cloud. The encoder identifies a reconstruction value of the attribute information of the points in the initial point cloud and constructs a reconstructed point cloud corresponding to the initial point cloud based on the reconstruction value. The encoder identifies a filtering coefficient based on the initial point cloud and the reconstructed point cloud. The encoder obtains a filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient. The encoder identifies filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud. The filtering flag information is used to identify whether to perform filtering processing on the reconstructed point cloud. When the filtering flag information indicates that filtering processing is to be performed on the reconstructed point cloud, the encoder writes the filtering flag information and the filtering coefficient into the bitstream. In other words, in the point cloud encoding / decoding method according to the present application, Symbolization On one side, the initial point cloud and the reconstructed point cloud are used to calculate a filtering coefficient for performing filtering processing. After it is determined that filtering processing is to be performed on the reconstructed point cloud, the filtering coefficient is transmitted to the decoding side. Accordingly, the decoding side can directly decode to obtain the filtering coefficient and perform filtering processing on the reconstructed point cloud by using the filtering coefficient. Thereby, the optimization of the reconstructed point cloud can be achieved, the quality of the point cloud can be improved, and the encoding / decoding efficiency can be greatly improved.

[0229] In an embodiment of the present application, a computer-readable storage medium is provided. A program is stored in the computer-readable storage medium, and when the program is executed by a processor, the method described in the above embodiment is realized.

[0230] Specifically, program instructions corresponding to the point cloud encoding method in this embodiment can be stored in a storage medium such as an optical disk, a hard disk, or a USB flash drive. When the program instructions corresponding to the point cloud encoding method in the storage medium are read or executed by an electronic device, the following steps are included. Specify the reconstruction value of the attribute information of the points in the initial point cloud, and construct a reconstructed point cloud corresponding to the initial point cloud based on the reconstruction value. Specify a filtering coefficient based on the initial point cloud and the reconstructed point cloud. Use the filtering coefficient to obtain a filtered point cloud corresponding to the reconstructed point cloud. Specify filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud. The filtering flag information is used to specify whether to perform a filtering process on the reconstructed point cloud. When the filtering flag information instructs to perform a filtering process on the reconstructed point cloud, write the filtering flag information and the filtering coefficient into the bit stream.

[0231] Specifically, program instructions corresponding to the point cloud decoding method in this embodiment can be stored in a storage medium such as an optical disk, a hard disk, or a USB flash drive. When the program instructions corresponding to the point cloud decoding method in the storage medium are read or executed by an electronic device, the following steps are included. By decoding the bit stream, specify the filtering flag information corresponding to the initial point cloud. The filtering flag information is used to specify whether to perform a filtering process on the reconstructed point cloud corresponding to the initial point cloud. When the filtering flag information instructs to perform filtering processing on the reconstructed point cloud, the filtering coefficient is specified. Using the filtering coefficient, a filtered point cloud corresponding to the reconstructed point cloud is obtained. The reconstructed point cloud is updated using the filtered point cloud.

[0232] It should be understood by those skilled in the art that the embodiments of the present application can provide a method, a system, or a computer processor product. Therefore, the present application can have hardware embodiments, software embodiments, or embodiments combining software and hardware. Further, the present application can be implemented in the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to magnetic disk storage devices, optical memories, etc.) containing computer-usable program code.

[0233] The present application will be described with reference to the flowcharts and / or block diagrams of the method, apparatus (system), and computer program product according to the embodiments of the present application. Each process and / or block in the flowchart and / or block diagram, and combinations of processes and / or blocks in the flowchart and / or block diagram can be realized by computer program instructions. These computer program instructions are provided to the processor of a general-purpose computer, a dedicated computer, an embedded processor, or other programmable data processing device to generate a machine. Thus, the instructions executed by the processor of the computer or other programmable data processing device result in an apparatus for realizing the functions specified in one or more processes of the flowchart and / or one or more blocks of the block diagram.

[0234] Those computer program instructions can be stored in a computer-readable storage medium that can direct a computer or other programmable data processing apparatus to execute in a specific manner. Thereby, the instructions stored in the computer-readable storage medium result in a manufactured article that includes an instruction device. The instruction device realizes the functions specified in one or more processes of a flowchart and / or one or more blocks of a block diagram.

[0235] Those computer program instructions can be loaded into a computer or other programmable data processing apparatus and execute a series of operation steps on the computer or other programmable apparatus to generate a process implemented by the computer. Thus, the instructions executed on the computer or other programmable apparatus provide the steps used to realize the functions specified in one or more processes of a flowchart and / or one or more blocks of a block diagram.

[0236] The above is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Industrial Applicability

[0237] In an embodiment of the present application, a point cloud encoding / decoding method, an encoder, a decoder, and a computer storage medium are disclosed. The decoder identifies filtering flag information corresponding to an initial point cloud by decoding a bitstream. The filtering flag information is used to identify whether to perform filtering processing on a reconstructed point cloud corresponding to the initial point cloud. When the filtering flag information indicates to perform filtering processing on the reconstructed point cloud, the decoder identifies a filtering coefficient. The decoder obtains a filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient. The decoder updates the reconstructed point cloud by using the filtered point cloud. The encoder identifies a reconstruction value of attribute information of points in the initial point cloud and constructs a reconstructed point cloud corresponding to the initial point cloud based on the reconstruction value. The encoder identifies a filtering coefficient based on the initial point cloud and the reconstructed point cloud. The encoder obtains a filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient. The encoder identifies filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud. The filtering flag information is used to identify whether to perform filtering processing on the reconstructed point cloud. When the filtering flag information indicates to perform filtering processing on the reconstructed point cloud, the encoder writes the filtering flag information and the filtering coefficient into the bitstream. In other words, in the point cloud encoding / decoding method according to the present application, Symbolization On one side, an initial point cloud and a reconstructed point cloud are used to calculate a filtering coefficient for performing filtering processing. After it is determined to perform filtering processing on the reconstructed point cloud, the filtering coefficient is transmitted to the decoding side. Accordingly, the decoding side can directly decode to obtain the filtering coefficient and perform filtering processing on the reconstructed point cloud by using the filtering coefficient. Thereby, the optimization of the reconstructed point cloud can be achieved, the quality of the point cloud can be improved, and the encoding / decoding efficiency can be greatly improved.

Claims

1. A point cloud decoding method applied to a decoder, the point cloud decoding method comprising: identifying filtering flag information corresponding to an initial point cloud by decoding a bit stream, the filtering flag information being used to identify whether to perform filtering processing on a reconstructed point cloud corresponding to the initial point cloud; identifying a filtering coefficient when the filtering flag information instructs to perform filtering processing on the reconstructed point cloud; obtaining a filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient; updating the reconstructed point cloud by using the filtered point cloud. A point cloud decoding method characterized by the above.

2. Before obtaining the filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient, the point cloud decoding method further comprises: identifying a filtering index parameter corresponding to the initial point cloud by decoding the bit stream; identifying the filtering coefficient when the filtering index parameter instructs to perform filtering processing on the reconstructed point cloud. The point cloud decoding method according to claim 1, characterized by the above.

3. When the value of the filtering index parameter is 0, it is determined that no filtering processing is performed on the reconstructed point cloud; When the value of the filtering index parameter is not 0, it is determined that filtering processing is performed on the reconstructed point cloud. The point cloud decoding method according to claim 2, characterized by the above.

4. When the value of the filtering index parameter is 1, it is determined that filtering processing is performed on the reconstructed point cloud by using a one-dimensional filter; When the value of the filtering index parameter is 2, it is determined that filtering processing is performed on the reconstructed point cloud by using a two-dimensional filter; The two-dimensional filter is any one of a rhombus filter and a rectangular filter. The point cloud decoding method according to claim 3, characterized by the above.

5. The point cloud decoding method further comprises: Identifying a reconstructed value of the attribute information of the points in the initial point cloud, and constructing the reconstructed point cloud corresponding to the initial point cloud based on the reconstructed value, further comprising, When the attribute information is color information in the RGB space, the color components include an R component, a G component, and a B component, When the attribute information is color information in the YUV space, the color components include a Y component, a U component, and a V component, When the attribute information is color information in the YCbCr space, the color components include a Y component, a Cb component, and a Cr component, The method for decrypting a point cloud according to claim 1 or claim 2, characterized in that.

6. The filtering coefficient includes a filtering coefficient vector corresponding to the color component, Obtaining the filtered point cloud corresponding to the reconstructed point cloud by using the filtering coefficient includes: Identifying a second attribute parameter corresponding to the color component according to the reconstructed component value of the color component of the color information of the points in the reconstructed point cloud and the filter type; Identifying a filtering value of the attribute information corresponding to the color component based on the filtering coefficient vector corresponding to the color component and the second attribute parameter; Constructing the filtered point cloud based on the filtering value, including. The method for decrypting a point cloud according to claim 5, characterized in that.

7. The filtering flag information includes identification information of the color component, When the value of the identification information of the color component is a first value, it instructs not to perform filtering processing on the color component, When the value of the identification information of the color component is a second value, it instructs to perform filtering processing on the color component, The first value is 0, and the second value is 1, or The first value is set to false, and the second value is set to true, The method for decrypting a point cloud according to claim 1, characterized in that.

8. The method for decrypting a point cloud includes: When the identification information of all the color components is the first value, it is specified that the filtering flag information instructs not to perform filtering processing on the reconstructed point cloud, When it is not the case that the identification information of all the color components is the first value, further including identifying that the filtering flag information instructs to perform filtering processing on the reconstructed point cloud. The point cloud decoding method according to claim 7, characterized in that.

9. When the filtering flag information instructs not to perform filtering processing on the reconstructed point cloud, the process of specifying the filtering coefficient is not executed, or When the value of the filtering index parameter instructs not to perform filtering processing on the reconstructed point cloud, the process of specifying the filtering coefficient is not executed. The point cloud decoding method according to claim 2, characterized in that.

10. The point cloud decoding method includes specifying the quantization parameter of the initial point cloud, and further including determining whether to perform filtering processing on the reconstructed point cloud based on the quantization parameter, the filtering flag information, and / or the filtering index parameter. The point cloud decoding method according to claim 2, characterized in that.

11. The point cloud decoding method includes further including, by decoding the bitstream, specifying information for instructing an optimal filter type and a filtering coefficient corresponding to the optimal filter type, wherein the optimal filter type is one of at least one filter type, and the filter type is used to indicate a filter order, and / or a filter shape, and / or a filter dimensionality. The point cloud decoding method according to claim 6, characterized in that.

12. A point cloud encoding method applied to an encoder, the point cloud encoding method including specifying a reconstruction value of the attribute information of the points in the initial point cloud and constructing a reconstructed point cloud corresponding to the initial point cloud based on the reconstruction value, specifying a filtering coefficient based on the initial point cloud and the reconstructed point cloud, and acquiring a filtered point cloud corresponding to the reconstructed point cloud using the filtering coefficient. Specifying filtering flag information corresponding to the initial point cloud based on the reconstructed point cloud and the filtered point cloud, where the filtering flag information is used to specify whether to perform filtering processing on the reconstructed point cloud, and the specifying; When the filtering flag information instructs to perform filtering processing on the reconstructed point cloud, writing the filtering flag information and the filtering coefficient into a bit stream; A point cloud encoding method characterized by the above.

13. An encoder, comprising: A first processor and a first memory storing instructions executable by the first processor; When the instructions are executed, the first processor executes the point cloud encoding method according to Claim 12; An encoder characterized by the above.

14. A decoder, comprising: A second processor and a second memory storing instructions executable by the second processor; When the instructions are executed, the second processor executes the point cloud decoding method according to any one of Claims 1 to 11; A decoder characterized by the above.

15. A computer storage medium, comprising: A computer program is stored in the computer storage medium; When the computer program is executed by a first processor, it executes the point cloud encoding method according to Claim 12, or when the computer program is executed by a second processor, it executes the point cloud decoding method according to any one of Claims 1 to 11; A computer storage medium characterized by the above.

Images