Color component prediction method, encoder, decoder and computer storage medium
By constructing the prediction model in the point cloud compression encoder and removing the correlation between color components, the coding efficiency problem caused by color component correlation before RAHT transformation is solved, and more efficient coding and decoding is achieved.
Patent Information
- Application Number
- CN201980097731.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-09-30
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2039-09-30
AI Technical Summary
In a geometry-based point cloud compression encoder framework, the correlation between color components before RAHT transformation results in an increase in the number of coded bits, reducing encoding efficiency.
By constructing a prediction model, one color component is used to predict another color component, the residual is removed, and the prediction residual is used in the RAHT transformation to reduce the encoding bit overhead.
It effectively reduces the encoding bit overhead, improves the encoding efficiency, and reduces the bit rate during the encoding and decoding process.
Smart Images

Figure CN114009014B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of video coding and decoding technology, and in particular to a color component prediction method, an encoder, a decoder, and a computer storage medium. Background Art
[0002] In the geometry-based point cloud compression (G-PCC) encoder framework, the geometric information of the point cloud and the attribute information corresponding to each point cloud are encoded separately. After the geometry encoding is completed, the geometry information is reconstructed, and the encoding of the attribute information will depend on the reconstructed geometry information.
[0003] At present, attribute information encoding is mainly aimed at the encoding of color information. First, the color information is converted from the RGB color space to the YUV color space. Then, the point cloud is recolored using the reconstructed geometric information so that the unencoded attribute information corresponds to the reconstructed geometric information. In color information encoding, there are two main transformation methods, one is the distance-based lifting transformation that relies on the level of detail (LOD) division, and the other is the direct region adaptive hierarchical transformation (RAHT). Both methods convert color information from the spatial domain to the frequency domain, obtain high-frequency coefficients and low-frequency coefficients through transformation, and finally quantize and encode the coefficients to generate a binary code stream.
[0004] However, before the RAHT transformation, since there is a certain correlation between the three color components (which can be represented by Y, U, and V respectively), the number of encoding bits will increase when the RAHT transformation is performed, resulting in poor encoding efficiency. Summary of the invention
[0005] The embodiments of the present application provide a color component prediction method, an encoder, a decoder and a computer storage medium, which can remove the correlation between the three color components in the RAHT transform, effectively reduce the coding bit overhead, and thus improve the coding and decoding efficiency.
[0006] The technical solution of the embodiment of the present application can be implemented as follows:
[0007] In a first aspect, an embodiment of the present application provides a color component prediction method, which is applied to an encoder, and the method includes:
[0008] Determine the spatial block where the to-be-coded point is located, and the first color component value and the second color component value of the coded point;
[0009] Constructing a prediction model according to the spatial block; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be encoded;
[0010] Using the prediction model and the first color component value, a second color component prediction value of the point to be encoded is obtained;
[0011] The difference between the second color component value and the predicted value of the second color component is calculated, the obtained difference is used as the residual of the point to be encoded, and the residual of the point to be encoded is subjected to a regional adaptive hierarchical transform (RAHT) transformation.
[0012] In a second aspect, an embodiment of the present application provides a color component prediction method, which is applied to a decoder, and the method includes:
[0013] Determine the spatial block where the point to be decoded is located;
[0014] Parsing the attribute bit stream to obtain a first color component value of the point to be decoded, an initial residual, and a first model parameter and a second model parameter corresponding to the spatial block;
[0015] Constructing a prediction model according to the first model parameter and the second model parameter; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be decoded;
[0016] Using the prediction model and the first color component value, a second color component prediction value of the point to be decoded is obtained;
[0017] Perform RAHT inverse transform on the resolved initial residual to obtain the residual of the point to be decoded;
[0018] A reconstructed value of the second color component of the point to be decoded is calculated according to the residual of the point to be decoded and the predicted value of the second color component.
[0019] In a third aspect, an embodiment of the present application provides an encoder, the encoder comprising a first determining unit, a first constructing unit, a first predicting unit, a first calculating unit and a transforming unit, wherein:
[0020] A first determining unit is configured to determine a spatial block where a to-be-coded point is located, and a first color component value and a second color component value of the coded point;
[0021] A first construction unit is configured to construct a prediction model according to the spatial block; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be encoded;
[0022] A first prediction unit is configured to obtain a second color component prediction value of a to-be-encoded point by using a prediction model and a first color component value;
[0023] A first calculation unit is configured to calculate a difference between a second color component value and a predicted value of the second color component, and use the obtained difference as a residual of a point to be encoded;
[0024] The transform unit is configured to perform a RAHT transform on the residual of the coding point.
[0025] In a fourth aspect, an embodiment of the present application provides a decoder, the decoder comprising a second determination unit, a parsing unit, a second construction unit, a second prediction unit, an inverse transformation unit, and a second calculation unit, wherein:
[0026] A second determining unit is configured to determine a spatial block where a to-be-decoded point is located;
[0027] A parsing unit configured to parse the attribute bit stream to obtain a first color component value of a point to be decoded, an initial residual, and a first model parameter and a second model parameter corresponding to the spatial block;
[0028] A second construction unit is configured to construct a prediction model according to the first model parameter and the second model parameter; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be decoded;
[0029] A second prediction unit is configured to obtain a second color component prediction value of a to-be-decoded point by using the prediction model and the first color component value;
[0030] An inverse transform unit is configured to perform a RAHT inverse transform on the resolved initial residual to obtain a residual of a point to be decoded;
[0031] The second calculation unit is configured to calculate a reconstructed value of the second color component of the point to be decoded according to the residual of the point to be decoded and the predicted value of the second color component.
[0032] In a fifth aspect, an embodiment of the present application provides an encoder, the encoder comprising a first memory and a first processor, wherein:
[0033] A first memory, for storing a computer program that can be run on the first processor;
[0034] The first processor is configured to execute the method according to the first aspect when running a computer program.
[0035] In a sixth aspect, an embodiment of the present application provides a decoder, the decoder comprising a second memory and a second processor, wherein:
[0036] A second memory for storing a computer program that can be run on a second processor;
[0037] The second processor is used to execute the method described in the second aspect when running the computer program.
[0038] In a seventh aspect, an embodiment of the present application provides a computer storage medium storing a computer program, which implements the method described in the first aspect when the computer program is executed by a first processor, or implements the method described in the second aspect when the computer program is executed by a second processor.
[0039] The embodiment of the present application provides a color component prediction method, an encoder, a decoder and a computer storage medium. The method can be applied to the encoder, by determining the spatial block where the point to be encoded is located, and the first color component value and the second color component value of the encoded point; constructing a prediction model based on the spatial block, the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be encoded; then using the prediction model and the first color component value, obtaining the second color component prediction value of the point to be encoded; finally, calculating the difference between the second color component value and the second color component prediction value, using the obtained difference as the residual of the point to be encoded, and performing RAHT transformation on the residual of the point to be encoded; in this way, using one color component to predict another color component to obtain a predicted value, thereby calculating the prediction residual based on the actual value and the predicted value, which can remove the correlation between the color components, effectively reduce the encoding bit overhead, and improve the encoding efficiency; the method can also be applied to the decoder The encoder determines the spatial block where the point to be decoded is located; parses the attribute bit stream to obtain the first color component value, initial residual and first model parameters and second model parameters corresponding to the spatial block of the point to be decoded; constructs a prediction model based on the first model parameters and the second model parameters, and the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be decoded; then uses the prediction model and the first color component value to obtain the second color component prediction value of the point to be decoded; finally, performs RAHT inverse transformation on the parsed initial residual to obtain the residual of the point to be decoded; calculates the second color component reconstruction value of the point to be decoded based on the residual of the point to be decoded and the second color component prediction value; in this way, one color component is used to predict another color component to obtain a prediction value, so as to reconstruct the attribute value based on the prediction value and the prediction residual obtained by parsing, and since the correlation between the color components is removed, the coding bit overhead is effectively reduced, thereby improving the encoding and decoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 A flow chart of G-PCC coding provided for related technical solutions;
[0041] Figure 2 A flow chart of G-PCC decoding provided for related technical solutions;
[0042] Figure 3A schematic diagram of a hierarchical RAHT transformation provided for a related technical solution;
[0043] Figure 4 A schematic diagram of a flow chart of a color component prediction method provided in an embodiment of the present application;
[0044] Figure 5 A flowchart of a G-PCC encoding provided in an embodiment of the present application;
[0045] Figure 6 A schematic diagram of a flow chart of another color component prediction method provided in an embodiment of the present application;
[0046] Figure 7 A flowchart of a G-PCC decoding process provided in an embodiment of the present application;
[0047] Figure 8 A schematic diagram of the structure of an encoder provided in an embodiment of the present application;
[0048] Fig. 9 A schematic diagram of a specific hardware structure of an encoder provided in an embodiment of the present application;
[0049] Fig.10 A schematic diagram of the structure of a decoder provided in an embodiment of the present application;
[0050] Fig.11 A schematic diagram of the specific hardware structure of a decoder provided in an embodiment of the present application. DETAILED DESCRIPTION
[0051] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.
[0052] In an embodiment of the present application, in a point cloud G-PCC encoder framework, after the point cloud of the input three-dimensional image model is divided into slices, each slice is independently encoded.
[0053] See also Figure 1 , which shows a G-PCC coding flow chart provided by the related technical solution. Figure 1In the flowchart of G-PCC coding shown in FIG. 1 , it is applied to the point cloud encoder. For the point cloud data to be encoded, the point cloud data is first divided into multiple slices by slice division. In each slice, the geometric information of the point cloud and the attribute information corresponding to each point cloud are encoded separately. In the process of geometric coding, the geometric information is transformed so that all the point clouds are contained in a bounding box, and then quantized. This step of quantization mainly plays a role in scaling. Due to the quantization rounding, the geometric information of a part of the point cloud is the same, so it is determined whether to remove duplicate points based on parameters. The process of quantization and removal of duplicate points is also called voxelization. Then the bounding box is divided into octrees. In the octree-based geometric information coding process, the bounding box is divided into 8 sub-cubes in eight equal parts, and the non-empty (containing points in the point cloud) sub-cubes are further divided into eight equal parts until the leaf node obtained by division is a 1×1×1 unit cube, and the division is stopped. The points in the leaf node are arithmetically encoded to generate a binary geometric bit stream, that is, a geometric code stream. In the process of geometric information encoding based on triangle soup (trisoup), octree division must also be performed first. However, unlike the geometric information encoding based on octree, the trisoup does not need to divide the point cloud into unit cubes with a side length of 1x1x1 step by step. Instead, the division stops when the block (sub-block) has a side length of W. Based on the surface formed by the distribution of the point cloud in each block, the surface and the twelve edges of the block generate at most twelve vertices (intersections), and the vertices are arithmetically encoded (surface fitting based on the intersections) to generate a binary geometric bit stream, i.e., a geometric code stream. Vertex is also used to implement the process of geometric reconstruction, and the reconstructed set information is used when encoding the attributes of the point cloud.
[0054] In the attribute encoding process, after the geometric encoding is completed and the geometric information is reconstructed, color conversion is performed to convert the color information (i.e., attribute information) from the RGB color space to the YUV color space. Then, the point cloud is recolored using the reconstructed geometric information so that the unencoded attribute information corresponds to the reconstructed geometric information. Attribute encoding is mainly performed on color information. In the color information encoding process, there are two main transformation methods: one is the distance-based lifting transformation that depends on the level of detail (LOD) division, and the other is the direct transformation of the region adaptive hierarchical transformation (RAHT). Both methods will convert the color information from the spatial domain to the frequency domain, and obtain high-frequency coefficients and low-frequency coefficients through transformation. Finally, the coefficients are quantized (i.e., quantized coefficients). Finally, the geometric encoding data after octree division and surface fitting and the attribute encoding data processed by quantization coefficients are sliced and synthesized, and the vertex coordinates of each block are encoded in turn (i.e., arithmetic encoding) to generate a binary attribute bit stream, i.e., attribute code stream.
[0055] See also Figure 2 , which shows a flow chart of G-PCC decoding provided by the related technical solution. Figure 2 In the flowchart of G-PCC decoding shown, it is applied to the point cloud decoder. For the binary code stream obtained, the geometric bit stream and attribute bit stream in the binary code stream are first decoded independently. When decoding the geometric bit stream, the geometric information of the point cloud is obtained through arithmetic decoding-octree synthesis-surface fitting-reconstruction of geometry-inverse coordinate conversion; when decoding the attribute bit stream, the attribute information of the point cloud is obtained through arithmetic decoding-inverse quantization-LOD-based lifting inverse transform or RAHT-based inverse transform-inverse color conversion, and the three-dimensional image model of the point cloud data to be encoded is restored based on the geometric information and attribute information.
[0056] Figure 1 In the flowchart of G-PCC coding shown in the figure, RAHT transform is the part that encodes the attribute information of the point cloud. It is lossy coded by the principle of Harr wavelet transform and is generally applicable to sparse point clouds. Specifically, before RAHT transform, the geometric coordinate information of the point cloud has been obtained. The Morton code corresponding to each point in the point cloud can be obtained using the coordinate information of the point. Morton coding is also called z-ordercode because its coding order follows the spatial z-order. The specific method for calculating the Morton code is described as follows. For each component of the three-dimensional coordinate represented by a d-bit binary number, the representation of its three coordinate components is achieved by the following:
[0057]
[0058] Among them, x l ,y l , z l ∈{0, 1) are the binary values corresponding to the highest bit (l=1) to the lowest bit (l=d) of x, y, and z respectively. The Morton code M is to arrange x, y, and z in a crosswise order starting from the highest bit. l ,y l , z l To the lowest bit, the calculation formula of M is as follows:
[0059]
[0060] Among them, m l’ ∈{0, 1} are the values of the highest bit (l'=1) to the lowest bit (l'=3d) of M. After obtaining the Morton code M of each point in the point cloud, the points in the point cloud are arranged in order from small to large Morton codes, and the weight w of each point is set to 1.
[0061] RAHT transformation is performed on the basis of the hierarchical structure obtained by dividing the point cloud data into octrees, starting from the bottom layer of the octree and performing the transformation layer by layer. Figure 3 As shown, after the octree division is completed, voxel block 1 (i.e. Figure 3 The three colors and depths of the geometric body are alternated, and each square represents a point in the point cloud). Start the RAHT transformation from the bottom layer, taking the transformation sequence xyz as an example, as follows Figure 3 As shown, RAHT transformation is first performed along the x direction. If there are adjacent voxel blocks in the x direction, RAHT is performed on the two to obtain the weighted average (DC coefficient) and residual (AC coefficient) of the attribute values of the two adjacent points. Among them, the DC coefficient obtained exists as the attribute information of the voxel block 2 of the parent node, and the RAHT transformation of the next layer is performed; and the AC coefficient is retained for the final encoding. If there are no adjacent points, the attribute value of the voxel block is directly passed to the second-layer parent node. During the second-layer RAHT transformation, it is performed along the y direction. If there are adjacent voxel blocks in the y direction, RAHT is performed on the two to obtain the weighted average (DC coefficient) and residual (AC coefficient) of the attribute values of the two adjacent points. After that, the third-layer RAHT transformation is performed along the z direction, and the parent node voxel block 3 with three color depths is obtained as the child node of the next layer in the octree, and then the RAHT transformation is performed cyclically along the x, y, and z directions until there is only one parent node in the entire point cloud.
[0062] In practice, when traversing points in a point cloud, the Morton code of the sorted point cloud is used. That is, whether two child nodes are under the same parent node can be determined by whether the values of the Morton code after shifting one bit to the right are equal.
[0063] For the attribute values c1 and c2 of two adjacent points, the specific RAHT transformation process is as follows:
[0064]
[0065]
[0066] Among them, w is the weight corresponding to the DC coefficient, which is calculated. The DC coefficient is the weighted average of the attribute, and the AC coefficient is the attribute residual of two adjacent points.
[0067] In the embodiment of the present application, the specific steps of RAHT transformation are as follows:
[0068] (1) The attribute values of the points in the point cloud are used as the first-layer DC coefficients and their weights are all set to 1, and the RAHT transformation begins.
[0069] (2) The DC coefficient and AC coefficient of this layer are filled into the next parent layer, i.e., the parent node, according to the corresponding index. If the DC coefficient and AC coefficient need to be filled in, but there is no AC coefficient, then the AC coefficient can be left blank.
[0070] (3) According to the index sorted by the Morton code, traverse the DC coefficient corresponding to the index.
[0071] (4) Shift the Morton codes corresponding to all DC coefficients right by one position. At this time, the Morton code of each DC coefficient represents the Morton code of its parent node.
[0072] (5) Determine whether the Morton codes of the two DC coefficients are the same. If they are the same, it means that under the same parent node, the two perform RAHT, fill the obtained DC coefficient into the DC coefficient of the parent node of the next layer, fill the AC coefficient into the last DC coefficient of the next layer, and assign the weight of the sum of the two DC coefficients to the DC coefficient of the parent node; if they are different, fill this DC coefficient and its weight directly into the next layer.
[0073] (6) Repeat steps (2)-(5) until a layer has only one DC coefficient.
[0074] (7) Finally, the DC coefficient is quantized, and the DC coefficient and AC coefficient attribute values of this layer are encoded.
[0075] When attribute values are subjected to RAHT transformation in pairs, three color components of the attribute information (which can be represented by Y, U, and V, respectively) are calculated by RAHT transformation respectively, and the three are independent of each other and have no influence on each other.
[0076] In this way, firstly, according to the last obtained DC coefficient, it is used as the parent node, and the child node can be upsampled to obtain the predicted attribute value of the child node. The specific process is as follows:
[0077]
[0078] Among them, d i Indicates the distance between the center point of the adjacent parent node i and the center point of the child node, a i Indicates the attribute value of the parent node.
[0079] Then, the child node with the predicted attribute value is used as the parent node of the next layer, and the child nodes of the next layer are upsampled until the attribute prediction value of the bottom layer is calculated. Finally, the residual of the attribute prediction value and the actual value of the child node is obtained, and the residual is encoded.
[0080] Accordingly, in Figure 2 In the flowchart of G-PCC decoding shown in the figure, the RAHT inverse transform is the part that decodes the attribute information of the point cloud, which is similar to the encoding process. Before the RAHT inverse transform, the Morton code of each point in the point cloud is calculated. After obtaining the Morton code M of each point, the points in the point cloud are arranged in order from small to large, and the weight of each point is set to 1. The RAHT inverse process still traverses all the points in the point cloud in the order after the Morton code is sorted.
[0081] Among them, since the RAHT transformation is performed in layers, starting from the bottom layer, the adjacent points in the point cloud are judged layer by layer, and the attribute values are transformed by RAHT according to the weights. In this way, the inverse RAHT transformation starts from the top layer and performs the inverse RAHT transformation from top to bottom; that is, when performing the inverse RAHT transformation, starting from the top layer, the adjacent nodes are judged according to the Morton code information of each layer, and the inverse RAHT transformation is performed using the obtained weight information and the decoded attribute information.
[0082] However, the current solution is to perform RAHT transformation on the three channels corresponding to the three color components of the attribute (which can be represented by Y, U, and V respectively), which are independent of each other. Because there is a certain correlation between the three color components before RAHT transformation, the number of encoded bits will increase during RAHT transformation, resulting in a decrease in encoding efficiency.
[0083] An embodiment of the present application provides a color component prediction method, which is applied to an encoder, by determining the spatial block where a point to be encoded is located, and the first color component value and the second color component value of the encoded point; constructing a prediction model based on the spatial block, the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be encoded; then using the prediction model and the first color component value, obtaining the second color component prediction value of the point to be encoded; finally, calculating the difference between the second color component value and the second color component prediction value, using the obtained difference as the residual of the point to be encoded, and performing RAHT transformation on the residual of the point to be encoded; in this way, using one color component to predict another color component to obtain a predicted value, thereby calculating the predicted residual based on the actual value and the predicted value, which can remove the correlation between the color components, effectively reduce the encoding bit overhead, and improve the encoding efficiency; the method is also applied to a decoder, by determining the point to be decoded The spatial block where the point is located; the attribute bit stream is parsed to obtain the first color component value of the point to be decoded, the initial residual and the first model parameter and the second model parameter corresponding to the spatial block; according to the first model parameter and the second model parameter, a prediction model is constructed, and the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be decoded; the prediction model and the first color component value are then used to obtain the predicted value of the second color component of the point to be decoded; finally, the RAHT inverse transform is performed on the parsed initial residual to obtain the residual of the point to be decoded; according to the residual of the point to be decoded and the predicted value of the second color component, the reconstruction value of the second color component of the point to be decoded is calculated; in this way, one color component is used to predict another color component to obtain a predicted value, so as to reconstruct the attribute value according to the predicted value and the predicted residual obtained by parsing. Since the correlation between the color components is removed, the coding bit overhead is effectively reduced, thereby improving the encoding and decoding efficiency.
[0084] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.
[0085] See also Figure 4 , which shows a schematic flow chart of a color component prediction method provided in an embodiment of the present application, the method is applied to an encoder (also referred to as a point cloud encoder), and the method may include:
[0086] S401: Determine a spatial block where a point to be encoded is located, and a first color component value and a second color component value of the encoding point;
[0087] S402: constructing a prediction model according to the spatial block; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be encoded;
[0088] It should be noted that the point to be encoded represents the point cloud data of the object to be encoded in the point cloud. By spatially dividing the point cloud, multiple spatial blocks can be obtained. From these multiple spatial blocks, the spatial block where the point to be encoded is located can be determined, so that a prediction model can be constructed based on the spatial block in the future, thereby realizing the prediction between color components.
[0089] It should also be noted that for each point to be encoded in the space, the first color component, the second color component and the third color component can be used to represent the three color attributes of the point to be encoded. Among them, these three color attributes can be represented by Y, U, and V; specifically, the first color component can be represented by Y, the second color component can be represented by U, and the third color component can be represented by V; or the first color component can be represented by Y, the second color component can be represented by V, and the third color component can be represented by U; the embodiments of the present application are not specifically limited.
[0090] In addition, these three color attributes can be represented by R, G, and B; specifically, the first color component can be represented by R, the second color component can be represented by G, and the third color component can be represented by B; or the first color component can be represented by R, the second color component can be represented by B, and the third color component can be represented by G; the embodiments of the present application are not specifically limited.
[0091] In this way, according to the spatial block where the point to be encoded is located, a prediction model can be constructed, and the prediction model can characterize the prediction relationship between the first color component and the second color component of the point to be encoded. That is to say, using this prediction model, the second color component of the point to be encoded can be predicted based on the first color component of the point to be encoded. Specifically, assuming that the first color component is represented by Y and the second color component is represented by U, then the prediction model can predict the U component based on the Y component of the point to be encoded, that is, the prediction method of the embodiment of the present application is applicable to the prediction of the U component using the Y component; assuming that the first color component is represented by Y and the second color component is represented by V, then the prediction model can also predict the V component based on the Y component of the point to be encoded, that is, the prediction method of the embodiment of the present application is also applicable to the prediction of the V component using the Y component.
[0092] S403: Obtain a second color component prediction value of the to-be-encoded point by using the prediction model and the first color component value;
[0093] It should be noted that after constructing the prediction model, in order to predict the second color component of the point to be encoded, it is also necessary to obtain the first color component value of the point to be encoded; in this way, based on the obtained first color component value of the point to be encoded, the prediction value of the second color component of the point to be encoded can be obtained through the prediction model.
[0094] It should also be noted that if the prediction model is used to predict the U component based on the Y component of the point to be encoded, then the first color component value obtained is the Y component value, and the second color component value is the U component value. In this way, based on the prediction model and the Y component value, the U component prediction value of the point to be encoded can be obtained, and then the residual can be calculated based on the U component value and the U component prediction value; if the prediction model is used to predict the V component based on the Y component of the point to be encoded, then the first color component value obtained is the Y component value, and the second color component value is the V component value. In this way, based on the prediction model and the Y component value, the V component prediction value of the point to be encoded can be obtained, and then the residual can be calculated based on the V component value and the V component prediction value.
[0095] S404: Calculate the difference between the second color component value and the second color component prediction value, use the obtained difference as the residual of the point to be encoded, and perform a regional adaptive hierarchical transform (RAHT) on the residual of the point to be encoded.
[0096] It should be noted that after obtaining the predicted value of the second color component of the point to be encoded, the residual of the point to be encoded can be obtained according to the difference between the second color component value (actual value) of the point to be encoded and the predicted value of the second color component, and then the RAHT transform can be performed according to the residual of the point to be encoded.
[0097] In this way, the embodiment of the present application can losslessly predict the attribute information of the point cloud, and effectively reduce the coding bit overhead while obtaining the same image coding quality, thereby improving the coding efficiency. Among them, the image coding quality can use objective quality evaluation criteria, such as Peak Signal to Noise Ratio (PSNR), or subjective quality evaluation criteria, such as Mean Opinion Score (MOS).
[0098] Furthermore, after performing the RAHT transformation, the residual of the point to be coded needs to be encoded to be written into the bitstream. Therefore, in some embodiments, after performing the RAHT transformation on the residual of the point to be coded, the method may further include:
[0099] The residual of the point to be encoded is encoded and written into an attribute bitstream.
[0100] It should be noted that the code stream may include a geometry bit stream and an attribute bit stream. The geometry bit stream includes the geometry information of the point cloud, such as the position coordinate information, etc.; the attribute bit stream includes the attribute information of the point cloud, such as the three color information, etc. In this way, after the RAHT transformation, the residual of the point to be encoded needs to be encoded and written into the attribute bit stream; then it is transmitted from the encoder side to the decoder side, and the attribute bit stream is parsed on the decoder side so that the decoder side can obtain the residual of the point to be encoded.
[0101] See also Figure 5 , which shows a flow chart of a G-PCC encoding provided by an embodiment of the present application. Figure 5 As shown, compared with the encoding block diagram shown in the related art, before the RAHT transformation, a part of the prediction transformation between attribute color channels is added. The prediction method of the embodiment of the present application is mainly applied to this part to realize the attribute prediction between color channels. In this way, before the RAHT transformation, one color component is used to predict another color component to obtain a predicted value, thereby removing the correlation between the three color channels, thereby improving the encoding efficiency.
[0102] An embodiment of the present application provides a color component prediction method, which can be applied to an encoder, by determining the spatial block where a point to be encoded is located, and the first color component value and the second color component value of the encoded point; constructing a prediction model based on the spatial block, the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be encoded; then using the prediction model and the first color component value to obtain the second color component prediction value of the point to be encoded; finally, calculating the difference between the second color component value and the second color component prediction value, using the obtained difference as the residual of the point to be encoded, and performing RAHT transformation on the residual of the point to be encoded; in this way, one color component is used to predict another color component to obtain a predicted value, thereby calculating the prediction residual based on the actual value and the predicted value, which can remove the correlation between the color components, effectively reduce the encoding bit overhead, and improve the encoding efficiency.
[0103] Furthermore, the spatial block may be obtained by spatially dividing the point cloud. Therefore, in some embodiments, for S401, determining the spatial block where the point to be encoded is located may include:
[0104] S401-a: spatially divide the point cloud to obtain at least one spatial block;
[0105] It should be noted that since the RAHT encoding process is to perform RAHT transformation along the z, y, and x directions according to the Morton code, the closer the distance between the three color components (such as the Y component, the U component, and the V component), the higher the correlation; based on this, the point cloud can be spatially divided to obtain at least one spatial block.
[0106] In some embodiments, the point cloud may be divided into a plurality of spatial blocks according to the spatial neighbor relationship between points. Therefore, for S401-a, the spatial division of the point cloud to obtain at least one spatial block may include:
[0107] Dividing the point cloud into a plurality of point sets according to a spatial neighbor relationship between points in the point cloud, and determining a first correlation between a first color component and a second color component in each point set;
[0108] The point cloud is spatially divided according to the determined first correlation to obtain the at least one spatial block.
[0109] That is to say, the point cloud can be divided into multiple point sets based on the spatial adjacent relationship between points in the point cloud, so that the first correlation between the first color component and the second color component in each point set can be determined, and the point cloud can be divided into spatial blocks with high correlation between color components.
[0110] Specifically, the point cloud may be spatially divided according to the first correlation between the Y component and the U component in each point set, or the point cloud may be spatially divided according to the first correlation between the Y component and the V component in each point set; since the correlation between points with closer distances between color components is higher, at least one spatial block can be obtained by performing spatial division according to the first correlation between color components.
[0111] In some embodiments, the point cloud may be divided into a plurality of spatial blocks according to the Morton code of the point in the point cloud. Therefore, for S401-a, the spatial division of the point cloud to obtain at least one spatial block includes:
[0112] Calculating a Morton code for a point in the point cloud;
[0113] Arrange the Morton codes of the points in the point cloud in a preset order, and determine the maximum value of the Morton code and the minimum value of the Morton code;
[0114] Calculate the difference between the maximum value of the Morton code and the minimum value of the Morton code, and perform bit shifting N bits on the obtained difference in a preset direction to obtain the shifted difference; wherein N is a positive integer greater than or equal to 1;
[0115] When the shifted difference satisfies a preset range, the point cloud is divided into space blocks equal to the shifted difference.
[0116] It should be noted that the preset order can be in ascending order or in descending order; in the embodiment of the present application, the preset order is preferably in ascending order. In addition, the preset direction can be a right shift or a left shift; in the embodiment of the present application, the preset direction is preferably a right shift, that is, the obtained difference is bit-shifted right by N bits.
[0117] It should be noted that in a point cloud, a point can be all points in the point cloud or some points in the point cloud, and these points are relatively concentrated in space. In this way, for the division of spatial blocks, the Morton codes of these points can be calculated and the calculated Morton codes can be arranged in ascending order, that is, the obtained Morton codes are arranged in order from small to large; then the maximum value of the Morton code and the minimum value of the Morton code are determined, and then the difference between the maximum value of the Morton code and the minimum value of the Morton code is calculated (which can be expressed as delta).
[0118] By right shifting the delta bit, when the delta bit is right shifted by N bits, the shifted delta is obtained, which can make the shifted delta meet the preset range. At this time, the point cloud can be divided into delta space blocks after the shift. Among them, the delta bit is right shifted by N bits, which can be regarded as the maximum value bit of the Morton code is right shifted by N bits, and the minimum value bit of the Morton code is right shifted by N bits, and then the difference between the two is calculated, and the difference obtained is the delta bit right shifted by N bits.
[0119] In addition, the preset range indicates the range that the number of spatial blocks divided by the point cloud needs to satisfy. Here, the preset range may be 16<delta≤32 or 8<delta≤16, which is not specifically limited in the embodiment of the present application.
[0120] Further, after calculating the Morton code of the point in the point cloud, the method may further include:
[0121] Performing a bit shift of N bits on the Morton code of the midpoint of the point cloud in a preset direction;
[0122] In the point cloud, if the Morton code values of some points after shifting are the same, it is determined that the some points belong to the same spatial block.
[0123] It should be noted that the preset direction is preferably shifted to the right, that is, the Morton code of the point in the point cloud is bit-shifted right by N bits. In this way, if the Morton code values of some points after shifting are the same, then it can be determined that these points belong to the same spatial block, that is, the points with the same Morton code value after right shift can be marked as "located in the same spatial block".
[0124] S401-b: Based on the geometric position of the point to be encoded, determine the spatial block where the point to be encoded is located from the at least one spatial block.
[0125] It should be noted that by spatially dividing the point cloud, at least one spatial block can be obtained; in this way, according to the geometric position of the point to be encoded, the spatial block where the point to be encoded is located can be determined from the at least one spatial block, so as to facilitate the subsequent construction of a prediction model based on the spatial block.
[0126] It should also be noted that the shape of the space block includes at least one of the following: a cube shape, a rectangular parallelepiped shape, and an irregular shape.
[0127] The space block may be a cuboid in three-dimensional space divided into point clouds, and the size of the cuboid is a×b×c; wherein the values of a, b, and c may be equal or unequal; here, when the values of a, b, and c are equal, the cube is regarded as a special cuboid.
[0128] In addition, the spatial block may also be an irregularly shaped spatial block with strong correlation between color components, such as a triangle or a polygon, which may also be used to remove the correlation of attribute colors to obtain the prediction residual of the second color component.
[0129] Further, after determining the spatial block of points to be encoded, some points may be selected from the spatial block to form a point set, and the point set is used to construct a prediction model. Therefore, in some embodiments, for 402, constructing a prediction model according to the spatial block may include:
[0130] Selecting K points from the spatial block, and forming a point set from the selected K points; wherein K is a positive integer greater than or equal to 20;
[0131] Determine model parameters based on K points in the point set; wherein the model parameters include a first model parameter and a second model parameter;
[0132] The prediction model is constructed according to the first model parameters and the second model parameters.
[0133] It should be noted that, within the spatial block, K points can be selected from the points contained in the spatial block, and the selected K points can be combined into a point set; here, the value of K can be a positive integer greater than or equal to 20, but the embodiment of the present application does not make any specific limitation.
[0134] For example, according to the order of the Morton code, one point can be selected from every k (100 < k < 500) points in the space block. If the number of points contained in the space block is too small, 20 points can be uniformly selected so that the number of points selected in each space block is not less than 20 points. Assume that the point set is represented by V, for each space block corresponding to the point set V i {i=0, 1, ..., n} represents the set of points selected in the i-th spatial block, where n represents the number of spatial blocks.
[0135] In this way, after the selected points are formed into a point set, the first model parameters and the second model parameters can be determined according to the color attribute values of these points in the point set (such as the first color component value, the second color component value or the third color component value), and then a prediction model can be constructed based on the first model parameters and the second model parameters.
[0136] Further, in some embodiments, determining the model parameters based on the K points in the point set may include:
[0137] Obtain the first color component value and the second color component value of each point in the point set;
[0138] Calculating the first model parameter using a first calculation submodel according to the first color component value and the second color component value of each point in the point set;
[0139] The second model parameters are calculated using a second calculation sub-model according to the first model parameters.
[0140] It should be noted that constructing a prediction model can be regarded as fitting a straight line. That is, after determining the first model parameter and the second model parameter, a fitting straight line can be obtained according to the first model parameter and the second model parameter. The following will take the fitting straight line using the least squares method as an example for a detailed description, and the specific process is as follows:
[0141] First, for any point (x i ,y i ), coordinate x i Represents the first color component value of any point (expressed by the Y component value), coordinate y i The second color component value of any point is represented by the U component value. Thus, the fitting straight line obtained according to the Y component value and the U component value is y=a+bx.
[0142] Secondly, for any point (x i ,y i ), the error is d i =y-(a+bx i ), then when When the minimum value is taken, the fitting degree of the fitting line is the highest. In this way, the first-order partial derivatives of a and b are obtained respectively.
[0143]
[0144]
[0145] Again, let and Then, by solving equation (5) and equation (6) together, we can get
[0146]
[0147]
[0148] Finally, formula (7) is used to represent the first calculation sub-model for calculating the first model parameter, and formula (8) is used to represent the second calculation sub-model for calculating the second model parameter; in this way, the first model parameter b can be obtained through the first calculation sub-model, and the second model parameter a can be obtained through the second calculation sub-model, so that the fitting straight line, that is, the prediction model, can be obtained.
[0149] In this way, after obtaining the prediction model between the Y component value and the U component value, the Y component can be used to predict the U component according to the prediction model to obtain the predicted value of the U component; then the actual value and the predicted value are used as a residual, and the obtained residual replaces the original attribute value, and then the subsequent RAHT transformation is performed.
[0150] It should also be noted that for building a prediction model, that is, fitting a straight line, after selecting a point set from the spatial block, in addition to using the least squares method to fit the straight line, you can also use methods different from the least squares method to fit the straight line, such as the gradient descent method or the Gauss-Newton method; in addition, you can also use the color attribute values of these points in the point set to fit a segmented straight line or curve to obtain a prediction model; then write the model parameters involved in the prediction model into the attribute bit stream.
[0151] Further, in some embodiments, for 405, after performing RAHT transformation on the residual of the point to be encoded, the method may further include:
[0152] The first model parameter and the second model parameter are written into a property bitstream.
[0153] It should be noted that after the first model parameters and the second model parameters are determined, the first model parameters and the second model parameters can be written into an attribute bit stream, which is then transmitted from the encoder side to the decoder side, and the attribute bit stream is parsed on the decoder side, so that the decoder side only needs to obtain the first model parameters and the second model parameters, thereby improving the encoding efficiency.
[0154] Furthermore, after determining the first model parameter and the second model parameter, the first model parameter and the second model parameter may be further modified. The first model parameter and the second model parameter may be quantized and then written into the attribute bitstream; or, only the model parameters of the first spatial block (including the first model parameter and the second model parameter) may be written into the attribute bitstream, and the model parameters of the remaining spatial blocks may be modified into model parameter residual values compared with the first spatial block, and then the residual values may be sequentially written into the attribute bitstream.
[0155] Optionally, in some embodiments, writing the first model parameter and the second model parameter into the attribute bitstream may include:
[0156] quantizing the first model parameter and the second model parameter to obtain a quantized first model parameter and a quantized second model parameter;
[0157] The quantized first model parameters and the quantized second model parameters are written into the attribute bitstream.
[0158] It should be noted that if the first model parameters and the second model parameters are quantized at the encoder side, then after the attribute bitstream is parsed at the decoder side, the first model parameters and the second model parameters also need to be dequantized.
[0159] Optionally, in some embodiments, writing the first model parameter and the second model parameter into a property bitstream may include:
[0160] Writing a first model parameter and a second model parameter corresponding to a first spatial block in the at least one spatial block into an attribute bitstream;
[0161] The model parameter residual values between the remaining spatial blocks in the at least one spatial block and the previous spatial block are calculated, and the model parameter residual values are sequentially written into the attribute bitstream.
[0162] It should be noted that at least one spatial block includes a first spatial block and remaining spatial blocks. For the first model parameter and the second model parameter, if the model parameters of the first spatial block (including the first model parameter and the second model parameter) are first written into the attribute bitstream on the encoder side, and the model parameters of the remaining spatial blocks are modified to model parameter residual values compared with the first spatial block, and then the residual values are sequentially written into the attribute bitstream; then after the attribute bitstream is parsed on the decoder side, the model parameters of the remaining spatial blocks need to be determined based on the model parameter residual values obtained by parsing.
[0163] Furthermore, in some embodiments, the method may also include:
[0164] For the spatial block, calculating a second correlation between a first color component and a second color component corresponding to the spatial block;
[0165] If the second correlation is less than a preset correlation threshold, the Morton code of each point in the space block is shifted right by N-1 bits to obtain two subspace blocks;
[0166] Calculating a third correlation between the first color component and the second color component corresponding to the two subspace blocks;
[0167] If the third correlation is greater than the second correlation, constructing two sub-prediction models according to the two subspace blocks;
[0168] The second color component prediction value of the point to be encoded is obtained by using the two sub-prediction models and the first color component value.
[0169] It should be noted that, in the point cloud, when dividing the spatial blocks, the correlation between the color channels is calculated based on the point set selected in the spatial block to obtain the second correlation, that is, the correlation of the parent spatial block. When the calculated second correlation is low, for the points in the spatial block, the right shift bit of the Morton code is reduced by one (assuming that the right shift bit is N, then the right shift bit is reduced by one to N-1). At this time, the spatial block can be divided into two sub-space blocks. Then the correlation between the color channels of the two sub-space blocks is calculated to obtain the third correlation, that is, the correlation between the two sub-space blocks. If the correlation between the two sub-space blocks is significantly greater than the correlation of the parent spatial block, then the spatial block can be divided into two sub-space blocks, that is, two sub-space blocks are used to perform predictions between the color channels respectively, that is, two sub-prediction models are constructed according to the two sub-space blocks to predict the second color component according to the first color component; otherwise, if the correlation between the two sub-space blocks is less than or equal to the correlation of the parent spatial block, then the parent spatial block can be used for prediction, that is, execution Figure 4 The forecasting method flow is shown.
[0170] In an embodiment of the present application, the encoding performance of the G-PCC attribute encoding part can be improved. Under the premise of basically not affecting the encoding performance, for example, without affecting the attribute PSNR, the bit rate of the first color channel (represented by Y), the second color channel (represented by U) and the third color channel (represented by V) of the attribute part can be reduced. Here, PSNR is an objective evaluation criterion for image encoding quality, and the larger the PSNR, the better the encoding quality of the image. In addition, BD-rate (Bjontegaard-Delta rate) can be used to measure the quality of encoding performance. Specifically, as shown in Table 1, if BD-rate is a negative value, it means that the performance has improved; and if the absolute value of BD-rate is larger, the performance gain is greater.
[0171] Table 1
[0172]
[0173] This embodiment provides a color component prediction method, which is applied to an encoder. The above embodiment is used to elaborate the specific implementation of the above embodiment, from which it can be seen that by using one color component to predict another color component to obtain a predicted value, and then calculating the prediction residual according to the actual value and the predicted value, the correlation between the color components can be removed, the coding bit overhead is effectively reduced, and the coding efficiency can be improved.
[0174] See also Figure 6 , which shows a flow chart of another color component prediction method provided by an embodiment of the present application, the method is applied to a decoder (also referred to as a point cloud decoder), and the method may include:
[0175] S601: Determine the spatial block where the point to be decoded is located;
[0176] It should be noted that the point to be decoded represents the point cloud data of the object to be decoded in the point cloud. By spatially dividing the point cloud, multiple spatial blocks can be obtained. From these multiple spatial blocks, the spatial block where the point to be decoded is located can be determined, so that a prediction model can be constructed based on the spatial block later, thereby realizing the prediction between color components.
[0177] It should also be noted that for each point to be decoded in the space, the three color attributes of the point to be decoded can be represented by the first color component, the second color component and the third color component. Among them, these three color attributes can be represented by Y, U and V; specifically, the first color component can be represented by Y, the second color component can be represented by U, and the third color component can be represented by V; or the first color component can be represented by Y, the second color component can be represented by V, and the third color component can be represented by U; the embodiments of the present application are not specifically limited.
[0178] In addition, these three color attributes can be represented by R, G, and B; specifically, the first color component can be represented by R, the second color component can be represented by G, and the third color component can be represented by B; or the first color component can be represented by R, the second color component can be represented by B, and the third color component can be represented by G; the embodiments of the present application are not specifically limited.
[0179] S602: Parse the attribute bit stream to obtain the first color component value of the point to be decoded, the initial residual, and the first model parameter and the second model parameter corresponding to the spatial block;
[0180] It should be noted that when the code stream is transmitted from the encoder side to the decoder side, the code stream can be parsed and processed by the decoder side. Here, the code stream may include a geometry bit stream and an attribute bit stream; wherein the geometry bit stream includes geometry information of the point cloud, and by parsing the geometry bit stream on the decoder side, the geometry information of the point cloud, such as the position coordinates of the point to be decoded, etc., can be obtained; the attribute bit stream includes attribute information of the point cloud, and by parsing the attribute bit stream on the decoder side, the attribute information of the point cloud can be obtained; such as the first color component value of the point to be decoded, the initial residual, and the first model parameter and the second model parameter corresponding to the spatial block.
[0181] S603: constructing a prediction model according to the first model parameters and the second model parameters; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be decoded;
[0182] It should be noted that, according to the first model parameter and the second model parameter, a prediction model can be constructed, and the prediction model can characterize the prediction relationship between the first color component and the second color component of the point to be decoded.
[0183] That is to say, using this prediction model, the second color component of the point to be decoded can be predicted based on the first color component of the point to be decoded. Specifically, assuming that the first color component is represented by Y and the second color component is represented by U, then the prediction model can predict the U component based on the Y component of the point to be decoded, that is, the prediction method of the embodiment of the present application is applicable to the prediction of the U component using the Y component; assuming that the first color component is represented by Y and the second color component is represented by V, then the prediction model can also predict the V component based on the Y component of the point to be decoded, that is, the prediction method of the embodiment of the present application is also applicable to the prediction of the V component using the Y component.
[0184] S604: Obtain a second color component prediction value of the to-be-decoded point by using the prediction model and the first color component value;
[0185] It should be noted that after constructing the prediction model, in order to predict the second color component of the point to be decoded, it is also necessary to obtain the first color component value of the point to be decoded; in this way, based on the obtained first color component value of the point to be decoded, the prediction model can be used to obtain the predicted value of the second color component of the point to be decoded.
[0186] It should also be noted that if the prediction model is used to predict the U component based on the Y component of the point to be decoded, then the first color component value obtained is the Y component value, and the second color component value is the U component value. In this way, based on the prediction model and the Y component value, the U component prediction value of the point to be decoded can be obtained; or, if the prediction model is used to predict the V component based on the Y component of the point to be decoded, then the first color component value obtained is the Y component value, and the second color component value is the V component value. In this way, based on the prediction model and the Y component value, the V component prediction value of the point to be decoded can be obtained.
[0187] S605: Perform RAHT inverse transformation on the parsed initial residual to obtain the residual of the point to be decoded;
[0188] S606: Calculate and obtain a reconstructed value of the second color component of the point to be decoded according to the residual of the point to be decoded and the predicted value of the second color component.
[0189] It should be noted that after the initial residual is obtained through analysis, it is necessary to perform an inverse RAHT transform on the initial residual so as to obtain the residual of the point to be decoded.
[0190] In this way, after obtaining the residual and the second color component prediction value of the point to be decoded, the residual and the second color component prediction value of the point to be decoded are superimposed to obtain the second color component reconstruction value of the point to be decoded.
[0191] See also Figure 7 , which shows a flow chart of a G-PCC decoding process provided by an embodiment of the present application. Figure 7As shown, compared with the decoding block diagram shown in the related art, after the RAHT inverse transform, the part of the inverse prediction transform between the attribute color channels is added. The prediction method of the embodiment of the present application is mainly applied to this part to realize the attribute prediction between color channels. In this way, after the RAHT inverse transform, one color component is used to predict another color component to obtain a predicted value, and the attribute information of the point cloud can be losslessly predicted, the correlation between the three color channels is removed, and the bit overhead is effectively reduced while obtaining the same image coding quality, thereby improving the decoding efficiency. Among them, the image decoding quality can use objective quality evaluation criteria, such as Peak Signal to Noise Ratio (PSNR), or subjective quality evaluation criteria, such as Mean Opinion Score (MOS).
[0192] An embodiment of the present application provides a color component prediction method, which can be applied to a decoder, by determining the spatial block where a point to be decoded is located; parsing the attribute bit stream to obtain the first color component value, initial residual and first model parameters and second model parameters corresponding to the spatial block of the point to be decoded; constructing a prediction model based on the first model parameters and the second model parameters, the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be decoded; then using the prediction model and the first color component value to obtain the second color component prediction value of the point to be decoded; finally, performing a RAHT inverse transform on the parsed initial residual to obtain the residual of the point to be decoded; calculating the reconstructed value of the second color component of the point to be decoded based on the residual of the point to be decoded and the second color component prediction value; in this way, one color component is used to predict another color component to obtain a prediction value, so that the attribute value is reconstructed based on the prediction value and the prediction residual obtained by parsing, and the decoding efficiency is improved because the correlation between the color components is removed.
[0193] Further, at the decoder side, the spatial block may also be obtained by spatially dividing the point cloud. Therefore, in some embodiments, for S601, determining the spatial block where the point to be decoded is located may include:
[0194] S601-a: spatially divide the point cloud to obtain at least one spatial block;
[0195] It should be noted that since the RAHT encoding process is to perform RAHT transform along the z, y, and x directions according to the Morton code, and the RAHT decoding process is to perform RAHT inverse transform along the z, y, and x directions according to the Morton code, the closer the distance between the three color components (such as the Y component, the U component, and the V component), the higher the correlation; based on this, the point cloud can be spatially divided to obtain at least one spatial block.
[0196] In some embodiments, the point cloud may be divided into a plurality of spatial blocks according to the spatial adjacency relationship between points. Therefore, for S601-a, the spatial division of the point cloud to obtain at least one spatial block may include:
[0197] Dividing the point cloud into a plurality of point sets according to a spatial neighbor relationship between points in the point cloud, and determining a first correlation between a first color component and a second color component in each point set;
[0198] The point cloud is spatially divided according to the determined first correlation to obtain the at least one spatial block.
[0199] That is to say, the point cloud can be divided into multiple point sets based on the spatial adjacent relationship between points in the point cloud, so that the first correlation between the first color component and the second color component in each point set can be determined, and the point cloud can be divided into spatial blocks with high correlation between color components.
[0200] Specifically, the point cloud may be spatially divided according to the first correlation between the Y component and the U component in each point set, or the point cloud may be spatially divided according to the first correlation between the Y component and the V component in each point set; since the correlation between points with closer distances between color components is higher, at least one spatial block can be obtained by performing spatial division according to the first correlation between color components.
[0201] In some embodiments, the point cloud may be divided into a plurality of spatial blocks according to the Morton code of the point in the point cloud. Therefore, for S601-a, the spatial division of the point cloud to obtain at least one spatial block includes:
[0202] Calculating a Morton code for a point in the point cloud;
[0203] Arrange the Morton codes of the points in the point cloud in a preset order, and determine the maximum value of the Morton code and the minimum value of the Morton code;
[0204] Calculate the difference between the maximum value of the Morton code and the minimum value of the Morton code, and perform bit shifting N bits on the obtained difference in a preset direction to obtain the shifted difference; wherein N is a positive integer greater than or equal to 1;
[0205] When the shifted difference satisfies a preset range, the point cloud is divided into space blocks equal to the shifted difference.
[0206] It should be noted that the preset order can be in ascending order or in descending order; in the embodiment of the present application, the preset order is preferably in ascending order. In addition, the preset direction can be a right shift or a left shift; in the embodiment of the present application, the preset direction is preferably a right shift, that is, the obtained difference is bit-shifted right by N bits.
[0207] It should be noted that in a point cloud, a point can be all points in the point cloud or some points in the point cloud, and these points are relatively concentrated in space. In this way, for the division of spatial blocks, the Morton codes of these points can be calculated and the calculated Morton codes can be arranged in ascending order, that is, the obtained Morton codes are arranged in order from small to large; then the maximum value of the Morton code and the minimum value of the Morton code are determined, and then the difference between the maximum value of the Morton code and the minimum value of the Morton code is calculated (which can be expressed as delta).
[0208] By right shifting the delta bit, when the delta bit is right shifted by N bits, the shifted delta is obtained, which can make the shifted delta meet the preset range. At this time, the point cloud can be divided into delta space blocks after the shift. Among them, the delta bit is right shifted by N bits, which can be regarded as the maximum value bit of the Morton code is right shifted by N bits, and the minimum value bit of the Morton code is right shifted by N bits, and then the difference between the two is calculated, and the difference obtained is the delta bit right shifted by N bits.
[0209] In addition, the preset range indicates the range that the number of spatial blocks divided by the point cloud needs to satisfy. Here, the preset range may be 16<delta≤32 or 8<delta≤16, which is not specifically limited in the embodiment of the present application.
[0210] Further, after calculating the Morton code of the point in the point cloud, the method may further include:
[0211] Performing a bit shift of N bits on the Morton code of the midpoint of the point cloud in a preset direction;
[0212] In the point cloud, if the Morton code values of some points after shifting are the same, it is determined that the some points belong to the same spatial block.
[0213] It should be noted that the preset direction is preferably shifted to the right, that is, the Morton code of the point in the point cloud is bit-shifted right by N bits. In this way, if the Morton code values of some points after shifting are the same, then it can be determined that these points belong to the same spatial block, that is, the points with the same Morton code value after right shift can be marked as "located in the same spatial block".
[0214] S601-b: Based on the geometric position of the point to be decoded, determine the spatial block where the point to be decoded is located from the at least one spatial block.
[0215] It should be noted that by spatially dividing the point cloud, at least one spatial block can be obtained; in this way, according to the geometric position of the point to be decoded, the spatial block where the point to be decoded is located can be determined from the at least one spatial block, so as to facilitate the subsequent construction of a prediction model based on the spatial block.
[0216] It should also be noted that the shape of the space block includes at least one of the following: a cube shape, a rectangular parallelepiped shape, and an irregular shape.
[0217] The space block may be a cuboid in three-dimensional space divided into point clouds, and the size of the cuboid is a×b×c; wherein the values of a, b, and c may be equal or unequal; here, when the values of a, b, and c are equal, the cube is regarded as a special cuboid.
[0218] In addition, the spatial block may also be an irregular-shaped spatial block with strong correlation between color components, such as a triangle or a polygon, which may also be used to remove the correlation of attribute colors to improve decoding efficiency.
[0219] Furthermore, for the first model parameters and the second model parameters, after parsing the attribute bit stream, the first model parameters and the second model parameters need to be dequantized; alternatively, the model parameters of the first spatial block (including the first model parameters and the second model parameters) and the model parameter residual values of the remaining spatial blocks can be obtained through parsing, and then the model parameters of the remaining spatial blocks can be obtained based on the model parameter residual values.
[0220] Optionally, in some embodiments, after parsing the attribute bitstream, the method may further include:
[0221] The first model parameters and the second model parameters obtained by analysis are dequantized to obtain the first model parameters and the second model parameters.
[0222] It should be noted that if the first model parameters and the second model parameters are quantized on the encoder side, then after the attribute bitstream is parsed on the decoder side, the first model parameters and the second model parameters need to be dequantized to obtain the first model parameters and the second model parameters.
[0223] Optionally, in some embodiments, after parsing the attribute bitstream, the method may further include:
[0224] Obtaining a first model parameter and a second model parameter corresponding to a first spatial block in the at least one spatial block and residual values of the model parameters corresponding to the remaining spatial blocks in the at least one spatial block;
[0225] The first model parameters and the second model parameters corresponding to the remaining spatial blocks are obtained according to the first model parameters and the second model parameters corresponding to the first spatial block and the model parameter residual values.
[0226] It should be noted that at least one spatial block includes a first spatial block and remaining spatial blocks. For the first model parameter and the second model parameter, if the model parameters of the first spatial block (including the first model parameter and the second model parameter) are first written into the attribute bitstream on the encoder side, and the model parameters of the remaining spatial blocks are modified to model parameter residual values compared with the first spatial block, and then the residual values are sequentially written into the attribute bitstream; then after the attribute bitstream is parsed on the decoder side, the model parameters of the remaining spatial blocks (including the first model parameter and the second model parameter) need to be determined based on the model parameter residual values obtained by parsing.
[0227] In an embodiment of the present application, in a point cloud, when dividing a spatial block, the correlation between the color channels is calculated based on the point set selected in the spatial block to obtain a second correlation, that is, the correlation of the parent spatial block. When the calculated second correlation is low, for the points in the spatial block, the number of right shifts of the Morton code is reduced by one (assuming that the number of right shifts is N, then the number of right shifts reduced by one is N-1), and the spatial block can be divided into two sub-space blocks. Then the correlation between the color channels of the two sub-space blocks is calculated to obtain a third correlation, that is, the correlation between the two sub-space blocks. If the correlation between the two sub-space blocks is significantly greater than the correlation of the parent spatial block, then the spatial block can be divided into two sub-space blocks, that is, two sub-space blocks are used to perform predictions between the color channels respectively, that is, two sub-prediction models are constructed based on the two sub-space blocks to predict the second color component based on the first color component; otherwise, if the correlation between the two sub-space blocks is less than or equal to the correlation of the parent spatial block, then the parent spatial block can be used for prediction, that is, execution Figure 4 The forecasting method flow is shown.
[0228] The embodiment of the present application is to predict the attributes between color channels through the correlation between color channels under RAHT transformation. On the encoder side, it is necessary to divide the spatial blocks and perform straight line fitting, write the calculated first model parameters and second model parameters into the bitstream, and transmit them to the decoder. On the decoder side, it is only necessary to divide the spatial blocks, and then predict the second color component value based on the first model parameters and second model parameters of the corresponding spatial blocks obtained by analysis and the first color component value, and finally superimpose the predicted value and the residual value obtained after analysis and RAHT inverse transformation to obtain the required attribute reconstruction value. In this way, the bit rate of the first color channel (represented by Y), the second color channel (represented by U), and the third color channel (represented by V) of the attribute part can be reduced without substantially affecting the performance.
[0229] This embodiment provides a color component prediction method, which is applied to a decoder. The specific implementation of the above embodiment is described in detail through the above embodiment, from which it can be seen that one color component is used to predict another color component to obtain a predicted value, and then the attribute value is reconstructed according to the predicted value and the prediction residual obtained by analysis, and the correlation between the color components is removed, thereby improving the decoding efficiency.
[0230] Based on the same inventive concept as the above embodiments, see Figure 8 , which shows a schematic diagram of the composition structure of an encoder 80 provided in an embodiment of the present application. The encoder 80 may include a first determination unit 801, a first construction unit 802, a first prediction unit 803, a first calculation unit 804 and a transformation unit 805, wherein:
[0231] The first determining unit 801 is configured to determine the space block where the to-be-coded point is located, and the first color component value and the second color component value of the coded point;
[0232] The first construction unit 802 is configured to construct a prediction model according to the spatial block; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be encoded;
[0233] The first prediction unit 803 is configured to obtain a second color component prediction value of the to-be-encoded point by using the prediction model and the first color component value;
[0234] The first calculation unit 804 is configured to calculate the difference between the second color component value and the second color component prediction value, and use the obtained difference as the residual of the to-be-encoded point;
[0235] The transform unit 805 is configured to perform RAHT transform on the residual of the point to be encoded.
[0236] In the above scheme, see Figure 8 The encoder 80 may further include a writing unit 806 configured to encode the residual of the point to be encoded and write it into the attribute bitstream.
[0237] In the above scheme, see Figure 8 , the encoder 80 may further include a first division unit 807 configured to spatially divide the point cloud to obtain at least one spatial block;
[0238] The first determining unit 801 is configured to determine, based on the geometric position of the point to be encoded, the spatial block where the point to be encoded is located from the at least one spatial block.
[0239] In the above solution, the first determining unit 801 is further configured to divide the point cloud into a plurality of point sets according to the spatial neighbor relationship between points in the point cloud, and determine a first correlation between the first color component and the second color component in each point set;
[0240] The first division unit 807 is configured to perform spatial division on the point cloud according to the determined first correlation to obtain the at least one spatial block.
[0241] In the above solution, the first calculation unit 804 is further configured to calculate the Morton code of the point in the point cloud;
[0242] The first determining unit 801 is further configured to arrange the Morton codes of the points in the point cloud in a preset order, and determine the maximum value of the Morton code and the minimum value of the Morton code;
[0243] The first calculation unit 804 is further configured to calculate the difference between the maximum value of the Morton code and the minimum value of the Morton code, and perform a bit shift N bits on the obtained difference in a preset direction to obtain the shifted difference; wherein N is a positive integer greater than or equal to 1;
[0244] The first division unit 807 is configured to divide the point cloud into space blocks equal to the shifted difference when the shifted difference satisfies a preset range.
[0245] In the above scheme, the first determination unit 801 is also configured to perform a bit shift of N bits on the Morton code of the point in the point cloud in a preset direction; and in the point cloud, if the Morton code values of some points after shifting are the same, it is determined that the some points belong to the same spatial block.
[0246] In the above scheme, see Figure 8The encoder 80 may further include a selection unit 808 configured to select K points from the spatial block and form a point set with the selected K points; wherein K is a positive integer greater than or equal to 20;
[0247] The first determining unit 801 is configured to determine model parameters based on the K points in the point set; wherein the model parameters include first model parameters and second model parameters;
[0248] The first construction unit 802 is configured to construct the prediction model according to the first model parameters and the second model parameters.
[0249] In the above scheme, see Figure 8 , the encoder 80 may further include a first acquisition unit 809 configured to acquire a first color component value and a second color component value of each point in the point set;
[0250] The first calculation unit 804 is further configured to calculate the first model parameters using the first calculation sub-model according to the first color component value and the second color component value of each point in the point set; and calculate the second model parameters using the second calculation sub-model according to the first model parameters.
[0251] In the above solution, the writing unit 806 is further configured to write the first model parameter and the second model parameter into the attribute bitstream.
[0252] In the above scheme, see Figure 8 The encoder 80 may further include a quantization unit 810 configured to perform quantization processing on the first model parameter and the second model parameter to obtain a quantized first model parameter and a quantized second model parameter;
[0253] The writing unit 806 is further configured to write the quantized first model parameter and the quantized second model parameter into the attribute bitstream.
[0254] In the above scheme, the writing unit 806 is also configured to write the first model parameter and the second model parameter corresponding to the first spatial block in the at least one spatial block into the attribute bit stream; and calculate the model parameter residual values of the remaining spatial blocks and the previous spatial block in the at least one spatial block, and write the model parameter residual values into the attribute bit stream in sequence; wherein the at least one spatial block includes the first spatial block and the remaining spatial blocks.
[0255] In the above solution, the first calculation unit 804 is further configured to calculate, for the spatial block, a second correlation between the first color component and the second color component corresponding to the spatial block;
[0256] The first division unit 807 is further configured to, if the second correlation is less than a preset correlation threshold, right-shift the Morton code of each point in the space block by N-1 bits to obtain two subspace blocks;
[0257] The first calculation unit 804 is further configured to calculate a third correlation between the first color component and the second color component corresponding to the two subspace blocks;
[0258] The first construction unit 802 is further configured to construct two sub-prediction models according to the two subspace blocks if the third correlation is greater than the second correlation;
[0259] The first prediction unit 803 is further configured to obtain a second color component prediction value of the to-be-encoded point by using the two sub-prediction models and the first color component value.
[0260] In the above scheme, the shape of the space block includes at least one of the following: a cube shape, a rectangular parallelepiped shape and an irregular shape.
[0261] It can be understood that in this embodiment, a "unit" can be a part of a circuit, a part of a processor, a part of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional module.
[0262] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.
[0263] Therefore, this embodiment provides a computer storage medium, which stores a color component prediction program. When the color component prediction program is executed by a first processor, the method described in any one of the above embodiments is implemented.
[0264] Based on the composition of the encoder 80 and the computer storage medium, see Fig. 9 , which shows the specific hardware structure of the encoder 80 provided in the embodiment of the present application, which may include: a first communication interface 901, a first memory 902 and a first processor 903; each component is coupled together through a first bus system 904. It can be understood that the first bus system 904 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 904 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Fig. 9 In the figure, various buses are marked as the first bus system 904.
[0265] The first communication interface 901 is used to receive and send signals during the process of sending and receiving information with other external network elements;
[0266] A first memory 902, used to store a computer program that can be run on the first processor 903;
[0267] The first processor 903 is configured to execute, when running the computer program:
[0268] Determine the spatial block where the to-be-coded point is located, and the first color component value and the second color component value of the coded point;
[0269] Constructing a prediction model according to the spatial block; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be encoded;
[0270] Obtaining a predicted value of a second color component of the point to be encoded by using the prediction model and the first color component value;
[0271] The difference between the second color component value and the predicted value of the second color component is calculated, the obtained difference is used as the residual of the point to be encoded, and a regional adaptive hierarchical transform (RAHT) is performed on the residual of the point to be encoded.
[0272] It can be understood that the first memory 902 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRRAM). The first memory 902 of the systems and methods described in the present application is intended to include, but is not limited to, these and any other suitable types of memory.
[0273] The first processor 903 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the first processor 903. The above-mentioned first processor 903 can be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor are combined and executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the first memory 902, and the first processor 903 reads the information in the first memory 902 and completes the steps of the above method in combination with its hardware.
[0274] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processors (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units or combinations thereof for performing functions described in the present application. For software implementation, the technology described in the present application can be implemented by a module (such as a process, function, etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in a processor or outside a processor.
[0275] Optionally, as another embodiment, the first processor 903 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.
[0276] This embodiment provides an encoder, which may include a first determination unit, a first construction unit, a first acquisition unit, a first prediction unit, a first calculation unit and a transformation unit. The first determination unit is configured to determine the spatial block where the point to be encoded is located; the first construction unit is configured to construct a prediction model based on the spatial block, and the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be encoded; the first acquisition unit is configured to obtain the first color component value and the second color component value of the point to be encoded; the first prediction unit is configured to obtain the second color component prediction value of the point to be encoded using the prediction model and the first color component value; the first calculation unit is configured to calculate the difference between the second color component value and the second color component prediction value, and use the obtained difference as the residual of the point to be encoded; the transformation unit is configured to perform RAHT transformation on the residual of the point to be encoded; in this way, one color component is used to predict another color component to obtain a predicted value, so that the prediction residual is calculated based on the actual value and the predicted value, which can remove the correlation between the color components, effectively reduce the coding bit overhead, and improve the coding efficiency.
[0277] Based on the same inventive concept as the above embodiments, see Fig.10 , which shows a schematic diagram of the structure of a decoder 100 provided in an embodiment of the present application. The decoder 100 may include a second determination unit 1001, a parsing unit 1002, a second construction unit 1003, a second prediction unit 1004, an inverse transformation unit 1005, and a second calculation unit 1006, wherein:
[0278] The second determining unit 1001 is configured to determine the spatial block where the to-be-decoded point is located;
[0279] The parsing unit 1002 is configured to parse the attribute bit stream to obtain the first color component value of the point to be decoded, the initial residual, and the first model parameter and the second model parameter corresponding to the spatial block;
[0280] The second construction unit 1003 is configured to construct a prediction model according to the first model parameter and the second model parameter; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be decoded;
[0281] The second prediction unit 1004 is configured to obtain a second color component prediction value of the to-be-decoded point by using the prediction model and the first color component value;
[0282] The inverse transform unit 1005 is configured to perform a RAHT inverse transform on the parsed initial residual to obtain the residual of the point to be decoded;
[0283] The second calculation unit 1006 is configured to calculate a reconstructed value of the second color component of the point to be decoded according to the residual of the point to be decoded and the predicted value of the second color component.
[0284] In the above scheme, see Fig.10 , the encoder 100 may further include a second division unit 1007, configured to perform spatial division on the point cloud to obtain at least one spatial block;
[0285] The second determining unit 1001 is configured to determine, based on the geometric position of the point to be decoded, the spatial block where the point to be decoded is located from the at least one spatial block.
[0286] In the above solution, the second determining unit 1001 is further configured to divide the point cloud into a plurality of point sets according to the spatial adjacent relationship between points in the point cloud, and determine a first correlation between the first color component and the second color component in each point set;
[0287] The second division unit 1007 is configured to perform spatial division on the point cloud according to the determined first correlation to obtain the at least one spatial block.
[0288] In the above solution, the second calculation unit 1006 is further configured to calculate the Morton code of the point in the point cloud;
[0289] The second determining unit 1001 is further configured to arrange the Morton codes of the points in the point cloud in a preset order, and determine the maximum value of the Morton code and the minimum value of the Morton code;
[0290] The second calculation unit 1006 is further configured to calculate the difference between the maximum value of the Morton code and the minimum value of the Morton code, and perform a bit shift N bits in a preset direction on the obtained difference to obtain the shifted difference; wherein N is a positive integer greater than or equal to 1;
[0291] The second division unit 1007 is configured to divide the point cloud into space blocks equal to the shifted difference when the shifted difference satisfies a preset range.
[0292] In the above scheme, the second determination unit 1001 is also configured to perform a bit shift of N bits on the Morton code of the point in the point cloud in a preset direction; and in the point cloud, if the Morton code values of some points after shifting are the same, it is determined that the some points belong to the same spatial block.
[0293] In the above scheme, see Fig.10The encoder 100 may further include a dequantization unit 1008 configured to perform dequantization processing on the first model parameter and the second model parameter obtained by parsing to obtain the first model parameter and the second model parameter.
[0294] In the above scheme, the parsing unit 1002 is also configured to obtain the first model parameters and the second model parameters corresponding to the first spatial block in the at least one spatial block and the model parameter residual values corresponding to the remaining spatial blocks in the at least one spatial block; and obtain the first model parameters and the second model parameters corresponding to the remaining spatial blocks based on the first model parameters and the second model parameters corresponding to the first spatial block and the model parameter residual values.
[0295] In the above scheme, the shape of the space block includes at least one of the following: a cube shape, a rectangular parallelepiped shape and an irregular shape.
[0296] It can be understood that in this embodiment, a "unit" can be a part of a circuit, a part of a processor, a part of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional module.
[0297] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer storage medium, which stores a color component prediction program. When the color component prediction program is executed by the second processor, the method described in any one of the above embodiments is implemented.
[0298] Based on the above-mentioned components of the decoder 100 and the computer storage medium, see Fig.11 , which shows the specific hardware structure of the decoder 100 provided in the embodiment of the present application, which may include: a second communication interface 1101, a second memory 1102, and a second processor 1103; each component is coupled together via a second bus system 1104. It can be understood that the second bus system 1104 is used to realize the connection and communication between these components. In addition to the data bus, the second bus system 1104 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in Fig.11 In the figure, various buses are labeled as the second bus system 1104. Among them,
[0299] The second communication interface 1101 is used for receiving and sending signals during the process of sending and receiving information with other external network elements;
[0300] The second memory 1102 is used to store a computer program that can be run on the second processor 1103;
[0301] The second processor 1103 is configured to execute, when running the computer program:
[0302] Determine the spatial block where the point to be decoded is located;
[0303] Parsing the attribute bit stream to obtain the first color component value of the to-be-decoded point, the initial residual, and the first model parameter and the second model parameter corresponding to the spatial block;
[0304] Constructing a prediction model according to the first model parameter and the second model parameter; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be decoded;
[0305] Obtaining a second color component prediction value of the point to be decoded by using the prediction model and the first color component value;
[0306] Performing an inverse RAHT transform on the resolved initial residual to obtain the residual of the point to be decoded;
[0307] A second color component reconstruction value of the point to be decoded is calculated according to the residual of the point to be decoded and the predicted value of the second color component.
[0308] Optionally, as another embodiment, the second processor 1103 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.
[0309] It can be understood that the hardware functions of the second memory 1102 and the first memory 902 are similar, and the hardware functions of the second processor 1103 and the first processor 903 are similar; they will not be described in detail here.
[0310] This embodiment provides a decoder, which may include a second determination unit, a parsing unit, a second construction unit, a second prediction unit, an inverse transformation unit, and a second calculation unit. The second determination unit is configured to determine the spatial block where the point to be decoded is located; the parsing unit is configured to parse the attribute bit stream to obtain the first color component value, the initial residual, and the first model parameter and the second model parameter corresponding to the spatial block of the point to be decoded; the second construction unit is configured to construct a prediction model according to the first model parameter and the second model parameter, and the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be decoded; the second prediction unit is configured to obtain the second color component prediction value of the point to be decoded using the prediction model and the first color component value; the inverse transformation unit is configured to perform RAHT inverse transformation on the parsed initial residual to obtain the residual of the point to be decoded; the second calculation unit is configured to calculate the second color component reconstruction value of the point to be decoded according to the residual of the point to be decoded and the second color component prediction value; in this way, one color component is used to predict another color component to obtain a prediction value, so that the attribute value is reconstructed according to the prediction value and the prediction residual obtained by parsing, and the decoding efficiency is improved because the correlation between the color components is removed.
[0311] It should be noted that, in this application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0312] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0313] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.
[0314] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.
[0315] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.
[0316] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
[0317] Industrial Applicability
[0318] In an embodiment of the present application, the method is applied to an encoder, by determining the spatial block where the point to be encoded is located, and the first color component value and the second color component value of the encoded point; based on the spatial block, a prediction model is constructed, and the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be encoded; then the prediction model and the first color component value are used to obtain the second color component prediction value of the point to be encoded; finally, the difference between the second color component value and the second color component prediction value is calculated, and the obtained difference is used as the residual of the point to be encoded, and the residual of the point to be encoded is subjected to RAHT transformation; in this way, one color component is used to predict another color component to obtain a predicted value, so that the predicted residual is calculated based on the actual value and the predicted value, which can remove the correlation between the color components, effectively reduce the encoding bit overhead, and improve the encoding efficiency; the method is also applied to a decoder, by determining the spatial block where the point to be decoded is located block; parse the attribute bit stream to obtain the first color component value of the point to be decoded, the initial residual and the first model parameter and the second model parameter corresponding to the spatial block; construct a prediction model based on the first model parameter and the second model parameter, and the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be decoded; then use the prediction model and the first color component value to obtain the second color component prediction value of the point to be decoded; finally, perform RAHT inverse transform on the parsed initial residual to obtain the residual of the point to be decoded; calculate the second color component reconstruction value of the point to be decoded based on the residual of the point to be decoded and the second color component prediction value; in this way, one color component is used to predict another color component to obtain a prediction value, so as to reconstruct the attribute value based on the prediction value and the prediction residual obtained by parsing, and since the correlation between the color components is removed, the coding bit overhead is effectively reduced, thereby improving the encoding and decoding efficiency.
Claims
1. A color component prediction method, applied to an encoder, the method include: Determine the spatial block where the to-be-coded point is located, and the first color component value and the second color component value of the coded point; Constructing a prediction model according to the spatial block; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be encoded; Obtaining a predicted value of a second color component of the point to be encoded by using the prediction model and the first color component value; The difference between the second color component value and the predicted value of the second color component is calculated, the obtained difference is used as the residual of the point to be encoded, and a regional adaptive hierarchical transform (RAHT) is performed on the residual of the point to be encoded.
2. The method according to claim 1, in, After performing RAHT transformation on the residual of the point to be encoded, the method further includes: The residual of the point to be encoded is encoded and written into an attribute bitstream.
3. The method according to claim 1, in, The determining of the spatial block where the point to be encoded is located includes: Perform spatial division on the point cloud to obtain at least one spatial block; Based on the geometric position of the point to be encoded, the spatial block where the point to be encoded is located is determined from the at least one spatial block.
4. The method according to claim 3, in, The step of spatially dividing the point cloud to obtain at least one spatial block includes: Dividing the point cloud into a plurality of point sets according to a spatial neighbor relationship between points in the point cloud, and determining a first correlation between a first color component and a second color component in each point set; The point cloud is spatially divided according to the determined first correlation to obtain the at least one spatial block.
5. The method according to claim 3, in, The step of spatially dividing the point cloud to obtain at least one spatial block includes: Calculating a Morton code for a point in the point cloud; Arrange the Morton codes of the points in the point cloud in a preset order, and determine the maximum value of the Morton code and the minimum value of the Morton code; Calculate the difference between the maximum value of the Morton code and the minimum value of the Morton code, and perform bit shifting N bits on the obtained difference in a preset direction to obtain the shifted difference; wherein N is a positive integer greater than or equal to 1; When the shifted difference satisfies a preset range, the point cloud is divided into space blocks equal to the shifted difference.
6. The method according to claim 5, in, After calculating the Morton code of the point in the point cloud, the method further comprises: Performing a bit shift of N bits on the Morton code of the midpoint of the point cloud in a preset direction; In the point cloud, if the Morton code values of some points after shifting are the same, it is determined that the some points belong to the same spatial block.
7. The method according to any one of claims 1 to 6, in, The constructing a prediction model according to the spatial block comprises: Selecting K points from the spatial block, and forming a point set from the selected K points; wherein K is a positive integer greater than or equal to 20; Determine model parameters based on K points in the point set; wherein the model parameters include a first model parameter and a second model parameter; The prediction model is constructed according to the first model parameters and the second model parameters.
8. The method according to claim 7, in, The determining of model parameters based on the K points in the point set includes: Obtain the first color component value and the second color component value of each point in the point set; Calculating the first model parameter using a first calculation sub-model according to the first color component value and the second color component value of each point in the point set; The second model parameters are calculated using a second calculation sub-model according to the first model parameters.
9. The method according to claim 7, in, After performing RAHT transformation on the residual of the point to be encoded, the method further includes: The first model parameter and the second model parameter are written into a property bitstream.
10. The method according to claim 9, in, The step of writing the first model parameter and the second model parameter into a property bitstream comprises: quantizing the first model parameter and the second model parameter to obtain a quantized first model parameter and a quantized second model parameter; The quantized first model parameters and the quantized second model parameters are written into the attribute bitstream.
11. The method according to claim 9, in, The step of writing the first model parameter and the second model parameter into a property bitstream comprises: Writing a first model parameter and a second model parameter corresponding to a first spatial block in the at least one spatial block into an attribute bitstream; Calculate the model parameter residual values of the remaining spatial blocks and the previous spatial block in the at least one spatial block, and write the model parameter residual values into the attribute bitstream in sequence; wherein the at least one spatial block includes the first spatial block and the remaining spatial blocks.
12. The method according to claim 1, in, The method further comprises: For the spatial block, calculating a second correlation between a first color component and a second color component corresponding to the spatial block; If the second correlation is less than a preset correlation threshold, the Morton code of each point in the space block is shifted right by N-1 bits to obtain two subspace blocks; Calculating a third correlation between the first color component and the second color component corresponding to the two subspace blocks; If the third correlation is greater than the second correlation, constructing two sub-prediction models according to the two subspace blocks; The second color component prediction value of the point to be encoded is obtained by using the two sub-prediction models and the first color component value.
13. The method according to claim 1, in, The shape of the space block includes at least one of the following: a cube shape, a rectangular parallelepiped shape and an irregular shape.
14. A color component prediction method, applied to a decoder, the method include: Determine the spatial block where the point to be decoded is located; Parsing the attribute bit stream to obtain the first color component value of the to-be-decoded point, the initial residual, and the first model parameter and the second model parameter corresponding to the spatial block; Constructing a prediction model according to the first model parameter and the second model parameter; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be decoded; Obtaining a second color component prediction value of the point to be decoded by using the prediction model and the first color component value; Performing an inverse RAHT transform on the resolved initial residual to obtain the residual of the point to be decoded; A second color component reconstruction value of the point to be decoded is calculated according to the residual of the point to be decoded and the predicted value of the second color component.
15. The method according to claim 14, in, The determining of the spatial block where the point to be decoded is located includes: Perform spatial division on the point cloud to obtain at least one spatial block; Based on the geometric position of the point to be decoded, the spatial block where the point to be decoded is located is determined from the at least one spatial block.
16. The method according to claim 15, in, The step of spatially dividing the point cloud to obtain at least one spatial block includes: Dividing the point cloud into a plurality of point sets according to a spatial neighbor relationship between points in the point cloud, and determining a first correlation between a first color component and a second color component in each point set; The point cloud is spatially divided according to the determined first correlation to obtain the at least one spatial block.
17. The method according to claim 15, in, The step of spatially dividing the point cloud to obtain at least one spatial block includes: Calculating a Morton code for a point in the point cloud; Arrange the Morton codes of the points in the point cloud in a preset order, and determine the maximum value of the Morton code and the minimum value of the Morton code; Calculate the difference between the maximum value of the Morton code and the minimum value of the Morton code, and perform bit shifting N bits on the obtained difference in a preset direction to obtain the shifted difference; wherein N is a positive integer greater than or equal to 1; When the shifted difference satisfies a preset range, the point cloud is divided into space blocks equal to the shifted difference.
18. The method according to claim 17, in, After calculating the Morton code of the point in the point cloud, the method further comprises: Performing a bit shift of N bits on the Morton code of the midpoint of the point cloud in a preset direction; In the point cloud, if the Morton code values of some points after shifting are the same, it is determined that the some points belong to the same spatial block.
19. The method according to claim 14, in, After parsing the attribute bitstream, the method further includes: The first model parameters and the second model parameters obtained by analysis are dequantized to obtain the first model parameters and the second model parameters.
20. The method according to claim 14, in, After parsing the attribute bitstream, the method further includes: Obtaining a first model parameter and a second model parameter corresponding to a first spatial block in at least one spatial block and residual values of model parameters corresponding to remaining spatial blocks in the at least one spatial block; The first model parameters and the second model parameters corresponding to the remaining spatial blocks are obtained according to the first model parameters and the second model parameters corresponding to the first spatial block and the model parameter residual values.
21. The method according to any one of claims 14 to 20, in, The shape of the space block includes at least one of the following: a cube shape, a rectangular parallelepiped shape and an irregular shape.
22. An encoder, comprising a first determining unit, a first constructing unit, a first predicting unit, a first calculating unit and a transforming unit, in, The first determining unit is configured to determine the space block where the to-be-encoded point is located, and the first color component value and the second color component value of the coded point; The first construction unit is configured to construct a prediction model according to the spatial block; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be encoded; The first prediction unit is configured to obtain a second color component prediction value of the to-be-encoded point by using the prediction model and the first color component value; The first calculation unit is configured to calculate a difference between the second color component value and the second color component prediction value, and use the obtained difference as the residual of the to-be-encoded point; The transform unit is configured to perform a RAHT transform on the residual of the point to be encoded.
23. A decoder, comprising a second determining unit, a parsing unit, a second constructing unit, a second predicting unit, an inverse transforming unit and a second calculating unit, in, The second determination unit is configured to determine the spatial block where the to-be-decoded point is located; The parsing unit is configured to parse the attribute bit stream to obtain the first color component value of the point to be decoded, the initial residual, and the first model parameter and the second model parameter corresponding to the spatial block; The second construction unit is configured to construct a prediction model according to the first model parameter and the second model parameter; wherein the prediction model is used to characterize the prediction relationship between the first color component and the second color component of the point to be decoded; The second prediction unit is configured to obtain a second color component prediction value of the to-be-decoded point by using the prediction model and the first color component value; The inverse transform unit is configured to perform a RAHT inverse transform on the parsed initial residual to obtain the residual of the point to be decoded; The second calculation unit is configured to calculate a second color component reconstruction value of the point to be decoded according to the residual of the point to be decoded and the second color component prediction value.
24. An encoder, comprising a first memory and a first processor, in, The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 1 to 13 when running the computer program.
25. A decoder comprising a second memory and a second processor, in, The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 14 to 21 when running the computer program.
26. A computer storage medium, in, The computer storage medium stores a computer program, which implements the method according to any one of claims 1 to 13 when executed by a first processor, or implements the method according to any one of claims 14 to 21 when executed by a second processor.
Citation Information
Patent Citations
Point cloud attribute compression method based on multi-angle adaptive intra-frame prediction
CN108632607A
A color attribute coding method based on TMC3 point cloud encoder
CN109257604A
Method and apparatus for adaptive transform in video encoding and decoding
EP3471418A1
Adaptive cross component residual prediction
US20160219283A1
Sorted geometry with color clustering (SGCC) for point cloud compression
US20190114504A1
