Decoding method, encoding method and decoding device
By switching entropy decoding and multivalued methods according to direction components in the encoding and decoding methods of three-dimensional data, the problems of low encoding efficiency and large data volume in the prior art are solved, and more efficient data processing is achieved.
Patent Information
- Application Number
- CN202380029929.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-19
- Filing Date
- 2023-03-13
- Publication Date
- 2024-11-08
AI Technical Summary
In the encoding and decoding processing of three-dimensional data, it is difficult for the prior art to effectively improve the encoding efficiency and reduce the amount of data processed by the decoding device, especially when processing large-scale point cloud data.
In the decoding method and encoding method, the entropy decoding and multivalued methods are dynamically switched to select suitable encoding and decoding methods based on whether the direction components in the position information of the three-dimensional point are equal, so as to improve the encoding efficiency and perform the following procedures if necessary Quantization and inverse quantization processing to reduce the amount of data.
The effect of improving coding efficiency and reducing the amount of decoded data is achieved, and the processing flow of three-dimensional data is optimized, which is suitable for the encoding and decoding devices of three-dimensional data.
Smart Images

Figure CN118922864A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a decoding method, an encoding method, a decoding device, and an encoding device. Background Art
[0002] Devices and services that effectively utilize three-dimensional data are expected to become more common in a wide range of fields, including computer vision for autonomous operation of cars or robots, map information, surveillance, infrastructure inspection, and image distribution. Three-dimensional data is obtained through various methods, such as distance sensors such as rangefinders, stereo cameras, or a combination of multiple monocular cameras.
[0003] As one of the methods of expressing three-dimensional data, there is a method called point cloud that expresses the shape of a three-dimensional structure through a group of points in a three-dimensional space. In a point cloud, the position and color of a point group are stored. Although point clouds are expected to become the mainstream method of expressing three-dimensional data, the amount of data in a point group is very large. Therefore, in the accumulation or transmission of three-dimensional data, like two-dimensional dynamic images (for example, MPEG-4AVC or HEVC standardized by MPEG), it is necessary to compress the data volume through encoding.
[0004] In addition, regarding compression of point clouds, some support is provided by a public library (Point Cloud Library) or the like that performs processing related to point clouds.
[0005] In addition, there is known a technique for searching and displaying facilities located around a vehicle using three-dimensional map data (see, for example, Patent Document 1).
[0006] Prior Art Literature
[0007] Patent Literature
[0008] Patent Document 1: International Publication No. 2014 / 020663 Summary of the invention
[0009] Problems to be solved by the invention
[0010] In the encoding and decoding of three-dimensional data, it is desirable to improve the encoding efficiency and reduce the amount of data to be processed by a decoding device.
[0011] An object of the present disclosure is to provide a decoding method, an encoding method, a decoding device, or an encoding device that can improve encoding efficiency or reduce the amount of data processed by a decoding device.
[0012] Means for solving problems
[0013] A decoding method involved in one embodiment of the present invention decodes multiple encoded three-dimensional points each having position information, the position information including a distance component, a first direction component, and a second direction component. In the decoding method, based on the number of second three-dimensional points among the decoded three-dimensional points having the first direction component and the second direction component that are substantially equal to the first direction component and the second direction component of the first encoded three-dimensional point included in the multiple encoded three-dimensional points, at least one method of entropy decoding method and multi-valued method of information related to the first encoded three-dimensional point is determined, and processing using the determined at least one method is performed.
[0014] A coding method involved in one embodiment of the present invention encodes multiple three-dimensional points each having position information, the position information including a distance component, a first direction component, and a second direction component. In the coding method, based on the number of second three-dimensional points among the encoded multiple three-dimensional points having the first direction component and the second direction component that are substantially equal to the first three-dimensional point included in the multiple three-dimensional points, at least one method of entropy coding method and binarization method for information related to the first three-dimensional point is determined, and processing using the determined at least one method is performed.
[0015] Effects of the Invention
[0016] The present disclosure can provide a decoding method, an encoding method, a decoding device, or an encoding device that can improve encoding efficiency or reduce the amount of data processed by a decoding device. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 This is a block diagram of a three-dimensional data encoding device according to an embodiment.
[0018] Figure 2 This is a block diagram of a three-dimensional data decoding device according to an embodiment.
[0019] Figure 3 It is a diagram showing an example of a point group involved in the embodiment.
[0020] Figure 4 This is a flowchart of an arithmetic coding process or an arithmetic decoding process according to an embodiment.
[0021] Figure 5 This is a diagram showing an example of the syntax of the node information of the three-dimensional point involved in the embodiment.
[0022] Figure 6 This is a flowchart of a modified example of the arithmetic coding process or the arithmetic decoding process involved in the embodiment.
[0023] Figure 7 This is a flowchart of a modified example of the arithmetic coding process or the arithmetic decoding process involved in the embodiment.
[0024] Figure 8 It is a diagram showing an example of a point group involved in the embodiment.
[0025] Fig. 9 This is a flowchart showing a modified example of the arithmetic coding process or the arithmetic decoding process involved in the embodiment.
[0026] Fig.10 This is a diagram showing the encoding order of a plurality of three-dimensional points according to the embodiment.
[0027] Fig.11 This is a diagram showing an example of the syntax of the position information involved in the implementation mode.
[0028] Fig.12 This is a diagram showing an example of the syntax of the position information involved in the implementation mode.
[0029] Fig.13 This is a diagram showing a reference range when selecting a context according to an embodiment.
[0030] Fig.14 This is a flowchart of a modified example of the arithmetic coding process or the arithmetic decoding process involved in the embodiment.
[0031] Fig.15 This is a flowchart of a three-dimensional data encoding method according to an embodiment of the present invention.
[0032] Fig.16 This is a flowchart of a three-dimensional data decoding method according to an embodiment. DETAILED DESCRIPTION
[0033] A decoding method involved in one embodiment of the present invention decodes multiple encoded three-dimensional points each having position information, the position information including a distance component, a first direction component, and a second direction component. In the decoding method, based on the number of second three-dimensional points among the decoded three-dimensional points having the first direction component and the second direction component that are substantially equal to the first direction component and the second direction component of the first encoded three-dimensional point included in the multiple encoded three-dimensional points, at least one method of entropy decoding method and multi-valued method of information related to the first encoded three-dimensional point is determined, and processing using the determined at least one method is performed.
[0034] Thus, in the decoding method, if the directional component is switched, the number of the second three-dimensional points that have been decoded changes, so at least one of the entropy decoding method and the multi-valued method is switched. In addition, the switching of the directional component is the switching of at least one of the first directional component and the second directional component. Thus, the decoding method can, for example, select at least one of the entropy decoding method and the multi-valued method that is suitable for the switching of the directional component. Thus, the decoding method can properly decode the bit stream with improved coding efficiency. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced. In addition, the directional components are substantially equal, for example, the difference between the directional components is below a threshold.
[0035] For example, in the determination, a context used for arithmetic decoding may be determined, and in the processing, arithmetic decoding may be performed on the information using the determined context.
[0036] Thus, the decoding method can switch the context used in arithmetic decoding in accordance with the switching of the directional component. Thus, the decoding method can, for example, select a context suitable for the switching of the directional component. Thus, the decoding method can properly decode the bit stream with improved coding efficiency. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced. In addition, entropy decoding is not limited to arithmetic decoding. For example, entropy decoding can also be Huffman decoding.
[0037] For example, the information may indicate which of the inter prediction mode and the intra prediction mode is used.
[0038] Here, at a plurality of points where the directional components (the first directional component and the second directional component) are substantially the same, there is a tendency that the intra-frame prediction mode is continuously selected. Thus, the decoding method can appropriately decode a bit stream with improved coding efficiency by switching at least one of the entropy decoding method and the multi-valued method in accordance with the switching of the directional components. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced.
[0039] For example, the information may represent a prediction residual of the first direction component or the second direction component.
[0040] Here, at a plurality of points where the directional components (the first directional component and the second directional component) are substantially the same, there is a tendency for the prediction residual to become smaller. Thus, the decoding method can appropriately decode a bit stream with improved coding efficiency by switching at least one of the entropy decoding method and the multi-valued method in accordance with the switching of the directional components. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced.
[0041] For example, the prediction residual may be information obtained by quantizing the prediction residual of the horizontal angle component.
[0042] Here, at a plurality of points where the directional components (the first directional component and the second directional component) are substantially the same, there is a tendency that the prediction residual of the horizontal angle component becomes smaller. Thus, the decoding method can appropriately decode a bit stream with improved coding efficiency by switching at least one of the entropy decoding method and the multi-valued method in accordance with the switching of the directional components. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced.
[0043] For example, the number may be the number of the second three-dimensional points having the quantized first direction component and the quantized second direction component substantially equal to the quantized first direction component and the quantized second direction component of the first encoded three-dimensional point.
[0044] For example, in the determination, the number may be limited by a predetermined upper limit value, and the at least one method may be determined based on the limited number.
[0045] Thus, for example, when the method is switched according to the number, the number of methods used can be reduced, and thus the amount of processing or the amount of memory used can be reduced.
[0046] For example, in the determination, the number of the second three-dimensional points may be quantized, and the at least one method may be determined based on the quantized number.
[0047] Thus, for example, when the method is switched according to the number, the number of methods used can be reduced, and thus the amount of processing or the amount of memory used can be reduced.
[0048] For example, in the determination, if the number is not within a predetermined range, the at least one method is determined based on the number, and if the number is within the predetermined range, the at least one method is determined based on historical information obtained based on the information related to the multiple three-dimensional points that have been decoded.
[0049] Thus, even when the switching of the method corresponding to the number is not effective, the decoding method can select an appropriate method using the history information. In addition, compared with the case where the method is switched for each combination of the number and the history information, the number of methods used can be reduced, thereby reducing the processing amount or the memory capacity used.
[0050] For example, the history information may be the number or pattern of 0 or 1 of the information of the plurality of three-dimensional points that have been decoded.
[0051] Therefore, the decoding method can select an appropriate method according to the number or pattern of 0 or 1 of the information of the plurality of three-dimensional points that have been decoded.
[0052] For example, the history information may be updated by the information of the three-dimensional points whose number is a predetermined value and not updated by the information of the three-dimensional points whose number is other than the predetermined value.
[0053] Thus, the decoding method can reflect information of points expected to be highly correlated in the history information, and thus can appropriately decode the bit stream with improved coding efficiency. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced.
[0054] For example, the information may indicate whether there is a three-dimensional point corresponding to the reference position.
[0055] Here, when the directional component is switched, the information indicating whether there is a three-dimensional point corresponding to the reference position tends to have a specific value. Thus, the decoding method can appropriately decode the bit stream with improved coding efficiency by switching at least one of the entropy decoding method and the multi-valued method in accordance with the switching of the directional component. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced.
[0056] A coding method involved in one embodiment of the present invention encodes multiple three-dimensional points each having position information, the position information including a distance component, a first direction component, and a second direction component. In the coding method, based on the number of second three-dimensional points among the encoded multiple three-dimensional points having the first direction component and the second direction component that are substantially equal to the first three-dimensional point included in the multiple three-dimensional points, at least one method of entropy coding method and binarization method for information related to the first three-dimensional point is determined, and processing using the determined at least one method is performed.
[0057] Thus, in the encoding method, if the directional component is switched, the number of the second three-dimensional points to be encoded changes, so at least one of the entropy encoding method and the binarization method is switched. In addition, the switching of the directional component is the switching of at least one of the first directional component and the second directional component. Thus, the encoding method can, for example, select at least one of the entropy encoding method and the binarization method suitable for the switching of the directional component, thereby improving the encoding efficiency.
[0058] A decoding device involved in one embodiment of the present disclosure decodes multiple encoded three-dimensional points each having position information, the position information including a distance component, a first direction component, and a second direction component. The decoding device includes a processor and a memory. The processor uses the memory to determine at least one of an entropy decoding method and a multi-valued method for information related to the first encoded three-dimensional point based on the number of second three-dimensional points among the decoded three-dimensional points having the first direction component and the second direction component that are substantially equal to the first encoded three-dimensional point included in the multiple encoded three-dimensional points, and performs processing using the determined at least one method.
[0059] In addition, these general or specific methods can also be implemented by systems, methods, integrated circuits, computer programs, or computer-readable recording media such as CD-ROMs, and can also be implemented by any combination of systems, methods, integrated circuits, computer programs, and recording media.
[0060] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. In addition, the embodiments described below all represent a specific example of the present disclosure. The numerical values, shapes, materials, constituent elements, configuration positions and connection methods of constituent elements, steps, and the order of steps shown in the following embodiments are examples and are not intended to limit the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims are described as arbitrary constituent elements.
[0061] (Implementation Method)
[0062] First, the configurations of a three-dimensional data encoding device and a three-dimensional data decoding device according to the present embodiment will be described. Figure 1 3D data encoding device 100 is a block diagram showing the structure of the 3D data encoding device 100. The 3D data encoding device 100 encodes point group (point cloud) data which is 3D data, thereby generating a bit stream (encoded stream).
[0063] A point group is a collection of multiple three-dimensional points that represents the three-dimensional shape of an object. Point group data includes position information and attribute information of multiple three-dimensional points. The position information represents the three-dimensional position of each three-dimensional point. In addition, position information is sometimes also called geometry information.
[0064] For example, the position information is represented by a polar coordinate system, including one distance component and two direction components (angle components). Specifically, the position information includes the distance d, the elevation angle θ, and the horizontal angle Φ. Point group data is, for example, data obtained by a laser sensor such as LiDAR. In addition, the position information can also be represented by an orthogonal coordinate system (x, y, z).
[0065] The attribute information indicates, for example, color, reflectance, or normal vector, etc. One three-dimensional point may have one piece of attribute information or may have a plurality of pieces of attribute information.
[0066] In addition, three-dimensional data is not limited to point group data, and can also be other three-dimensional data such as mesh data. Mesh data (also called three-dimensional mesh data) is a data format used in CG (Computer Graphics), which represents the three-dimensional shape of an object by a collection of surface information. For example, mesh data includes point group information (such as vertex information). Therefore, the same method as the method corresponding to point group data can be applied to the point group information.
[0067] In addition, Figure 1 , the processing unit related to the encoding of the position information of the point cloud data is described, but the three-dimensional data encoding device 100 may also include other processing units such as a processing unit that encodes attribute information.
[0068] The three-dimensional data encoding device 100 includes a subtraction unit 102, a quantization unit 103, an entropy encoding unit 104, an inverse quantization unit 105, an addition unit 106, a buffer 108, an intra-frame prediction unit 109, a buffer 110, a motion detection and compensation unit 111, an inter-frame prediction unit 112 and a switching unit 113.
[0069] The subtraction unit 102 subtracts the prediction value from the position information included in the input point group data of the encoding object, thereby generating a prediction residual. The quantization unit 103 quantizes the prediction residual. The entropy coding unit 104 performs entropy coding on the quantized prediction residual to generate a bit stream. In addition, the entropy coding unit 104 performs entropy coding on the control information and adds the coded information to the bit stream.
[0070] The inverse quantization unit 105 inversely quantizes the quantized prediction residual obtained by the quantization unit 103 to generate a prediction residual. The addition unit 106 adds the prediction value to the prediction residual generated by the inverse quantization unit 105 to restore the position information. The buffer 108 stores the restored position information as a reference point group for intra-frame prediction. The buffer 110 stores the restored position information as a reference point group for inter-frame prediction.
[0071] In addition, the restored position information contains quantization errors, so it is sometimes not completely consistent with the original position information. In addition, the three-dimensional point restored by encoding and decoding processing in this way is called an encoded three-dimensional point, a decoded three-dimensional point, or a processed three-dimensional point.
[0072] The intra prediction unit 109 calculates the prediction value using the position information of one or more reference points, which are other processed three-dimensional points belonging to the same frame as the three-dimensional point of the processing object (hereinafter referred to as the object point). For example, the intra prediction unit 109 uses a prediction tree to perform intra prediction. The prediction tree is a tree structure that represents the reference relationship in the prediction process. For example, in the prediction process of the object node (object point), the position information of the parent node is referenced. In addition, the position information of multiple nodes (such as a grandparent node and a great-grandparent node) including the parent node can also be referenced in the prediction process.
[0073] The motion detection and compensation unit 111 detects displacement between a target frame, which is a frame including the target point, and a reference frame, which is a frame different from the target frame (performs motion detection), and corrects position information of a point group included in the reference frame based on the detected displacement (motion compensation). Information indicating the detected displacement (motion information) is stored in a bit stream, for example.
[0074] The inter-frame prediction unit 112 calculates a prediction value using position information of one or more reference points included in the point group after motion compensation. In addition, motion detection and motion compensation may not be performed.
[0075] The switching unit 113 selects one of the prediction value calculated by the intra-frame prediction unit 109 and the prediction value obtained by the inter-frame prediction unit 112, and outputs the selected prediction value to the subtraction unit 102 and the addition unit 106. That is, the switching unit 113 switches whether to use intra-frame prediction or inter-frame prediction. For example, in this switching, the cost (amount of code) when using intra-frame prediction can be compared with the cost when using inter-frame prediction, and the method with lower cost can be used, or it can be decided based on instructions from the outside, or it can be decided based on the point group or the information attached to the point group. In addition, information indicating whether to use intra-frame prediction or inter-frame prediction is stored in the bit stream.
[0076] Next, the configuration of a three-dimensional data decoding device 200 that decodes a bit stream generated by the three-dimensional data encoding device 100 will be described. Figure 2 is a block diagram of a three-dimensional data decoding device 200 according to this embodiment. Figure 2 , the processing unit related to decoding the position information of the point group is described, but the three-dimensional data decoding device 200 may also include other processing units such as a processing unit for decoding the attribute information of the point group. Figure 1 The bit stream generated by the three-dimensional data encoding device 100 shown is decoded to generate decoded point group data.
[0077] The three-dimensional data decoding device 200 includes an entropy decoding unit 201 , an inverse quantization unit 202 , an addition unit 203 , a buffer 205 , an intra prediction unit 206 , a buffer 207 , a motion compensation unit 208 , an inter prediction unit 209 , and a switching unit 210 .
[0078] The three-dimensional data decoding device 200 acquires the bit stream generated by the three-dimensional data encoding device 100 .
[0079] The entropy decoding unit 201 performs entropy decoding on the bit stream to generate quantized prediction residuals, control information, and the like.
[0080] The inverse quantization unit 202 generates a prediction residual by inverse quantizing the quantized prediction residual obtained by the entropy decoding unit 201. The adding unit 203 restores the position information by adding the prediction value to the prediction residual generated by the inverse quantization unit 202. The position information is output as decoded point group data.
[0081] The buffer 205 stores the restored position information as a reference point group for intra prediction. The buffer 207 stores the restored position information as a reference point group for inter prediction. The intra prediction unit 206 calculates a prediction value using the position information of one or more reference points, which are other three-dimensional points belonging to the same frame as the object point. For example, the intra prediction unit 206 performs intra prediction using a prediction tree.
[0082] The motion compensation unit 208 obtains motion information indicating the displacement between the target frame and the reference frame from the bit stream, and corrects (motion compensates) the position information of the point group included in the reference frame based on the displacement indicated by the motion information. The inter-frame prediction unit 209 calculates the prediction value using the position information of one or more reference points included in the point group after motion compensation. In addition, motion compensation may not be performed.
[0083] The switching unit 210 selects one of the prediction value calculated by the intra prediction unit 206 and the prediction value obtained by the inter prediction unit 209, and outputs the selected prediction value to the adding unit 203. For example, the switching is performed based on information indicating whether intra prediction or inter prediction is used, which is included in the bitstream.
[0084] Next, a method of performing entropy coding on each point of the prediction tree is described. Figure 3 is a diagram showing an example of a point group obtained by LiDAR, which is a top view of the point group. In this example, the position of each point is represented by polar coordinates, that is, a distance component and two direction components. In addition, Figure 3 This is a top view, so the elevation angle is not shown.
[0085] Figure 3The point group shown is an example of a point group obtained based on an operation mode such as multi-return, which detects laser pulses reflected from multiple object surfaces and processes each of them as a point. In multi-return, for example, points located inside a semi-transparent point (object) in the same angular direction (e.g. horizontal angle) can also be detected. There are multiple points with different distances in each angular direction in . In addition, Figure 3 The elevation angles of the multiple points shown are, for example, the same.
[0086] The dotted arrows in the figure are an example of the order in which the multiple points are encoded or decoded. Multiple points with the same angle are encoded in ascending (or descending) order according to the distance. As a result, the correlation between the residual information during intra-frame prediction becomes higher, and the efficiency of entropy coding (such as arithmetic coding, etc.) of the residual information is improved.
[0087] In addition, the example in which 4 to 6 points exist in the same angular direction is shown here, but it is sufficient to have 2 or more points in the same angle. In addition, it is not necessary to have multiple points in each of all angular directions, as long as there are 2 or more points in at least one angular direction.
[0088] In addition, the above example illustrates a point group obtained by multiple returns as an example of multiple points with different distances in the same angular direction, but by quantizing the direction components (horizontal angle, elevation angle), multiple points with the same angular component may sometimes be generated. The method of this embodiment can also be applied to such a situation.
[0089] In the encoding of multiple points in the same angular direction, the same first prediction method (e.g., intra-frame prediction) is continuously selected. As a result, for example, there is a tendency that intra_pred_flag is continuously set to the first value (e.g., true (value 1)). Here, intra_pred_flag indicates whether the prediction method applied to the encoding or decoding of the object point is intra-frame prediction (i.e., intra-frame prediction or inter-frame prediction). For example, a value of 1 indicates intra-frame prediction, and a value of 0 indicates inter-frame prediction.
[0090] By continuously selecting the first prediction method in this way, the context used for arithmetic coding is updated to a state suitable for the first prediction method. Therefore, at points A to C, which are points just after the encoding process is transferred to the next angle direction (that is, the first point encoded in each angle direction), even if the second prediction method (for example, inter-frame prediction) different from the first prediction method can reduce residual information more, the amount of code cannot be sufficiently reduced when intra_pred_flag indicates the second prediction method. As a result, there is a problem that the encoding efficiency cannot be fully improved, such as when the first prediction method is easily selected in the encoding device.
[0091] To address this problem, a three-dimensional data encoding device or a three-dimensional data decoding device uses Figure 3 In this way, the context used when arithmetic coding or decoding is performed on the syntax element corresponding to each point is selected according to the count value assigned to each point according to the number of points located in the same angular direction. Here, the syntax element is, for example, intra_pred_flag or quantized_1st_residual_value[j]. quantized_1st_residual_value[j] (j is 0 to 2) represents the quantized value (quantized prediction residual) of the prediction residual of the directional component among the prediction residuals (difference information) between the coordinate value of the object point to be coded or decoded and the coordinate value (prediction value) of the prediction point.
[0092] For example, the three-dimensional data encoding apparatus quantizes the prediction residual of the horizontal angle component and the prediction residual of the elevation angle component using the quantization step value, thereby generating quantized_1st_residual_value[j] (j is 0 or 1).
[0093] exist Figure 3 In the example shown, the count value increases by 1 each time a point is encoded or decoded. In addition, the count value is reset to 0 after encoding or decoding of a point just after the encoding process is transferred to the next angular direction, such as points A to C. That is, in this example, a different context is selected each time a point is encoded or decoded, and after encoding or decoding of a point just after the encoding process is transferred to the next angular direction, the context is reset to a count value of 0. In other words, in this example, a different context is selected according to the number of points that have been encoded or decoded at a certain horizontal angle, and when the number of points that have been encoded or decoded in each angular direction is 1, the context is reset to a count value of 0.
[0094] Thus, it is possible to set appropriate contexts corresponding to the prediction residuals of each of a plurality of points having the same horizontal angle and elevation angle and at different distances. For example, at a point with a small count value, the context is updated to a state suitable for the first prediction method with a high selection frequency. At a point with a large count value, the context is updated to a state suitable for the second prediction method, or a state that is neutral to both the first prediction method and the second prediction method. Thus, it is possible to maintain the coding efficiency when encoding points in the same angular direction in the first prediction method, and to improve the coding efficiency of the points initially encoded or decoded in the angular direction. Thus, there is a possibility of improving the coding efficiency of the entire bitstream.
[0095] As such, the smaller the count value, the more dominant the context suitable for the state of the first prediction method becomes, and the larger the count value, the more dominant the context suitable for the state of the second prediction method becomes. In addition, the context for each point is switched so that the context suitable for the state of the first prediction method becomes dominant after the encoding or decoding of the point just after the encoding process is transferred to the next angle direction.
[0096] In addition, the context switching is to select the context to be used from a plurality of contexts. In addition, each context can be either a context with an updated probability or a context with a fixed probability. In the context with an updated probability, the probability is updated according to the value (0, 1) of the signal after arithmetic coding, and the updated probability is used in the subsequent arithmetic coding process using the same context.
[0097] In addition, the context suitable for a state of a certain prediction method is a context that tends to reduce the amount of code when the method is used compared to when the method is not used. Specifically, in the case of a context whose usage probability is updated, the probability is automatically updated as the encoding or decoding process proceeds. Thus, if there is a tendency that a certain method is used more, the context is updated to a state suitable for the method. In addition, the initial value of the probability can be either a predetermined value (for example, the probability of occurrence of 0 and 1 is the same) or a state suitable for the method. In other words, multiple contexts may also have initial values of probability that are different depending on the corresponding count values. For example, the initial value of the context corresponding to the smaller count value may be a state that is more suitable for the first prediction method, and the initial value of the context corresponding to the larger count value may be a state that is more suitable for the second prediction method.
[0098] In addition, when using a context with a fixed probability, for example, a context corresponding to a smaller count value is a state more suitable for the first prediction method, and a context corresponding to a larger count value is a state more suitable for the second prediction method.
[0099] In addition, Figure 3 In the example shown, the count value of the point that is first encoded or decoded in a certain angular direction depends on the number of points in the immediately preceding angular direction, and the count value of the point that is encoded or decoded after the second in the angular direction has a count value that depends on the number of points in the angular direction that the point is. In view of the above-mentioned issues, the count value of the point that is first encoded or decoded in a certain angular direction also preferably has a count value that depends on the number of points in the angular direction that the point is. However, in the stage where the point is to be decoded in the three-dimensional data decoding device (that is, the stage before the point is decoded), it is impossible to determine the angular direction of the point, nor can it be determined whether the point is the first point in the angular direction. For these reasons, in Figure 3In the example shown, the count value at the second point in the angular direction is set to 0.
[0100] Figure 4 It means use Figure 3 The flowchart of an example of a process of arithmetic encoding or decoding of a syntax element corresponding to an object point to be encoded or decoded according to the count value described in . In addition, the operation of the three-dimensional data encoding device is mainly described here, but the operation of the three-dimensional data decoding device is the same. The operation of the three-dimensional data decoding device can be replaced by decoding as described below.
[0101] In this example, first, the three-dimensional data encoding device sets the count value to an initial value of 0 (S101). Next, the three-dimensional data encoding device starts loop processing of each point included in the prediction tree of the processing object (encoding object or decoding object) (S102).
[0102] First, the three-dimensional data encoding device determines a context according to the count value, and uses the determined context to perform arithmetic coding on the syntactic element (S103). For example, the context may be set for each value of the count value, and the three-dimensional data encoding device selects a context corresponding to the count value from a plurality of contexts.
[0103] In addition, the three-dimensional data encoding device may also quantize the count value and select a context based on the quantized count value. That is, a context may be set for each range of the count value, and the three-dimensional data encoding device selects a context corresponding to the range including the count value from a plurality of contexts. Thus, the number of contexts used can be reduced, and thus the capacity of the memory used to store the context can be suppressed.
[0104] Furthermore, when encoding a plurality of syntax elements for an object point, the three-dimensional data encoding device performs the above-described context determination and arithmetic encoding for each syntax element.
[0105] Next, the three-dimensional data encoding device derives the decoded coordinates of the object point based on the syntax elements that have been arithmetically encoded in step S103 (S104). In addition, when there is a difference between the coordinates before encoding and the decoded coordinates, the three-dimensional data encoding device derives the decoded coordinates of the object point by decoding the encoded information. Thus, the same coordinates (decoded coordinates) can be used in the three-dimensional data encoding device and the three-dimensional data decoding device.
[0106] On the other hand, in the case of encoding and decoding processing in which there is no difference between the coordinates before encoding and the decoded coordinates (lossless encoding and decoding), the three-dimensional data encoding device may not implement the processing of step S104, but refer to the coordinates before encoding instead of the decoded coordinates in subsequent processing.
[0107] Furthermore, in the three-dimensional data decoding device, the syntax elements are decoded in step S103, and in step S104, the decoded coordinates of the object point are derived based on the decoded syntax elements.
[0108] Next, the three-dimensional data encoding device determines whether the object point is the first point to be encoded or decoded in each angular direction. Specifically, the three-dimensional data encoding device determines whether the object point has a parent node in the prediction tree (S105). In the case where the object point has a parent node (S105: Yes), the three-dimensional data encoding device determines whether the difference between the directional component (horizontal angle and elevation angle) of the decoded coordinates of the object point and the directional component of the parent node is less than a threshold value (S106).
[0109] For example, the three-dimensional data encoding device determines that the difference between the directional component of the decoded coordinates of the object point and the directional component of the parent node is less than the threshold value when (1) the first difference between the horizontal angle of the decoded coordinates of the object point and the horizontal angle of the parent node is less than the first threshold value, and (2) the second difference between the elevation angle of the decoded coordinates of the object point and the elevation angle of the parent node is less than the second threshold value. In other cases, the three-dimensional data encoding device determines that the difference between the directional component of the decoded coordinates of the object point and the directional component of the parent node is greater than the threshold value. In addition, the first threshold value and the second threshold value can be the same value or different values. In addition, the three-dimensional data encoding device can also compare the calculated value calculated based on the first difference and the second difference, such as the addition value, average value or weighted addition value of the first difference and the second difference, with a threshold value.
[0110] When the object point does not have a parent node (S105: No), or when the difference between the directional component of the decoded coordinates and the directional component of the parent node is greater than the threshold (S106: No), the three-dimensional data encoding device determines that the object point is the point initially encoded in the angular direction and resets the count value to 0 (S107).
[0111] On the other hand, when the difference between the directional component of the decoded coordinates and the directional component of the parent node is less than the threshold value (S106: Yes), the three-dimensional data encoding device determines that the object point is the second or later point to be encoded in the angular direction, and adds a predetermined value to the count value (S108). In addition, the three-dimensional data encoding device ends the loop processing for each point (S109). That is, by repeatedly performing the processing of steps S103 to S108 for each point, arithmetic coding or arithmetic decoding is performed on a plurality of points included in the prediction tree.
[0112] In addition, the threshold used in step S106 may be set based on the sampling interval or resolution of each of the plurality of directional components of the sensor. For example, the threshold may be set to a value approximately half of the sampling interval or resolution.
[0113] In addition, the prescribed value in step S108 may be "1" or "the number of repeated points + 1" obtained by adding the number of repeated points (points whose coordinates are the same as the object point and only the color or reflectance is encoded or decoded) generated by the downward transformation of the input point group. In addition, an upper limit value may be set for the count value, and when the count value exceeds the upper limit value after adding the prescribed value, the count value is limited by the upper limit value. In this way, the number of contexts used can be reduced, and the capacity of the memory used to store the contexts can be suppressed.
[0114] Thus, the context used in the point with a small count value is updated to a state suitable for the first prediction method with a high selection frequency. The context of the point with a large count value is updated to a state suitable for the second prediction method, or a state neutral to both the first prediction method and the second prediction method. Thus, the coding efficiency when encoding multiple points in the same angular direction in the first prediction method can be maintained, and the coding efficiency of the point that is first encoded or decoded in the angular direction can be improved. Thus, there is a possibility that the coding efficiency of the entire bit stream can be improved.
[0115] Figure 5 : is a diagram showing an example of the syntax of the node information (geometry_prediction_tree_node) of the three-dimensional point according to the embodiment. This syntax is an example of the information of each node in the prediction tree.
[0116] InterFrameFlag indicates whether inter-frame prediction can be used. InterFrameFlag is set according to the syntax of the upper level (superior) (SPS, GPS or slice header, etc.). In addition, SPS (Sequence Parameter Set) is a parameter set (control information) in units of a sequence including multiple frames. In addition, SPS is a parameter set common to position information and attribute information. In addition, GPS (Geometry Parameter Set) is a parameter set in units of frames, and is a parameter set for position information.
[0117] intra_pred_flag indicates whether the prediction method applied in encoding or decoding of the target point is intra prediction (ie, intra prediction or inter prediction). For example, a value of 1 indicates intra prediction, and a value of 0 indicates inter prediction.
[0118] For example, when InterFrameFlag indicates that inter-frame prediction can be used, intra_pred_flag is included in the node information, and when InterFrameFlag indicates that inter-frame prediction cannot be used (inter-frame prediction is invalid), intra_pred_flag is not included in the node information.
[0119] When intra prediction is applied (intra_pred_flag=1), the node information includes pred_mode. pred_mode indicates the prediction mode applied in encoding or decoding of the object point. The prediction mode is information indicating the method of calculating the intra prediction point applied to the object point. For example, the prediction mode indicates the method of calculating the prediction point based on the position of one or more upper (superior) nodes of the object point in the prediction tree.
[0120] When inter prediction is applied (intra_pred_flag=0), the node information includes one or more inter_ref_frame_idx and one or more inter_ref_point_idx.
[0121] NumRefFrames indicates the number of frames (point groups) referenced in inter-frame prediction, and is set according to the upper (superior) syntax (SPS, GPS, or slice header, etc.).
[0122] When inter prediction is applied (intra_pred_flag=0) and NumRefFrames is greater than 1, inter_ref_frame_idx is included in the node information. inter_ref_frame_idx indicates a frame referenced in inter prediction of the target point.
[0123] NumRefPoints indicates the number of prediction points referenced in inter prediction. When inter prediction is applied (intra_pred_flag=0) and NumRefPoints is greater than 1, inter_ref_point_idx is included in the node information. inter_ref_point_idx indicates a prediction point referenced in inter prediction of a target point.
[0124] gps_alt_coordinates_flag indicates whether the encoding or decoding process of the point group is performed in orthogonal coordinates (value 0) or in other than orthogonal coordinates (eg polar coordinates) (value 1). This gps_alt_coordinates_flag is added to the bit stream.
[0125] When gps_alt_coordinates_flag indicates the use of polar coordinates (for example, when gps_alt_coordinates_flag=1), the node information includes quantized_1st_residual_value[j].
[0126] quantized_1st_residual_value[j] (j is 0 to 2) represents a quantized value (quantized prediction residual) of a directional component of the prediction residual (difference information) between the coordinate value of a target point to be encoded / decoded and the coordinate value (prediction value) of a prediction point.
[0127] For example, the three-dimensional data encoding device quantizes the prediction residual of the horizontal angle component and the prediction residual of the elevation angle component using the quantization step value, thereby generating quantized_1st_residual_value[j] (j is 0 or 1). In addition, information indicating the quantization step value is stored in, for example, the syntax of the upper level (superior) of the bit stream (SPS, GPS or slice header, etc.).
[0128] In addition, the three-dimensional data encoding device may store not only quantized_1st_residual_value[j] but also a residual component which is the difference between the prediction residual before quantization and the prediction residual after quantization (quantized value) in the bitstream. For example, the residual component may be stored in the bitstream as 1st_residual_value[i]. Specifically, the residual components of the horizontal angle and the elevation angle may be stored in the bitstream as the 1st_residual_value of the horizontal angle and the elevation angle, respectively.
[0129] Thus, when encoding point cloud data obtained by a sensor whose direction changes at a certain speed, the three-dimensional data encoding device may be able to suppress the code amount of the prediction residual of the directional component by setting a quantization step value corresponding to the speed. In addition, when encoding point cloud data generated by a sensor rotating in one direction and obtaining the three-dimensional position of surrounding objects, such as a laser sensor that performs rotational scanning such as LiDAR, the three-dimensional data device may store the quantized value and the residual component in the bit stream for one component (e.g., the horizontal angle) in the same direction as the rotation direction of the sensor, and store only the quantized value or only the prediction residual (1st_residual_value) before quantization in the bit stream for other directional components (e.g., the elevation angle) and the distance component.
[0130] 1st_residual_value[i] represents the prediction residual of each component (horizontal angle, elevation angle, distance, or x, y, z) of the position information of the target point. When the residual component is stored in the bitstream as described above, 1st_residual_value[i] represents the residual component of the corresponding component.
[0131] gps_coordinate_trans_enabled_flag indicates whether coordinate transformation of position coordinates is performed before encoding or after decoding. When coordinate transformation is performed (gps_coordinate_trans_enabled_flag = 1), the 3D data encoding device transforms the input position information of the orthogonal coordinate system into the position information of the polar coordinate system and then performs encoding. In addition, the 3D data decoding device transforms the decoded position information of the polar coordinate system into the position information of the orthogonal coordinate system and then outputs it.
[0132] When coordinate transformation is performed (gps_coordinate_trans_enabled_flag=1), the node information includes 2nd_residual_value[i]. 2nd_residual_value[i] represents the difference between the position information of the orthogonal coordinate system generated by the coordinate transformation and the position information of the polar coordinate system. The three-dimensional data decoding device transforms the decoded position information of the polar coordinate system into the position information of the orthogonal coordinate system, and adds the difference to the transformed position information of the orthogonal coordinate system, thereby restoring the position information of the original orthogonal coordinate system.
[0133] For example, by Figure 4 The processing shown in the figure selects the syntactic element of the context according to the count value. Figure 5 For example, the three-dimensional data encoding device may also select at least one of the plurality of syntactic elements shown. Figure 5 The context of intra_pred_flag and quantized_1st_residual_value[j] shown in FIG. 1 is a context of intra_pred_flag and quantized_1st_residual_value[j] shown in FIG. 1 . In addition, the three-dimensional data encoding device may not select syntactic elements other than intra_pred_flag and quantized_1st_residual_value[j] in accordance with the count value, for example, a common context is used regardless of the count value. In this way, the number of contexts used can be reduced, and thus the capacity of the memory used to store the contexts can be suppressed.
[0134] In addition, Figure 4In the example shown, whether the object point is the first point encoded in the angular direction is determined by the determination in steps S105 and S106, but any method other than this method may be used as long as it can determine whether the object point is the first point encoded in the angular direction.
[0135] Figure 6 This is a flowchart showing a modified example of the process of performing arithmetic coding or arithmetic decoding on a syntax element corresponding to an object point. Figure 6 The processing shown is similar to Figure 4 The processing shown is different in that S105A is included instead of steps S105 and S106.
[0136] For example, in a case where the prediction tree is constructed to branch at the first node in the angular direction and the last node in the angular direction becomes a leaf node, the three-dimensional data encoding device replaces steps S105 and S106 and determines whether the node encoded immediately before is a leaf node (S105A) to determine whether the object point is the first point encoded in the angular direction. The three-dimensional data encoding device implements step S107 when the node (point) encoded immediately before is a leaf node (S105A: Yes), and implements step S108 when the node encoded immediately before is not a leaf node (S105A: No). In this case, the three-dimensional data encoding device can also be set to the same Figure 3 The examples shown have the same count values.
[0137] In addition, both the three-dimensional data encoding device and the three-dimensional data decoding device can determine whether the immediately preceding node is a leaf node before deriving the decoded coordinates of the object point. Therefore, the three-dimensional data encoding device can also determine whether the immediately preceding node is a leaf node before implementing step S103, and implement step S107 if the immediately preceding node is a leaf node, and implement step S108 if the immediately preceding node is not a leaf node. In this case, the three-dimensional data encoding device can convert the point that was originally encoded in the angular direction (for example Figure 3 The count values of points A, B, and C shown are set to 0.
[0138] Furthermore, the three-dimensional data encoding device may determine the context based on whether the object point is the first point encoded in the angular direction (that is, whether the immediately previously encoded node is a leaf node) without setting a count value before implementing step S103.
[0139] Figure 7 This is a flowchart showing a modified example of the process of performing arithmetic coding or arithmetic decoding on a syntax element corresponding to an object point.
[0140] The three-dimensional data encoding device starts a loop processing of each point for a plurality of points included in the prediction tree of the processing object (S111). Next, the three-dimensional data encoding device determines whether the node encoded immediately before is a leaf node (S112). In the case where the node encoded immediately before is a leaf node (S112: Yes), the three-dimensional data encoding device selects the second context (S113) assumed to be selected for the second prediction method (e.g., inter-frame prediction). On the other hand, in the case where the node encoded immediately before is not a leaf node (S112: No), the three-dimensional data encoding device selects the first context (S114) assumed to be selected for the first prediction method (e.g., intra-frame prediction).
[0141] Next, the three-dimensional data encoding device performs arithmetic coding on the syntactic element using the context selected in step S113 or S114 (S115). In addition, when encoding multiple syntactic elements, the three-dimensional data encoding device performs the above-mentioned context determination and arithmetic coding for each syntactic element. In addition, the three-dimensional data encoding device ends the loop processing for each point (S116).
[0142] In addition, Figure 4 In the example shown, the three-dimensional data encoding device performs determination based on the information of the parent node in steps S105 and S106 , but information of a node encoded immediately before the target point may be used instead of the parent node.
[0143] In addition, in step S106, the three-dimensional data encoding device uses both the horizontal angle and the elevation angle of the decoded coordinates to make a determination, but it is also possible to make a determination using only one of the horizontal angle and the elevation angle. For example, the three-dimensional data encoding device may make a determination using only the horizontal angle when scanning and encoding points sequentially in the horizontal direction. In addition, the three-dimensional data encoding device may make a determination using only the elevation angle when scanning and encoding points sequentially in the vertical direction. In addition, the three-dimensional data encoding device may make a determination using either the quantized horizontal angle or the quantized elevation angle.
[0144] In addition, it is not limited to always having to perform Figure 4 All the processing described above may be performed, or only a part of the processing may be performed.
[0145] In addition, Figure 3 In FIG. 1 , an example is shown in which a plurality of points have the same elevation angle but different distances at each horizontal angle. However, the same processing can be performed in the case in which a plurality of points have the same horizontal angle but different distances at each elevation angle. In this case, the encoding or decoding order is, for example, the order from the smallest distance to the largest distance at each elevation angle.
[0146] In addition, Figure 3In the example, the encoding or decoding order is the order from small to large distance at each horizontal angle, but it can also be the order from large to small distance.
[0147] In the above example, the two direction components of each point are the horizontal angle component and the elevation angle component, but the two directions are not limited to these. The two directions only need to be orthogonal to each other.
[0148] In addition, the counter value may be updated by a method other than the method shown in the above example. For example, in the above example, the counter value is updated each time one point is processed (encoded or decoded), but it may be changed each time multiple points are processed. Figure 4 Although step S106 is executed every time the determination result of step S105 is "yes", step S106 is executed every time the number of times the determination result of step S105 is "yes" reaches a predetermined number of times.
[0149] Furthermore, the three-dimensional data encoding device may also switch contexts without using the count value. Figure 4 If the determination result of step S105 is No or step S106 is No, the first context is used, and if the determination result of step S106 is Yes, the second context is used.
[0150] In the above example, the predetermined value added to the count value is set to the number of repeated points + 1, but the predetermined value is not limited thereto. For example, a value greater than the number of repeated points + 1 may be used as the predetermined value to facilitate context switching after encoding or decoding repeated points.
[0151] In addition, in the above description, Figure 3 The case where a plurality of points having different distances in the angular direction are present is described as an example, but the above-described processing can also be applied to cases other than this case, including a case where only one point exists in the angular direction.
[0152] Figure 8 This is a diagram showing an example of a point group acquired by LiDAR. Figure 8 and Figure 3 This is also a top view of a point group. In this example, the position of each point is represented by polar coordinates, i.e., a distance component and two direction components. In addition, Figure 6 This is a top view, so the elevation angle is not shown.
[0153] This example and Figure 3 Different from the example, in each angular direction (such as horizontal angle In other words, the point group is an example of a point group obtained by a single return. A single return is an action mode in which only one point is detected in one angular direction. In addition, Figure 8 The elevation angles of the plurality of points shown are the same. The dashed arrows in the figure are an example of the order in which the plurality of points are encoded or decoded.
[0154] If Figure 3 The process shown in the flowchart is applicable to Figure 8 As for the point group shown, since the difference between the direction component of the decoded coordinates and the direction component of the parent node is larger than the threshold value (S106: No), count value = 0 is given to point A, point B, and point C.
[0155] about Figure 8 The multiple points in the adjacent angular directions shown are assumed to be multiple points on the surface of the same object, and there are many cases where the correlation is high. Therefore, the prediction mode used in the points in the adjacent angular directions (for example, Figure 5 A certain correlation is also observed between the intra_pred_flag shown in the figure, etc.). Using this feature, the three-dimensional data encoding device may also maintain the values of one or more intra_pred_flags that have been encoded as history information, and use the history information to select the context of arithmetic coding of intra_pred_flag. Here, the history information is, for example, the number of points whose intra_pred_flag is 0 or 1 among a predetermined number of points that were encoded immediately before. Alternatively, the history information may be a pattern in which the intra_pred_flag of a predetermined number of points that were encoded immediately before is 0 or 1. Here, the predetermined number is not particularly limited, and may be, for example, 3. In addition, the point to which the history is referenced only needs to be a point that has been processed (encoded or decoded), and may not be a point that was encoded immediately before.
[0156] Alternatively, this method can be combined with Figure 4 This combination of processing may improve the coding efficiency regardless of whether the point group to be processed is acquired in the multi-return mode or the single-return mode.
[0157] In the above combination, the three-dimensional data encoding device may determine the context using the history information only when the count value is a specified value such as 0, and determine the context based on the count value without using the history information when the count value is not the specified value.
[0158] Fig. 9 This is a flowchart showing an example of a process of performing arithmetic coding or arithmetic decoding on a syntax element corresponding to an object point in this case. Fig. 9 The treatment shown is relative to Figure 4The processing shown adds steps S121 and S122. In step 121, the three-dimensional data encoding device determines whether the count value of the object point is 0. When the count value of the object point is 0 (S121: Yes), the three-dimensional data encoding device determines a context based on the historical information, and uses the determined context to perform arithmetic coding on the syntax (S122). For example, it is also possible to set a context for each value of the historical information, and the three-dimensional data encoding device selects a context corresponding to the value of the historical information from multiple contexts. In addition, when encoding multiple syntactic elements, the three-dimensional data encoding device may also determine the context for each syntactic element. That is, the three-dimensional data encoding device may also maintain the historical information of each of the multiple syntactic elements, and use the historical information of the syntactic element of the processing object to determine the context of the syntactic element of the processing object.
[0159] On the other hand, when the count value of the object point is not 0 ( S121 : No), the three-dimensional data encoding device performs the processing of step S103 described above.
[0160] Here, when setting a context for each combination of a count value and history information, contexts whose number is equal to the possible values of the count value × the possible values of the history information are required. Fig. 9 As shown, by using the history information only when the count value is 0, the number of contexts can be suppressed compared to the above case. Fig. 9 The processing shown has the possibility of improving the encoding efficiency regardless of whether the point group to be processed is obtained by the multi-pass mode or the single-pass mode.
[0161] In addition, the history information may be updated each time a point is processed (encoded or decoded), or may be updated only when the count value is 0 or a predetermined value. Fig. 9 In the process shown, the history information may be updated in steps S103 and S122. Alternatively, the history information may be updated only in step S122. In other words, only the information of the points with a count value of 0 may be used as the history information. By updating the history information only in step S122, only the information of the points expected to be highly correlated can be reflected in the history information. Thus, there is a possibility that the coding efficiency can be improved.
[0162] In addition, when the prediction tree is configured to branch at the first node in the angular direction as described above, and the last node in the angular direction becomes a leaf node, the three-dimensional data encoding device may also determine whether the node encoded immediately before is a leaf node before implementing step S103. In this case, the three-dimensional data encoding device may also use the history information to determine the context when the node encoded immediately before is a leaf node, and determine the context without using the history information when the node encoded immediately before is not a leaf node. In addition, the context is determined without using the history information, and a method using a count value or another method may be used. Another method may also be a method using one common context, for example. In addition, when the count value is not used, steps S101, S105 to S108 may not be performed.
[0163] Furthermore, when the prediction tree does not branch at the initial node in the angular direction, the three-dimensional data encoding device may determine the context for the node to be encoded next to the initial node using the history information.
[0164] In addition, Fig. 9 In the example shown, the context is determined based on the history information when the count value is 0, but the context may be determined using the history information when the count value is any one of more than one predetermined value. Figure 3 In the example shown, when the count value is 4, 5, or 0, the context may be determined based on the history information.
[0165] In addition, the above description describes an example in which a plurality of points with different distances exist in the same angular direction obtained by multiple returns or the like, but the above processing can also be applied to the following cases.
[0166] Fig.10 It is a diagram showing the encoding order (processing order) of multiple three-dimensional points (multiple reference positions) in the encoding process. The three-dimensional data encoding device generates conversion information by converting the position information included in the input point group data of the encoding object. Specifically, the three-dimensional data encoding device generates information for establishing correspondence between multiple reference positions and three-dimensional points. In addition, the three-dimensional data encoding device converts the position information of the three-dimensional point using the reference position. For example, the conversion information is the difference between the reference position and the position information of the three-dimensional point. In addition, the three-dimensional data encoding device performs the above-mentioned encoding process on the conversion information.
[0167] Furthermore, the three-dimensional data decoding device performs inverse conversion of the conversion process performed by the three-dimensional data encoding device on the conversion information obtained by decoding the bit stream, thereby restoring the position information.
[0168] exist Fig.10In the horizontal direction, the horizontal angle of the polar coordinates is The vertical direction represents the elevation angle θ of the polar coordinates. The three-dimensional data encoding device sets a plurality of reference positions rm (where m=0, 1, 2, ...) (also referred to as reference points). Here, the reference position rm is a horizontal angle. That is, the reference position rm is represented by three components (d, θ, ) of the two components (θ, ) performance. In addition, Fig.10 In the example shown, the sampling interval in the horizontal direction of the LiDAR is and the scanning line interval Δθk of the LiDAR (where k = 1, 2, 3), the reference position rm shown by the box in the figure is set. That is, multiple reference positions are set according to the combination of multiple predetermined horizontal angles and multiple elevation angles. and elevation angle θ are arranged in a matrix shape. Fig.10 In the example shown, multiple horizontal angles of multiple reference positions (where j = 0, 1, 2, ...) In addition, the intervals between a plurality of elevation angles θk (where k=0, 1, 2, 3) of a plurality of reference positions can be set individually.
[0169] In addition, the three-dimensional data encoding device performs encoding processing (conversion processing) of points pn (where n=0, 1, 2, ...) represented by diamonds located near each reference position in the order shown by the dotted arrows in the figure. In addition, a shaded box indicates a first reference position where a point referenced to the reference position exists, and a non-shaded box indicates a second reference position where a point referenced to the reference position does not exist.
[0170] The reference point is a point based on the reference position and is a point that is associated with the reference position (encoded (converted) using the reference position) as described later. and the elevation angle θ are included in the prescribed range including the reference position. For example, the point with reference to the reference position is More than and less than The horizontal angle range is not limited to this, and can also be More than and less than wait.
[0171] In addition, as Fig.10 The processing order (coding order) shown in FIG. 1 is to set a plurality of reference positions (e.g., r0 to r3) having the same horizontal angle as the processing unit (corresponding to Fig.10 ), and in order based on elevation angle within the processing unit (in Fig.10 In ascending order, multiple reference positions are processed (encoded). In addition, multiple processing units (corresponding to Fig.10 The columns of Fig.10 In other words, the plurality of reference positions are processed in ascending order of the elevation angle for each plurality of reference positions having the same value of the horizontal angle. In addition, the plurality of reference positions may also be processed in ascending order of the horizontal angle for each plurality of reference positions having the same value of the elevation angle.
[0172] The three-dimensional data encoding device generates a position ( In addition, the three-dimensional data encoding device generates an offset ( θ o n), and information for determining the distance information dn of the object point. is the horizontal angle of the reference position The difference in horizontal angle from the object point, θ o n is the difference between the elevation angle θk of the reference position and the elevation angle of the target point.
[0173] In addition, information on the position of the reference position to which the target point is referred, and the offset from the reference position to the target point ( θ o n), information for determining the distance information dn of the object point may be information for determining a difference value between the predicted value generated based on the processed information, or information for determining the value itself.
[0174] In addition, the three-dimensional data encoding device 100 may also set the sampling interval of the LiDAR in the horizontal direction to and the scanning line interval Δθk of LiDAR are stored in the bit stream. For example, the three-dimensional data encoding device can also store and Δθk are stored in the header of SPS or GPS. Thus, the three-dimensional data decoding device can use and Δθk to set multiple reference positions.
[0175] Next, the syntax of the position information will be described. Fig.11 is a diagram showing an example of the syntax for representing the position information of each point. Fig.11 and Fig.12 In the syntax example shown, the parameters (signals) stored in the bit stream are recorded in bold. The three-dimensional data encoding device repeatedly applies this syntax to each reference position rm, and generates a horizontal angle representing the reference position rm as the reference of the point pn to be processed next. The column_pos indexed by and the row_pos indexed by the elevation angle θk are used to generate parameters related to the point pn.
[0176] In this example, the three-dimensional data encoding device initializes the variables before processing the front point. The first corresponding syntax first_point_in_column is set to 1, column_pos is set to 0, and row_pos is set to 0. Alternatively, the 3D data encoding device may notify the 3D data decoding device of the value of column_pos and the value of row_pos of the leading point before the syntax corresponding to the leading point. In this case, the 3D data encoding device and the 3D data decoding device set first_point_in_column to 0 and use these values to apply the syntax.
[0177] Next, the three-dimensional data encoding device generates next_column_flag at the reference position rm corresponding to the position with an elevation angle of θ0 (that is, when first_point_in_column is 1). next_column_flag indicates the position of the reference position rm at a horizontal angle corresponding to the position of the reference position rm. In other words, next_column_flag indicates the horizontal angle between the horizontal angle and the reference position rm. Whether there is a point that is referenced by one of the same plurality of reference positions. For example, at a horizontal angle corresponding to the position of the reference position rm When there is more than one reference point (for example, Fig.10 Horizontal angle shown and ), next_column_flag is set to 0, and the horizontal angle corresponding to the position of the reference position rm is When the reference point does not exist (for example, Fig.10 Horizontal angle shown ), next_column_flag is set to 1. In addition, next_column_flag is set to 1 for each horizontal angle. ( Fig.10 ) settings for each column.
[0178] The three-dimensional data encoding device can generate a horizontal angle corresponding to the point pn to be processed next by repeatedly generating next_column_flag until next_column_flag is 0. Thus, there is a possibility that the amount of code required for the notification of next_row_flag described below can be reduced. In addition, by determining whether row_pos is 0, the following can also be implemented. Fig.12 However, by using first_point_in_column for determination, it is possible to avoid unnecessary notification of next_column_flag even when there are multiple points at the position where row_pos is 0, so there is a possibility that the amount of code can be reduced.
[0179] The three-dimensional data encoding device generates next_row_flag at each candidate position of the reference position rm, which is the reference of the point pn to be processed next. Next_row_flag indicates whether there is a point pn to be processed at the position of the elevation angle θk. In other words, next_row_flag indicates whether there is a point with reference to the reference position rm. For example, in the case where there is a point pn to be processed at the position of the elevation angle θk, next_row_flag is set to 0 (for example, Fig.10 r0, r1), when there is no point pn to be processed at the position of the elevation angle θk (for example, Fig.10 r2, r3), next_row_flag is set to 1. In addition, next_row_flag is set for each reference position.
[0180] When next_row_flag is 1, the three-dimensional data encoding device repeatedly applies Fig.11 The syntax shown in the figure continuously generates next_row_flag corresponding to each candidate position. The three-dimensional data encoding device repeatedly performs this process until next_row_flag becomes 0, thereby generating information that can identify the elevation angle θk corresponding to the point pn to be processed next.
[0181] In addition, when row_pos reaches the number of scan lines ( Fig.11 num_rows) shown in the figure, process the next horizontal angle At this time, the three-dimensional data encoding device sets row_pos to 0, increases column_pos by 1, and sets first_point_in_column to 1.
[0182] Thus, the three-dimensional data encoding device can generate a horizontal angle φ that can determine the reference position rm as a reference of the point pn to be processed. and elevation angle θk information (next_column_flag, next_row_flag).
[0183] Next, the three-dimensional data encoding device generates information related to the distance of the object point pn, information related to the offset of the horizontal angle from the reference position rm to the object point pn, and pred_mode which is information related to the prediction method of these parameters. Here, the information related to the distance is, for example, residual_radius representing the difference between the distance of the object point and the predicted value generated by a predetermined method. In addition, the information related to the offset of the horizontal angle is, for example, The residual_phi is the difference between the predicted value and the predicted value generated by the specified method.
[0184] In addition, the prediction value is calculated based on information of the processed three-dimensional point, for example. For example, the prediction value is at least a part of the parameters of one or more processed three-dimensional points located near the object point. In addition, in this example, the three-dimensional data encoding device assumes that the elevation angle bias is always 0, and omits the generation of information related to the elevation angle bias, but it can also generate information related to the elevation angle bias from the reference position rm to the processing object point pn and store it in the bit stream. For example, the information related to the elevation angle bias is the bias θ indicating the elevation angle. o The residual residual_theta is the difference between n and the predicted value generated by the specified method.
[0185] Next, other examples of syntax are described. Fig.12 is a diagram showing an example of syntax for representing position information of each point. The three-dimensional data encoding device repeatedly applies this syntax to each reference position rm, and generates a reference position rm representing the horizontal angle of the reference position rm as the reference of the point pn to be processed next. The index column_pos and the index row_pos representing the elevation angle θk are used to generate parameters related to the point pn. In addition, Fig.12 The example shown is relative to Fig.11 In the example shown, the generation methods of next_row_flag and next_column_flag used to determine the values of column_pos and row_pos are different.
[0186] In this example, the three-dimensional data encoding device first initializes the variables before applying the syntax to the front point. Specifically, the three-dimensional data encoding device notifies the three-dimensional data decoding device of the column_pos value and the row_pos value of the front point before the syntax corresponding to the front point. That is, the three-dimensional data encoding device stores the column_pos value and the row_pos value of the front point in the bit stream, for example. In addition, the three-dimensional data encoding device and the three-dimensional data decoding device use these values to apply the syntax.
[0187] Next, the three-dimensional data encoding device generates next_row_flag for the reference position rm of the positions indicated by next_row_flag and next_column_flag, and notifies the three-dimensional data decoding device whether there is a point pn based on this rm.
[0188] When next_row_flag is 1, the three-dimensional data encoding device first increases row_pos by 1. Next, the three-dimensional data encoding device determines whether row_pos has reached the number of scan lines ( Fig.12 When row_pos reaches the number of scan lines, the three-dimensional data encoding device determines that the candidate position is transferred to the next horizontal angle. Set row_pos to 0 and increment column_pos by 1. Next, the three-dimensional data encoding device determines whether row_pos is 0. If row_pos is 0, the three-dimensional data encoding device generates one or more next_column_flags and repeatedly increments column_pos by 1 until next_column_flag is 0. Thereafter, the three-dimensional data encoding device repeatedly applies Fig.12 The syntax shown is until next_row_flag is 0.
[0189] In addition, when next_row_flag is 0, the three-dimensional data encoding device determines the values indicated by next_row_flag and next_column_flag at that time as the horizontal angle of the reference position rm serving as the reference of the point pn to be processed next. The index of and the index of the elevation angle θk, and Fig.11 In the example shown, the parameters related to the point pn (for example, Fig.12 In addition, the horizontal angle By using The index values and the horizontal sampling interval of LiDAR can be used In addition, the elevation angle θk can be calculated based on each index value and the scanning line interval Δθk of the LiDAR.
[0190] Furthermore, when the coordinate system is not transformed, residual_x, residual_y, and residual_z may not be included in the bitstream. Alternatively, residual_theta may be included in the bitstream.
[0191] Next, the arithmetic coding process of next_row_flag will be described. Fig.13 This is a diagram for explaining an example of a method of selecting a context (probability table) when performing arithmetic coding on next_row_flag. Fig.13 The reference position rm shown is a reference position corresponding to next_row_flag to be encoded.
[0192] The entropy coding unit 104 can use the encoding process of next_row_flag corresponding to the reference position rm. Fig.13 The entropy coding unit 104 stores information on reference positions included in the range of processing completed by the dashed line and indicated by the shade. For example, the entropy coding unit 104 stores a predetermined number of first reference positions (e.g., reference positions A0, B0, C0 of the scanning line with elevation angle θ0, etc., which are shaded boxes) of the most recently processed points in each scanning line that refer to the reference position in the memory. The entropy coding unit 104 switches the context used in the arithmetic coding of next_row_flag based on the information on the first reference positions stored in the memory.
[0193] For example, the entropy coding unit 104 uses information of at least one of the reference positions A1, B1, and C1 located on the same scanning line as the reference position rm. Specifically, the entropy coding unit 104 may also use the difference in column_pos between at least one of the reference positions A1, B1, and C1 and the reference position rm. For example, the entropy coding unit 104 may also use the difference in column_pos between the nearest reference position A1 and the reference position rm. Alternatively, the entropy coding unit 104 may also use a combination of the difference in column_pos between the closest reference position A1 and the reference position rm and the difference in column_pos between the second closest reference position B1 and the reference position rm. In this way, the entropy coding unit 104 may also determine the context based on whether one or more reference positions located on the same scanning line as the reference position rm are the first reference positions (whether there are points that refer to the reference positions). Here, in the point cloud data obtained by LiDAR, etc., there is a possibility that the correlation between the points located on the same scanning line is high. Thus, by selecting the context with reference to the information of the points located on the same scanning line, the context can be selected appropriately.
[0194] Alternatively, the entropy coding unit 104 may also use the information of the first reference position (e.g., reference position A0) that has been processed most recently. Specifically, the entropy coding unit 104 may also switch the context based on the number of times that next_row_flag is consecutively 1 from the reference position A0 to the reference position rm. In addition, the entropy coding unit 104 is not limited to switching the context based on the information of the reference position held in the memory, but may also switch the context based on row_pos of the reference position rm itself.
[0195] The above-mentioned context determination method of using the count value can also be applied to using Figure 10 to Figure 13 For example, in the case of a point group obtained in a multi-return mode in the same angular direction (e.g., horizontal angle When multiple points with different distances are arranged in each angular direction in the range, after the next_row_flag value of 0 is continuously encoded, when moving to the next angular direction, the next_row_flag value of 1 is encoded. Therefore, when encoding the next_row_flag value of 1, the context is updated in accordance with the next_row_flag value of 0, and there is a problem that the amount of code cannot be suppressed.
[0196] On the other hand, by applying a context determination method based on a count value, the amount of code can be suppressed. Fig.14 This is a flowchart showing a modified example of the process of arithmetic coding or arithmetic decoding of a syntax element corresponding to an object point. Fig.14 The processing shown is similar to Figure 4The processing shown is different in that S105B is included instead of steps S105 and S106.
[0197] like Fig.14 As shown, the three-dimensional data encoding device is Figure 4 In the process shown, instead of the determination process of steps S105 and S106, it is determined whether next_row_flag is 0 (S105B). The three-dimensional data encoding device performs step S107 when next_row_flag is not 0 (S105B: No), and performs step S108 when next_row_flag is 0 (S105B: Yes).
[0198] Thus, for example, the context used for a point with a small count value is updated to a state suitable for a next_row_flag value of 0 with a high selection frequency. In addition, the context used for a point with a large count value is updated to a state suitable for a next_row_flag value of 1, or a state that is neutral to both the value 0 and the value 1 of next_row_flag. Thus, the coding efficiency when encoding the next_row_flag value of 0 can be maintained, and the coding efficiency of the next_row_flag when the processing is transferred to the next angular direction can be improved. Thus, there is a possibility that the coding efficiency of the entire bit stream can be improved.
[0199] In addition, you can also use Fig.11 The described next_row_flag context determination method using information of the processed reference point is combined with the next_row_flag context determination method based on the count value. This combination may improve the coding efficiency regardless of whether the point group to be processed is obtained in the multi-return mode or the single-return mode.
[0200] In addition, in the above combination, the three-dimensional data encoding device may determine the context using the information of the processed reference point only when the count value is a specified value such as 0, and determine the context based on the count value without using the information of the processed reference point when the count value is not the specified value. Thus, the increase in the number of contexts can be suppressed. In addition, the three-dimensional data encoding device may update the information of the processed reference point each time the point is encoded or decoded, but may update it only when the count value is a specified value such as 0. Thus, the three-dimensional data encoding device can reflect only the information of the point expected to be highly correlated to the information of the processed reference point maintained, so there is a possibility that the encoding efficiency can be improved.
[0201] In addition, through Fig.14The process shown in FIG. 1 selects the syntactic element of the context according to the count value, for example, Fig.11 or Fig.12 For example, the three-dimensional data encoding device may also select, based on the count value, Fig.11 or Fig.12 The context of next_row_flag is shown. This makes it possible to reduce the amount of memory required to hold the context.
[0202] In addition, it is not limited to always having to use Figure 4 , Figure 6 , Fig. 9 or Fig.14 Of all the processing described, only a part of the processing may be performed.
[0203] In addition, in the above description, an example of selecting a context used in arithmetic coding included in entropy coding according to a count value is described, but the binarization method included in entropy coding can also be changed. For example, the 0 and 1 of the output signal (binary signal) of the conversion table used in binarization can also be swapped according to the counter. Thus, even if the same context is used at the first point where the frequency of occurrence of the value 0 is high and the second point where the frequency of occurrence of the value 1 is high, for example, the value 1 of the signal at the second point where the frequency of occurrence of the value 1 is high can be converted to the value 0, so in either case, the frequency of occurrence of 0 of the signal (binarized signal) of the encoding object is calculated to be high. Thus, even when the same context is used, there is a possibility that the encoding efficiency can be improved. For example, the three-dimensional data encoding device can also use the first binarization method at a point where the count value is larger than the threshold, and use the second binarization method that swaps the 0 and 1 of the output signal relative to the first binarization method at a point where the count value is below the threshold.
[0204] Furthermore, in the decoding process in the three-dimensional data decoding device, the binarization method included in the above-mentioned entropy encoding is replaced with a multi-valued method in entropy decoding.
[0205] As described above, the encoding device (three-dimensional data encoding device) involved in this embodiment performs Fig.15The processing shown. The encoding device encodes a plurality of three-dimensional points each having position information including a distance component, a first direction component, and a second direction component (e.g., a horizontal angle component and an elevation angle component). The encoding device determines at least one of a method of entropy encoding and a method of binarization of information related to the first three-dimensional point based on the number of second three-dimensional points (e.g., a count value) among the encoded plurality of three-dimensional points having first direction components and second direction components that are substantially equal to the first direction components and second direction components of the first three-dimensional point included in the plurality of three-dimensional points (S201), and performs processing using at least one of the determined methods (S202).
[0206] Thus, in the encoding device, if the directional component is switched, the number of the second three-dimensional points that have been decoded changes, so at least one of the entropy coding method and the binarization method is switched. In addition, the directional component switching is the switching of at least one of the first directional component and the second directional component. Thus, the encoding device can, for example, select at least one of the entropy coding method and the binarization method suitable for the switching of the directional component, thereby improving the coding efficiency. In addition, the directional components are substantially equal, for example, the difference between the directional components is less than a threshold value.
[0207] For example, the encoding device determines a context used in arithmetic coding in at least one of the methods of determining entropy coding and binarization (S201), and in processing (S202), uses the determined context to arithmetically encode information related to the first three-dimensional point.
[0208] Thus, the coding device can switch the context used in arithmetic coding in accordance with the switching of the directional component. Thus, the coding device can, for example, select a context suitable for the switching of the directional component, thereby improving the coding efficiency. In addition, entropy coding is not limited to arithmetic coding. For example, entropy coding can also be Huffman coding.
[0209] For example, information related to the first three-dimensional point (e.g., intra_pred_flag) indicates which of the inter-frame prediction mode and the intra-frame prediction mode is used. Here, in a plurality of points where the directional components (the first directional component and the second directional component) are substantially the same, there is a tendency that the intra-frame prediction mode is continuously selected. Thus, by switching at least one of the entropy encoding method and the binarization method in accordance with the switching of the directional components, the encoding efficiency can be improved.
[0210] For example, the information (quantized_1st_residual_value) related to the first three-dimensional point represents the prediction residual of the first directional component or the second directional component. Here, at a plurality of points where the directional components (the first directional component and the second directional component) are substantially the same, there is a tendency for the prediction residual to become smaller. Thus, by switching at least one of the entropy coding method and the binarization method in accordance with the switching of the directional components, the coding efficiency can be improved.
[0211] For example, the prediction residual (quantized_1st_residual_value) of the first directional component or the second directional component is information obtained by quantizing the prediction residual of the horizontal angle component. Here, at a plurality of points where the directional components (the first directional component and the second directional component) are substantially the same, there is a tendency that the prediction residual of the horizontal angle component becomes smaller. Thus, by switching at least one of the entropy coding method and the binarization method in accordance with the switching of the directional components, the coding efficiency can be improved.
[0212] For example, the number is the number of second three-dimensional points whose horizontal angle components are substantially equal to the first three-dimensional point. For example, when points are sequentially scanned and encoded in the horizontal angle direction, the switching of the direction components can be determined using only the horizontal angle component. In addition, by determining only using the horizontal angle component, the amount of processing can be reduced.
[0213] For example, the number is the number of second three-dimensional points having a quantized first directional component and a quantized second directional component that are substantially equal to the quantized first directional component and the quantized second directional component of the first three-dimensional point, respectively. For example, the number is the number of second three-dimensional points having a quantized horizontal angle component that is substantially equal to the quantized horizontal angle component of the first three-dimensional point.
[0214] For example, the encoding device limits the number of the second three-dimensional points in determining at least one of the entropy encoding method and the binarization method, and determines at least one of the entropy encoding method and the binarization method for information related to the first three-dimensional point based on the limited number. Thus, for example, when switching methods according to the number, the number of methods used can be reduced, thereby reducing the amount of processing or the amount of memory used.
[0215] For example, when determining at least one of the entropy coding method and the binarization method, the coding device quantizes the number of the second three-dimensional points, and determines at least one of the entropy coding method and the binarization method for information related to the first three-dimensional point based on the quantized number. Thus, for example, when switching methods according to the number, the number of methods used can be reduced, thereby reducing the amount of processing or the amount of memory used.
[0216] For example, when the encoding device determines at least one of the entropy encoding method and the binarization method, if the number is not within a predetermined range (for example, Fig. 9 S121: No), determine at least one of the entropy coding method and binarization method of the information related to the first three-dimensional point according to the number of the second three-dimensional points (for example, S103), and when the number is within a predetermined range (for example, S121: Yes), determine at least one of the entropy coding method and binarization method of the information related to the first three-dimensional point (for example, S122) according to the historical information obtained based on the information related to the encoded three-dimensional points. For example, the predetermined range can be multiple continuous values, multiple discontinuous values, or a single value. For example, in Fig. 9 In the example shown, the predetermined range includes only the value 0.
[0217] Thus, the encoding device can select an appropriate method using the history information even when there is a state where the switching of the method corresponding to the number is not effective. In addition, compared with the case where the method is switched for each combination of the number and the history information, the number of methods used can be reduced, thereby reducing the processing amount or the memory capacity used.
[0218] For example, the history information is the number or pattern of 0 or 1 of the information of the encoded multiple three-dimensional points. Thus, the encoding device can select an appropriate method according to the number or pattern of 0 or 1 of the information of the encoded multiple three-dimensional points.
[0219] For example, the history information is updated by information of three-dimensional points whose number is a predetermined value, and is not updated by information of three-dimensional points whose number is other than the predetermined value. Thus, the encoding device can reflect information of points expected to be highly correlated in the history information, thereby improving encoding efficiency.
[0220] For example, information related to the first three-dimensional point (e.g., next_row_flag) indicates whether there is a three-dimensional point corresponding to the reference position. Here, when the direction component is switched, the information indicating whether there is a three-dimensional point corresponding to the reference position tends to have a specific value. Thus, by switching at least one of the entropy coding method and the binarization method in accordance with the switching of the direction component, the coding efficiency can be improved.
[0221] For example, in the encoding device, the smaller the number is, the more suitable context for indicating that there is a three-dimensional point corresponding to the reference position is used to perform arithmetic coding on the information related to the first three-dimensional point. That is, when the number is a first value, the encoding device performs arithmetic coding on the information related to the first three-dimensional point using a first context, and when the number is a second value smaller than the first value, the encoding device performs arithmetic coding on the information related to the first three-dimensional point using a second context that is more suitable than the first context for the information related to the first three-dimensional point indicating that there is a three-dimensional point corresponding to the reference position.
[0222] For example, the encoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.
[0223] In addition, the decoding device (three-dimensional data decoding device) according to the present embodiment performs Fig.16 The processing shown. The decoding device decodes a plurality of encoded three-dimensional points, each of which has position information including a distance component, a first direction component, and a second direction component (e.g., a horizontal angle component and an elevation angle component). The decoding device determines at least one of a method of entropy decoding and a method of multi-valued information related to the first encoded three-dimensional point based on the number of second three-dimensional points (e.g., a count value) among the decoded three-dimensional points that have first direction components and second direction components that are substantially equal to the first direction components and second direction components of the first encoded three-dimensional point included in the plurality of encoded three-dimensional points (S211), and performs processing using at least one of the determined methods (S212).
[0224] Thus, in the decoding device, if the directional components (the first directional component and the second directional component) are switched, the number of the second three-dimensional points that have been decoded changes, so at least one of the entropy decoding method and the multi-valued method is switched. Thus, the decoding device can, for example, select at least one of the entropy decoding method and the multi-valued method that is suitable for the switching of the directional components. Thus, the decoding device can properly decode the bit stream with improved coding efficiency. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced. In addition, the directional components are substantially equal, for example, the difference between the directional components is below a threshold.
[0225] For example, the decoding device determines the context used in arithmetic decoding in at least one of the methods of determining entropy decoding and the multi-valued method (S211), and in processing (S212), uses the determined context to arithmetically decode information related to the encoded first three-dimensional point.
[0226] Thus, the decoding device can switch the context used in arithmetic decoding in accordance with the switching of the directional component. Thus, the decoding device can, for example, select a context suitable for the switching of the directional component. Thus, the decoding device can properly decode the bit stream with improved coding efficiency. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced. In addition, entropy decoding is not limited to arithmetic decoding. For example, entropy decoding can also be Huffman decoding.
[0227] For example, information related to the first encoded three-dimensional point (e.g., intra_pred_flag) indicates which of the inter-frame prediction mode and the intra-frame prediction mode is used. Here, in a plurality of points where the directional components (the first directional component and the second directional component) are substantially the same, there is a tendency for the intra-frame prediction mode to be continuously selected. Thus, the decoding device can properly decode a bitstream with improved coding efficiency by switching at least one of the entropy decoding method and the multi-valued method in accordance with the switching of the directional components. In addition, by improving the coding efficiency of the bitstream, the amount of data processed by the decoding device can be reduced.
[0228] For example, the information (quantized_1st_residual_value) related to the first encoded three-dimensional point represents the prediction residual of the first directional component or the second directional component. Here, in a plurality of points where the directional components (the first directional component and the second directional component) are substantially the same, there is a tendency for the prediction residual to become smaller. Thus, the decoding device can properly decode the bit stream with improved coding efficiency by switching at least one of the entropy decoding method and the multi-valued method in accordance with the switching of the directional components. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced.
[0229] For example, the prediction residual (quantized_1st_residual_value) of the first directional component or the second directional component is information obtained by quantizing the prediction residual amount of the horizontal angle component. Here, at a plurality of points where the directional components (the first directional component and the second directional component) are substantially the same, there is a tendency for the prediction residual of the horizontal angle component to become smaller. Thus, the decoding device can properly decode a bit stream with improved coding efficiency by switching at least one of the entropy decoding method and the multi-valued method in accordance with the switching of the directional components. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced.
[0230] For example, the number is the number of second three-dimensional points whose horizontal angle components are substantially equal to the first three-dimensional point. For example, when points are sequentially scanned and decoded in the horizontal angle direction, the switching of the direction components can be determined using only the horizontal angle components. In addition, by determining only using the horizontal angle components, the amount of processing can be reduced.
[0231] For example, the number is the number of second three-dimensional points having a quantized first directional component and a quantized second directional component that are substantially equal to the quantized first directional component and the quantized second directional component of the first encoded three-dimensional point, respectively. For example, the number is the number of second three-dimensional points having a quantized horizontal angle component that is substantially equal to the quantized horizontal angle component of the first encoded three-dimensional point.
[0232] For example, the decoding device limits the number of the second three-dimensional points in determining at least one of the entropy decoding method and the multi-valued method (S211), and determines at least one of the entropy decoding method and the multi-valued method for information related to the first three-dimensional point based on the limited number. Thus, for example, when switching methods according to the number, the number of methods used can be reduced, thereby reducing the amount of processing or the amount of memory used.
[0233] For example, the decoding device quantizes the number of the second three-dimensional points in determining at least one of the entropy decoding method and the multi-valued method (S211), and determines at least one of the entropy decoding method and the multi-valued method for information related to the first three-dimensional point based on the quantized number. Thus, for example, when switching methods by number, the number of methods used can be reduced, thereby reducing the amount of processing or the amount of memory used.
[0234] For example, in determining at least one of the entropy decoding method and the multi-valued method (S211), when the number is not within a predetermined range (for example, Fig. 9 S121: No), determine at least one of the entropy decoding method and the multi-valued method of the information related to the first three-dimensional point according to the number of the second three-dimensional points (for example, S103), and when the number is within a predetermined range (for example, S121: Yes), determine at least one of the entropy decoding method and the multi-valued method of the information related to the first three-dimensional point according to the historical information obtained based on the information related to the multiple decoded three-dimensional points (for example, S122). For example, the predetermined range can be multiple continuous values, multiple discontinuous values, or a single value. For example, in Fig. 9 In the example shown, the predetermined range includes only the value 0.
[0235] Thus, the decoding device can select an appropriate method using the history information even when there is a state where the switching of the method corresponding to the number is not effective. In addition, compared with the case where the method is switched for each combination of the number and the history information, the number of methods used can be reduced, thereby reducing the processing amount or the memory capacity used.
[0236] For example, the history information is the number or pattern of 0 or 1 of the decoded 3D point information. Thus, the decoding device can select an appropriate method according to the number or pattern of 0 or 1 of the decoded 3D point information.
[0237] For example, the history information is updated by information of three-dimensional points whose number of decodes is a predetermined value, and is not updated by information of three-dimensional points whose number of decodes is other than the predetermined value. As a result, the decoding device can reflect information of points expected to be highly correlated in the history information, and thus can properly decode the bit stream with improved coding efficiency. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced.
[0238] For example, information related to the first three-dimensional point (e.g., next_row_flag) indicates whether there is a three-dimensional point corresponding to the reference position. Here, when the directional component is switched, the information indicating whether there is a three-dimensional point corresponding to the reference position has a tendency to become a specific value. Thus, the decoding device can properly decode the bit stream with improved coding efficiency by switching at least one of the entropy decoding method and the multi-valued method in accordance with the switching of the directional component. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced.
[0239] For example, in the decoding device, the smaller the number is, the more suitable context for the information related to the first three-dimensional point indicating that there is a three-dimensional point corresponding to the reference position is used to perform arithmetic decoding on the information related to the first three-dimensional point. That is, when the number is a first value, the decoding device performs arithmetic decoding on the information related to the first three-dimensional point using the first context, and when the number is a second value smaller than the first value, the information related to the first three-dimensional point is performed arithmetic decoding using the second context that is more suitable than the first context for the information related to the first three-dimensional point indicating that there is a three-dimensional point corresponding to the reference position.
[0240] For example, the decoding device includes a processor and a memory, and the processor performs the above-mentioned processing using the memory.
[0241] For example, the encoding device encodes a plurality of three-dimensional points each having position information including a distance component, a first direction component, and a second direction component (e.g., a horizontal angle component and an elevation angle component). The encoding device determines at least one of an entropy encoding method and a binarization method for information related to the first three-dimensional point based on whether a point encoded immediately before the first three-dimensional point corresponds to a leaf node in a prediction tree, and performs processing using the determined method.
[0242] Here, the node when the directional component is switched has a tendency to be a leaf node. Thus, the encoding device can switch at least one of the entropy encoding method and the binarization method in accordance with the switching of the directional component. Thus, the encoding device can, for example, select at least one of the entropy encoding method and the binarization method suitable for the switching of the directional component, thereby improving the encoding efficiency.
[0243] For example, the encoding device determines a context to be used in arithmetic coding in the determination, and performs arithmetic coding on the information related to the first three-dimensional point using the determined context in the processing.
[0244] For example, in the decision, when the point encoded immediately before the first three-dimensional point corresponds to the leaf node, the first context is selected, and when the point encoded immediately before the first three-dimensional point does not correspond to the leaf node, the second context is selected, and the first context is more suitable for inter-frame prediction than the second context.
[0245] Here, there is a tendency to use inter-frame prediction for the point after the directional component is switched. This allows the encoding device to improve encoding efficiency.
[0246] For example, the encoding device encodes a plurality of three-dimensional points each having position information including a distance component, a first direction component, and a second direction component (e.g., a horizontal angle component and an elevation angle component). The encoding device entropy encodes or binarizes information related to a first three-dimensional point to be encoded initially among the plurality of three-dimensional points having the same value of the first direction component, using a first method of at least one of entropy encoding and binarization. The encoding device entropy encodes or binarizes information related to a second three-dimensional point other than the first three-dimensional point among the plurality of three-dimensional points having the same value of the first direction component, using a second method of at least one of entropy encoding and binarization. The second method of entropy encoding is different from the first method of entropy encoding, and the second method of binarization is different from the first method of binarization.
[0247] Here, the point after the first direction component is switched tends to be processed differently from other points. Therefore, the encoding device can improve encoding efficiency by using different methods when the first direction component is switched.
[0248] For example, the encoding device uses a first context to arithmetically encode the information related to the first three-dimensional point in the entropy encoding or the binarization of the information related to the first three-dimensional point, and uses a second context different from the first context to arithmetically encode the information related to the second three-dimensional point in the entropy encoding or the binarization of the information related to the second three-dimensional point.
[0249] For example, the first context is suitable for inter-frame prediction, and the second context is suitable for intra-frame prediction. Here, there is a tendency to use inter-frame prediction for the point after the first direction component is switched and intra-frame prediction for other points. Thus, the encoding device can improve encoding efficiency.
[0250] For example, the first direction component is a horizontal direction component.
[0251] For example, the encoding device encodes a plurality of three-dimensional points each having position information including a distance component, a first direction component, and a second direction component (e.g., a horizontal angle component and an elevation angle component). When the first three-dimensional point corresponds to a leaf node in a prediction tree, the encoding device determines at least one of an entropy encoding method and a binarization method for information related to the first three-dimensional point based on history information obtained based on information related to the plurality of encoded three-dimensional points.
[0252] As a result, the encoding device can use the historical information to select an appropriate method, thereby improving encoding efficiency.
[0253] For example, when the first three-dimensional point does not correspond to the leaf node, the encoding device determines at least one of the entropy encoding method and the binarization method of the information related to the first three-dimensional point based on the number of second three-dimensional points (e.g., a count value) whose differences between the first direction component and the second direction component and the first three-dimensional point are below a threshold.
[0254] For example, the decoding device decodes a plurality of three-dimensional points each having position information including a distance component, a first direction component, and a second direction component (e.g., a horizontal angle component and an elevation angle component). The decoding device determines at least one of an entropy decoding method and a multi-valued method for information related to the first three-dimensional point, based on whether a point decoded immediately before the first three-dimensional point corresponds to a leaf node in a prediction tree, and performs processing using the determined method.
[0255] Here, the node when the first directional component and the second directional component are switched has a tendency to be a leaf node. Thus, the decoding device can switch at least one of the entropy decoding method and the multi-valued method in accordance with the switching of the directional component. Thus, the decoding device can, for example, select at least one of the entropy decoding method and the multi-valued method suitable for the switching of the directional component. Thus, the decoding device can properly decode the bit stream with improved coding efficiency. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced.
[0256] For example, the decoding device determines a context to be used in arithmetic decoding in the determination, and performs arithmetic decoding on the information related to the first three-dimensional point using the determined context in the processing.
[0257] For example, in the decision, when the point decoded immediately before the first three-dimensional point corresponds to the leaf node, the first context is selected, and when the point decoded immediately before the first three-dimensional point does not correspond to the leaf node, the second context is selected, and the first context is more suitable for inter-frame prediction than the second context.
[0258] Here, there is a tendency to use inter-frame prediction for the point after the directional component is switched. As a result, the decoding device can properly decode the bit stream with improved coding efficiency. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced.
[0259] For example, the decoding device decodes a plurality of three-dimensional points each having position information including a distance component, a first direction component, and a second direction component (e.g., a horizontal angle component and an elevation angle component). The decoding device performs entropy decoding or multi-valuedization using a first method of at least one of entropy decoding and multi-valuedization for information related to a first three-dimensional point to be decoded initially among the plurality of three-dimensional points having the same value of the first direction component. The decoding device performs entropy decoding or multi-valuedization using a second method of at least one of entropy decoding and multi-valuedization for information related to a second three-dimensional point other than the first three-dimensional point among the plurality of three-dimensional points having the same value of the first direction component. The second method of entropy decoding is different from the first method of entropy decoding, and the second method of multi-valuedization is different from the first method of multi-valuedization.
[0260] Here, the point after the first direction component is switched has a tendency to be processed differently from other points. Thus, the decoding device can properly decode the bit stream with improved coding efficiency by using a different method when the first direction component is switched. In addition, the amount of data processed by the decoding device can be reduced by improving the coding efficiency of the bit stream.
[0261] For example, the decoding device uses a first context to arithmetically decode the information related to the first three-dimensional point in the entropy decoding or the multi-valuedization of the information related to the first three-dimensional point, and uses a second context different from the first context to arithmetically decode the information related to the second three-dimensional point in the entropy decoding or the multi-valuedization of the information related to the second three-dimensional point.
[0262] For example, the first context is suitable for inter-frame prediction, and the second context is suitable for intra-frame prediction. Here, there is a tendency to use inter-frame prediction for the point after the first direction component is switched and to use intra-frame prediction for other points. As a result, the decoding device can properly decode the bit stream with improved coding efficiency. In addition, by improving the coding efficiency of the bit stream, the amount of data processed by the decoding device can be reduced.
[0263] For example, the first direction component is a horizontal direction component.
[0264] For example, the decoding device decodes a plurality of three-dimensional points each having position information including a distance component, a first direction component, and a second direction component (e.g., a horizontal angle component and an elevation angle component). When the first three-dimensional point corresponds to a leaf node in a prediction tree, the decoding device determines at least one of an entropy decoding method and a multi-valued method for information related to the first three-dimensional point based on history information obtained based on information related to the plurality of decoded three-dimensional points.
[0265] Thus, the decoding device can select an appropriate method using the history information, and thus can appropriately decode the bit stream with improved coding efficiency. In addition, by improving the coding efficiency of the bit stream, the amount of data to be processed by the decoding device can be reduced.
[0266] For example, when the first three-dimensional point does not correspond to the leaf node, the decoding device determines at least one of the entropy decoding method and the multi-valued method of the information related to the first three-dimensional point based on the number of second three-dimensional points (e.g., a count value) whose differences between the first direction component and the second direction component and the first three-dimensional point are below a threshold.
[0267] As mentioned above, the three-dimensional data encoding device (encoding device) and the three-dimensional data decoding device (decoding device) etc. according to the embodiment and the modification of this disclosure have been described, but this disclosure is not limited to these embodiments.
[0268] In addition, each processing unit included in the three-dimensional data encoding device and three-dimensional data decoding device involved in the above-mentioned embodiment is typically implemented by an LSI which is an integrated circuit. It can be a single chip individually or in a manner that includes part or all of them as a single chip.
[0269] Furthermore, the integrated circuit is not limited to LSI, and can be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI can also be used.
[0270] In addition, in each of the above-mentioned embodiments, each component may be formed by dedicated hardware, or implemented by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory.
[0271] In addition, the present disclosure can also be implemented as a three-dimensional data encoding method (encoding method) or a three-dimensional data decoding method (decoding method) executed by a three-dimensional data encoding device (encoding device) and a three-dimensional data decoding device (decoding device).
[0272] In addition, the division of the functional modules in the block diagram is an example, and multiple functional modules can also be implemented as one functional module, or one functional module can be divided into multiple modules, or a part of the function can be transferred to other functional modules. In addition, the functions of multiple functional modules with similar functions can also be processed in parallel or in time-sharing by a single hardware or software.
[0273] In addition, the order in which each step in the flowchart is executed is an example order for specifically explaining the present disclosure, and may be an order other than the above. In addition, part of the above steps may be executed simultaneously (in parallel) with other steps.
[0274] In the above, a three-dimensional data encoding device and a three-dimensional data decoding device, etc., according to one or more embodiments are described based on the embodiments, but the present disclosure is not limited to the embodiments. As long as it does not deviate from the main purpose of the present disclosure, the embodiments obtained by applying various modifications that can be thought of by those skilled in the art to the present embodiment, and the embodiments constructed by combining the constituent elements in different embodiments may also be included in the scope of one or more embodiments.
[0275] Industrial Applicability
[0276] The present disclosure can be applied to a three-dimensional data encoding device and a three-dimensional data decoding device.
[0277] Description of reference numerals:
[0278] 100 Three-dimensional data encoding device
[0279] 102 Subtraction Department
[0280] 103 Quantitative Department
[0281] 104 Entropy Coding Unit
[0282] 105, 202 Inverse Quantization Unit
[0283] 106, 203 Addition Department
[0284] 108, 110, 205, 207 buffers
[0285] 109, 206 Intra-frame prediction unit
[0286] 111 Motion detection compensation unit
[0287] 112, 209 Inter-frame prediction unit
[0288] 113, 210 Switching Unit
[0289] 200 Three-dimensional data decoding device
[0290] 201 Entropy Decoding Department
[0291] 208 Motion Compensation Unit
Claims
1. A decoding method for decoding a plurality of encoded three-dimensional points each having position information, wherein the position information includes a distance component, a first direction component, and a second direction component, wherein: determining at least one of an entropy decoding method and a multi-valued method for information related to the first encoded 3D point according to the number of second 3D points having the first directional component and the second directional component substantially equal to the first directional component and the second directional component of the first encoded 3D point included in the plurality of encoded 3D points among the decoded 3D points, A process using the determined at least one method is performed.
2. The decoding method according to claim 1, In the determining, determining a context to be used in arithmetic decoding, In the process, the information is arithmetically decoded using the determined context.
3. The decoding method according to claim 1, The information indicates which of the inter prediction mode and the intra prediction mode is used.
4. The decoding method according to claim 1, The information indicates a prediction residual of the first direction component or the second direction component.
5. The decoding method according to claim 4, The prediction residual is information obtained by quantizing the prediction residual of the horizontal angle component.
6. The decoding method according to claim 1, The number is the number of the second three-dimensional points having the quantized first direction component and the quantized second direction component substantially equal to the quantized first direction component and the quantized second direction component of the first encoded three-dimensional point.
7. The decoding method according to claim 1, In the determination, the number is limited by a predetermined upper limit value, and the at least one method is determined based on the limited number.
8. The decoding method according to claim 1, In the determination, the number of the second three-dimensional points is quantized, and the at least one method is determined based on the quantized number.
9. The decoding method according to claim 1, In the said decision, When the number is not within a predetermined range, determining the at least one method according to the number, When the number is within the predetermined range, the at least one method is determined based on history information obtained based on the information related to the plurality of decoded three-dimensional points.
10. The decoding method according to claim 9, The history information is the number or pattern of 0 or 1 of the information of the plurality of three-dimensional points that have been decoded.
11. The decoding method according to claim 9, The history information is updated by the information of the three-dimensional points whose number is a predetermined value and whose number is decoded is not updated by the information of the three-dimensional points whose number is other than the predetermined value and whose decoded is completed.
12. The decoding method according to claim 1, The information indicates whether there is a three-dimensional point corresponding to the reference position.
13. A coding method for coding a plurality of three-dimensional points each having position information, wherein the position information includes a distance component, a first direction component and a second direction component, wherein: Determining at least one of an entropy encoding method and a binarization method for information related to the first three-dimensional point based on the number of second three-dimensional points among the encoded three-dimensional points having the first direction component and the second direction component substantially equal to the first direction component and the second direction component of the first three-dimensional point included in the plurality of three-dimensional points, A process using the determined at least one method is performed.
14. A decoding device for decoding a plurality of encoded three-dimensional points each having position information, wherein the position information includes a distance component, a first direction component, and a second direction component, the decoding device comprising: Processor; and Memory, The processor uses the memory, determining at least one of an entropy decoding method and a multi-valued method for information related to the first encoded 3D point according to the number of second 3D points having the first directional component and the second directional component substantially equal to the first directional component and the second directional component of the first encoded 3D point included in the plurality of encoded 3D points among the decoded 3D points, A process using the determined at least one method is performed.
