Point cloud reconstruction method and device and related equipment

Abstract

The invention discloses a point cloud reconstruction method and device and related equipment, and belongs to the technical field of computers.The point cloud reconstruction method comprises the steps that point cloud information of a first node in a first frame at a node corresponding to a second frame is obtained, and the second frame is a reference frame of the first frame; and determining the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

Description

Technical Field

[0001] This application belongs to the technical field of computer technology, and particularly relates to a point cloud reconstruction method, apparatus and related devices. Background Art

[0002] For point cloud encoding, in some related technologies, inter-frame encoding is mainly adopted, and the inter-frame encoding adopted needs to encode each node in the frame to obtain the point cloud information of each node. Since the point cloud information of each node is obtained through encoding, in addition to including the point cloud information, the code stream also needs to include the encoding code stream of each node. This results in a large code stream. Summary of the Invention

[0003] Embodiments of this application provide a point cloud reconstruction method, apparatus and related devices, which can solve the problem of a large code stream.

[0004] In a first aspect, a point cloud reconstruction method is provided, including:

[0005] Obtain the point cloud information of the node corresponding to the first node in the first frame in the second frame, where the second frame is the reference frame of the first frame;

[0006] Determine the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

[0007] In a second aspect, a point cloud reconstruction method is provided, including:

[0008] The decoding end determines the mode of the first node of the first frame based on the code stream;

[0009] The decoding end obtains the point cloud information of the node corresponding to the first node in the second frame based on the mode of the first node, where the second frame is the reference frame of the first frame;

[0010] The decoding end determines the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

[0011] In a third aspect, a point cloud reconstruction apparatus is provided, including:

[0012] An obtaining module, configured to obtain the point cloud information of the node corresponding to the first node in the first frame in the second frame, where the second frame is the reference frame of the first frame;

[0013] A first determining module, configured to determine the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

[0014] In a fourth aspect, a point cloud reconstruction apparatus is provided, including:

[0015] A first determination module, configured to determine the mode of the first node of the first frame based on the bitstream;

[0016] An acquisition module, configured to acquire the point cloud information of the node corresponding to the first node in the second frame based on the mode of the first node, where the second frame is a reference frame of the first frame;

[0017] A second determination module, configured to determine the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

[0018] In a fifth aspect, an electronic device is provided. The terminal includes a processor and a memory. The memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented.

[0019] In a sixth aspect, an electronic device is provided, including a processor and a communication interface. The processor is configured to acquire the point cloud information of the node corresponding to the first node in the first frame in the second frame, where the second frame is a reference frame of the first frame; a first determination module, configured to determine the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node. Alternatively, the processor is configured to determine the mode of the first node of the first frame based on the bitstream; acquire the point cloud information of the node corresponding to the first node in the second frame based on the mode of the first node, where the second frame is a reference frame of the first frame; a second determination module, configured to determine the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node

[0020] In a seventh aspect, an electronic device is provided, including: a memory configured to store video data, and a processing circuit configured to implement the steps of the method described in the first aspect, or implement the steps of the method described in the second aspect.

[0021] In an eighth aspect, a readable storage medium is provided. A program or instruction is stored on the readable storage medium. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented, or the steps of the method described in the second aspect are implemented.

[0022] In a ninth aspect, an encoding and decoding system is provided, including: an encoding end device and a decoding end device. The encoding end device can be used to execute the steps of the method described in the first aspect, and the decoding end device can be used to execute the steps of the method described in the second aspect.

[0023] In a tenth aspect, a chip is provided, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run programs or instructions to implement the steps of the method described in the first aspect or the steps of the method described in the second aspect.

[0024] In an eleventh aspect, a computer program / program product is provided. The computer program / program product is stored in a storage medium and is executed by at least one processor to implement the steps of the method described in the first aspect or the steps of the method described in the second aspect.

[0025] In an embodiment of the present application, the point cloud information of the node corresponding to the first node in the second frame is obtained from the first frame, where the second frame is a reference frame of the first frame; the point cloud information of the node corresponding to the first node in the second frame is determined as the point cloud information of the first node. In this way, since the point cloud information of the first node is the point cloud information of the node corresponding to the first node in the second frame, there is no need to encode the first node, thereby saving the encoding bitstream of the first node to reduce the bitstream size. In addition, since there is no need to encode the first node, the encoding efficiency can also be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a schematic diagram of an encoding and decoding system provided by an embodiment of the present application;

[0027] Figure 2a is a flowchart of encoding executed by an encoder provided by an embodiment of the present application;

[0028] Figure 2b is another flowchart of encoding executed by an encoder provided by an embodiment of the present application;

[0029] Figure 3a is a flowchart of decoding executed by a decoder provided by an embodiment of the present application;

[0030] Figure 3b is another flowchart of decoding executed by a decoder provided by an embodiment of the present application;

[0031] Figure 4 is a schematic diagram of module division provided by an embodiment of the present application;

[0032] Figure 5 is a schematic diagram of encoding a motion vector provided by an embodiment of the present application;

[0033] Figure 6 is a schematic diagram of motion search provided by an embodiment of the present application;

[0034] Figure 7It is a flowchart of a point cloud reconstruction method provided by an embodiment of the present application;

[0035] Figure 8 It is a schematic diagram of a common-edge node provided by an embodiment of the present application;

[0036] Figure 9 It is a flowchart of a point cloud reconstruction method provided by an embodiment of the present application;

[0037] Figure 10 It is a schematic diagram of a point cloud reconstruction method provided by an embodiment of the present application;

[0038] Figure 11 It is a schematic diagram of another point cloud reconstruction method provided by an embodiment of the present application;

[0039] Figure 12 It is a structural diagram of a point cloud reconstruction device provided by an embodiment of the present application;

[0040] Figure 13 It is a structural diagram of another point cloud reconstruction device provided by an embodiment of the present application;

[0041] Figure 14 It is a structural diagram of an electronic device provided by an embodiment of the present application;

[0042] Figure 15 It is a structural diagram of a terminal provided by an embodiment of the present application. Detailed implementation manners

[0043] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope protected by the present application.

[0044] The terms "first", "second", etc. in the present application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first" and "second" are usually of the same type, and the number of objects is not limited. For example, the first object can be one or multiple. In addition, "or" in the present application means at least one of the connected objects. For example, "A or B" covers three scenarios, namely, Scenario 1: including A and not including B; Scenario 2: including B and not including A; Scenario 3: including both A and B. The character " / " generally indicates an "or" relationship between the associated objects before and after.

[0045] Before introducing the technical solutions provided by the embodiments of this application, the meanings of some terms are introduced first.

[0046] Point Cloud: A point cloud refers to a set of discrete points in space that are irregularly distributed and represent the spatial structure and surface attributes of a three-dimensional object or a three-dimensional scene. Point clouds can be classified into different categories according to different classification criteria. For example, according to the acquisition method of the point cloud, they can be divided into dense point clouds and sparse point clouds; another example is that according to the temporal type of the point cloud, they can be divided into static point clouds and dynamic point clouds.

[0047] Point Cloud Data: The geometric coordinate information and attribute information of each point in the point cloud together constitute the point cloud data. Among them, the geometric coordinate information can also be called three-dimensional position information. The geometric coordinate information of a certain point in the point cloud refers to the spatial coordinates (x, y, z) of the point, which can include the coordinate values of the point in the directions of each coordinate axis in the three-dimensional coordinate system. For example, the coordinate value x in the X-axis direction, the coordinate value y in the Y-axis direction, and the coordinate value z in the Z-axis direction. The attribute information of a certain point in the point cloud can include at least one of the following: color information, material information, and laser reflection intensity information (which can also be called reflectivity). Usually, each point in the point cloud has the same number of attribute information. For example, each point in the point cloud can have two attribute information, namely color information and laser reflection intensity, or each point in the point cloud can have three attribute information, namely color information, material information, and laser reflection intensity information.

[0048] Point Cloud Compression (PCC): Point cloud compression refers to the process of encoding the geometric coordinate information and attribute information of each point in the point cloud to obtain a compressed bitstream. Point cloud compression can include two main processes: geometric coordinate information encoding and attribute information encoding. Currently, the point cloud compression framework that can compress point clouds can be the geometry-based point cloud compression (G-PCC) codec framework provided by the Moving Picture Experts Group (MPEG), or the video-based point cloud compression (V-PCC) codec framework, or the AVS-PCC codec framework provided by the Audio Video Standard (AVS).

[0049] Point cloud decoding: Point cloud decoding refers to the process of decoding the compressed bitstream obtained from point cloud encoding to reconstruct the point cloud. Specifically, it refers to the process of reconstructing the geometric coordinate information and attribute information of each point in the point cloud based on the geometric bitstream and attribute bitstream in the compressed bitstream. After obtaining the compressed bitstream at the decoding end, for the geometric bitstream, entropy decoding is first performed to obtain the quantized information of each point in the point cloud, and then inverse quantization is performed to reconstruct the geometric coordinate information of each point in the point cloud. For the attribute bitstream, entropy decoding is first performed to obtain the quantized attribute residual information or quantized transform coefficients of each point in the point cloud; then inverse quantization is performed on the quantized attribute residual information to obtain the reconstructed residual information, inverse quantization is performed on the quantized transform coefficients to obtain the reconstructed transform coefficients, and the reconstructed transform coefficients are inverse-transformed to obtain the reconstructed residual information. Based on the reconstructed residual information of each point in the point cloud, the attribute information of each point in the point cloud can be reconstructed. The reconstructed attribute information of each point in the point cloud is sequentially corresponded to the reconstructed geometric coordinate information one by one to reconstruct the point cloud.

[0050] Figure 1 It is a schematic diagram of the codec system 10 provided by an embodiment of the present application. The technical solution of the embodiment of the present application relates to encoding and decoding (CODEC) (including encoding or decoding) point cloud data.

[0051] As Figure 1 shown, the codec system 10 includes a source device 100, and the source device 100 provides encoded point cloud data to be decoded and displayed by a destination device 110. Specifically, the source device 100 provides point cloud data to the destination device 110 via a communication medium 120. The source device 100 and the destination device 110 may include any one or more of a desktop computer, a notebook (i.e., laptop) computer, a tablet computer, a set-top box, a mobile phone, a wearable device (such as a smart watch or a wearable camera), a television, a camera, a display device, a vehicle-mounted device, a virtual reality (VR) device, an augmented reality (AR) device, a mixed reality (MR) device, a digital media player, a video game console, a video conferencing device, a video streaming device, a broadcast receiver device, a broadcast transmitter device, a spacecraft, an airplane, a robot, a satellite, etc.

[0052] In Figure 1 the example, the source device 100 includes a data source 101, a memory 102, an encoder 200, and an output interface 104. The destination device 110 includes an input interface 111, a decoder 300, a memory 113, and a display device 114. The source device 100 represents an example of an encoding device, and the destination device 110 represents an example of a decoding device. In other examples, the source device 100 and the destination device 110 may not include Figure 1Some components in, or may also include Figure 1 Other components outside. For example, the source device 100 can obtain point cloud data through an external capture device. Similarly, the destination device 110 can be interfaced with an external display device without including an integrated display device. For another example, the memories 102 and 113 can be external memories.

[0053] Although Figure 1 The source device 100 and the destination device 110 are depicted as separate devices, but in some examples, the two can also be integrated into one device. In such embodiments, the functions corresponding to the source device 100 and the functions corresponding to the destination device 110 can be implemented using the same hardware or software, or using separate hardware or software, or any combination thereof.

[0054] In some examples, the source device 100 and the destination device 110 can perform unidirectional data transmission or bidirectional data transmission. If it is bidirectional data transmission, the source device 100 and the destination device 110 can operate in a substantially symmetric manner, that is, each of the source device 100 and the destination device 110 includes an encoder and a decoder.

[0055] The data source 101 represents the source of the point cloud data (i.e., the original, uncoded point cloud data) and provides the point cloud data to the encoder 200, and the encoder 103 encodes the point cloud data. The source device 100 can include a capture device (such as a camera device, a sensing device, or a scanning device), an archive including previously captured point cloud data, or a feed interface for receiving point cloud data from a data content provider. Among them, the camera device can include a common camera, a stereo camera, a light field camera, etc., the sensing device can include a laser device, a radar device, etc., and the scanning device can include a three-dimensional laser scanning device, etc. Point cloud data can be obtained by collecting the visual scene of the real world through the capture device. Alternatively, the data source 101 can generate computer graphics-based data as the source data, or combine real-time data, archived data, and computer-generated data. For example, the data source generates point cloud data according to virtual objects (such as virtual three-dimensional objects and virtual three-dimensional scenes obtained through three-dimensional modeling).

[0056] The encoder 200 encodes the captured, pre-captured, or computer-generated data. The encoder 200 can rearrange the point cloud data from the received order (sometimes referred to as the "display order") into the encoding order. The encoder 200 can generate a bitstream including the encoded point cloud data. The source device 100 can then output the encoded point cloud data to the communication medium 120 via the output interface 104 for reception or retrieval by, for example, the input interface 111 of the destination device 110.

[0057] The memory 102 of the source device 100 and the memory 113 of the destination device 110 represent general memories. In some examples, the memory 102 may store the original data from the data source 101, and the memory 113 may store the decoded point cloud data from the decoder 300. Additionally or alternatively, the memories 102, 113 may store software instructions that can be executed by, for example, the encoder 200 and the decoder 300, respectively. Although the memory 102 and the memory 113 are shown separately from the encoder 200 and the decoder 300 in this example, it should be understood that the encoder 200 and the decoder 300 may also include internal memories for functionally similar or equivalent purposes. If the encoder 200 and the decoder 300 are deployed on the same hardware device, the memory 102 and the memory 113 may be the same memory. Furthermore, the memories 102, 113 may store, for example, the encoded point cloud data output from the encoder 200 and input to the decoder 300. In some examples, portions of the memories 102, 113 may be allocated as one or more point cloud buffers, for example, for storing the original, decoded, or encoded point cloud data.

[0058] In some examples, the source device 100 may output the encoded data from the output interface 104 to the memory 113. Similarly, the destination device 110 may access the encoded data from the memory 113 via the input interface 111. The memory 113 or the memory 102 may include any one of various distributed or local access data storage media, such as a hard drive, a Blu-ray disc, a Digital Versatile Disc (DVD), a Compact Disc Read-Only Memory (CD-ROM), a flash memory, a volatile or non-volatile memory, or any other suitable digital storage media for storing the encoded point cloud data.

[0059] The output interface 104 may include any type of medium or device capable of sending the encoded point cloud data from the source device 100 to the destination device 110. For example, the output interface 104 may include a transmitter or transceiver, such as an antenna, configured to directly and in real time send the encoded point cloud data from the source device 100 to the destination device 110. The encoded point cloud data may be modulated according to the communication standard of a wireless communication protocol and sent to the destination device 110.

[0060] The communication medium 120 may include a transient medium, such as a wireless broadcast or a wired network transmission. For example, the communication medium 120 may include a radio frequency (RF) spectrum or one or more physical transmission lines (e.g., cables). The communication medium 120 may form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium 120 may also take the form of a storage medium (e.g., a non-transitory storage medium), such as a hard disk, a flash drive, a compact disc, a digital versatile disc, a Blu-ray disc, a volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded point cloud data.

[0061] In some embodiments, the communication medium 120 may include a router, a switch, a base station, or any other device that may be used to facilitate communication from the source device 100 to the destination device 110. For example, a server (not shown) may receive encoded point cloud data from the source device 100 and provide it to the destination device 110, e.g., via a network transmission to the destination device 110. The server may include, for example, a web server (for a website), a server configured to provide a file transfer protocol service (such as the File Transfer Protocol (FTP) or the File Delivery Over Unidirectional Transport (FLUTE) protocol), a content delivery network (CDN) device, a Hypertext Transfer Protocol (HTTP) server, a Multimedia Broadcast Multicast Services (MBMS) or evolved Multimedia Broadcast Multicast Service (eMBMS) server, or a Network-attached storage (NAS) device, etc. The server may implement one or more HTTP streaming protocols, such as the MPEG Media Transport (MMT) protocol, the Dynamic Adaptive Streaming over HTTP (DASH) protocol, the HTTP Live Streaming (HLS) protocol, or the Real Time Streaming Protocol (RTSP), etc.

[0062] The destination device 110 can access the encoded point cloud data from the server, for example, via a wireless channel (such as a Wi-Fi connection) or a wired connection (such as a Digital Subscriber Line (DSL), cable modem, etc.) for accessing the encoded point cloud data stored on the server.

[0063] The output interface 104 and the input interface 111 can represent a wireless transmitter / receiver, a modem, a wired networking component (such as an Ethernet card), a wireless communication component operating according to the IEEE 802.11 standard or the IEEE 802.15 standard (such as ZigBeeTM), the Bluetooth standard, etc.), or other physical components. In an example where the output interface 104 and the input interface 111 include wireless components, the output interface 104 and the input interface 111 can be configured to transmit data, such as the encoded point cloud data, according to WIFI, Ethernet, a cellular network (such as 4G, LTE (Long Term Evolution), advanced LTE, 5G, 6G, etc.).

[0064] The technology provided by the embodiments of the present application can be applied to support one or more of the following application scenarios: machine perception of point clouds, which can be used in scenarios such as autonomous navigation systems, real-time inspection systems, geographic information systems, vision sorting robots, disaster relief robots, etc.; human eye perception of point clouds, which can be used in point cloud application scenarios such as digital cultural heritage, free viewpoint broadcasting, three-dimensional immersive communication, three-dimensional immersive interaction, etc.

[0065] The input interface 111 of the destination device 110 receives the encoded bitstream from the communication medium 120. The encoded bitstream can include high-level syntax elements and encoded data units (such as sequences, groups of pictures, pictures, slices, blocks, etc.), where the high-level syntax elements are used to decode the encoded data units to obtain the decoded point cloud data. The display device 114 displays the decoded point cloud data to the user. The display device 114 can include a cathode ray tube (CRT), a liquid-crystal display (LCD), a plasma display, an organic light-emitting diode (OLED) display, or other types of display devices. In some examples, the destination device 110 may not have the display device 114. For example, if the decoded point cloud data is used to determine the position of a physical object, the display device 114 can be replaced by a processor.

[0066] The encoder 200 and the decoder 300 can be implemented as one or more of various processing circuits, which may include a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), discrete logic, hardware, or any combination thereof. When all or part of the technology is implemented in software, the device can store instructions for the software in a suitable non-transitory computer-readable storage medium and execute the instructions in hardware using one or more processors to perform the technology provided by the embodiments of the present application.

[0067] Taking the G-PCC and AVS-PCC encoding and decoding frameworks as examples, the basic principles of the encoder 200 and the decoder 300 provided by the embodiments of the present application are introduced below.

[0068] The encoding and decoding frameworks of G-PCC and AVS-PCC are roughly the same. As Figure 2a shows the encoding flowchart executed by the encoder of the encoding framework based on AVS-PCC, as Figure 2b shows the encoding flowchart executed by the encoder of the encoding framework based on MPEG G-PCC. The above encoder can be Figure 1 the encoder 200 shown. Generally, the above encoding frameworks can be divided into a geometric coordinate information encoding process and an attribute information encoding process. In the geometric information encoding process, the geometric coordinate information of each point in the point cloud is encoded to obtain a geometric bitstream; in the attribute information encoding process, the attribute information of each point in the point cloud is encoded to obtain an attribute bitstream; the geometric bitstream and the attribute bitstream together form the compressed bitstream of the point cloud.

[0069] For the geometric information encoding process, the encoding flow executed by the encoder 200 is as follows:

[0070] 1. Pre-Processing: It may include Transform Coordinates and Voxelize. Through operations of scaling and translation, the pre-processing converts the point cloud data in three-dimensional space into an integer form and moves its minimum geometric position to the origin of coordinates. In some examples, the encoder 200 may not perform pre-processing.

[0071] 2. Geometric Encoding: For the AVS-PCC coding framework, geometric encoding includes two modes, namely octree-based geometric encoding and prediction tree-based geometric encoding. For the G-PCC coding framework, geometric encoding includes three modes, namely octree-based geometric encoding, triangle representation (Trisoup)-based geometric encoding, and prediction tree-based prediction encoding. Among them:

[0072] Octree-based Geometric Encoding: An octree is a tree-shaped data structure that evenly divides a pre-set bounding box in three-dimensional space, and each node has eight child nodes. By using "1" and "0" to indicate whether each child node of the octree is occupied or not, occupancy code information (Occupancy Code) is obtained as the bitstream of the point cloud geometric information.

[0073] Prediction Tree-based Geometric Encoding: A prediction tree is generated using a prediction strategy. Starting from the root node of the prediction tree, each node is traversed, and the residual coordinate values corresponding to each traversed node are encoded.

[0074] Triangle Representation-based Geometric Encoding: The point cloud is divided into blocks of a certain size, and the intersection points (referred to as vertices) of the point cloud surface at the edges of the blocks are located. Geometric information compression is achieved by encoding whether there are intersection points on each edge of the block and the positions of the intersection points.

[0075] 3. Geometric Entropy Encoding: Statistical compression encoding is performed on the occupancy code information of the octree, the prediction residual information of the prediction tree, and the vertex information of the triangle representation, and finally a binary (0 or 1) compressed bitstream is output. Statistical encoding is a lossless encoding method that can effectively reduce the bit rate required to represent the same signal. The commonly used statistical encoding method is context-based binary arithmetic coding (Content Adaptive Binary Arithmetic Coding, CABAC).

[0076] 4. Geometric Reconstruction: The geometric information after geometric encoding is decoded and reconstructed.

[0077] For the attribute information encoding process, the encoding process executed by the encoder 200 is as follows:

[0078] 1. Color Transformation: A transformation is applied to transform the color information of the attribute to a different domain. For example, the color information can be transformed from the RGB color space to the YCbCr color space.

[0079] 2. Attribute Recoloring: In the case of lossy coding, after encoding the geometric coordinate information, the encoder needs to decode and reconstruct the geometric information, that is, restore the geometric information of each point in the point cloud. Find the attribute information corresponding to one or more neighboring points in the original point cloud as the attribute information of the reconstructed point.

[0080] In some examples, the encoder 200 may not perform color transformation or attribute recoloring.

[0081] 3. Attribute Information Processing: In AVS-PCC, attribute information processing can include three modes, namely Prediction coding, Transform coding, and Prediction&Transform coding. These three coding modes can be used under different conditions.

[0082] Among them, Prediction coding means that according to information such as distance or spatial relationship, determine the neighboring points of the point to be encoded as the prediction points among the already encoded points. Based on the set criteria, calculate the predicted attribute information of the point to be encoded according to the attribute information of the prediction points. Calculate the difference between the true attribute information and the predicted attribute information of the point to be encoded as the attribute residual information, and perform quantization, transformation (optional), and entropy coding on the attribute residual information.

[0083] Transform coding means using transformation methods such as Discrete Cosine Transform (DCT) and Haar Transform (Haar) to group and transform the attribute information, quantize the transform coefficients; obtain the attribute reconstruction information through inverse quantization and inverse transformation; calculate the difference between the true attribute information and the attribute reconstruction information to obtain the attribute residual information and quantize it; perform entropy coding on the quantized transform coefficients and attribute residuals.

[0084] Prediction&Transform coding means using the attribute residual information obtained by prediction for transformation, and performing quantization and entropy coding on the transform coefficients.

[0085] In MPEG G-PCC, attribute information processing can include three modes, namely PredictionTransform coding, Lifting Transform coding, and RegionAdaptive Hierarchical Transform (RAHT) coding. These three coding modes can be used under different conditions.

[0086] Among them, predictive transform coding refers to selecting a subset of sub - points according to distance, dividing the point cloud into multiple different levels of detail (LoD), and realizing a multi - quality level point cloud representation from rough to refined. Prediction can be achieved from bottom to top between adjacent layers, that is, the attribute information of the points introduced in the fine layer is predicted by the neighboring points in the rough layer to obtain the corresponding attribute residual information. Among them, the points in the bottom - most layer are used as reference information for coding.

[0087] Enhanced transform coding refers to introducing a weight update strategy for neighboring points on the basis of LoD adjacent - layer prediction, and finally obtaining the predicted attribute information of each point to obtain the corresponding attribute residual information.

[0088] Hierarchical region - adaptive transform coding means that the attribute information undergoes the RAHT transform to convert the signal into the transform domain, which is called the transform coefficient.

[0089] 4. Attribute Quantization: The fineness of quantization is usually determined by the quantization parameter. The transform coefficients or attribute residual information obtained by processing the attribute information are quantized, and the quantized results are entropy - coded. For example, in predictive transform coding and enhanced transform coding, the quantized attribute residual information is entropy - coded; in RAHT, the quantized transform coefficients are entropy - coded.

[0090] 5. Entropy Coding: The quantized attribute residual information and / or transform coefficients generally use run - length coding (RLC) and arithmetic coding (AC) to achieve final compression. Information such as the corresponding coding mode and quantization parameter is also encoded using an entropy encoder.

[0091] The encoder 200 encodes the geometric coordinate information of each point in the point cloud to obtain a geometric bitstream, and encodes the attribute information of each point in the point cloud to obtain an attribute bitstream. The encoder 200 can transmit the encoded geometric bitstream and attribute bitstream to the decoder 300 together.

[0092] Figure 3a shows the decoding flow chart executed by the decoder of the decoding framework based on AVS - PCC, as Figure 3b shows the decoding flow chart executed by the decoder of the decoding framework based on MPEG G - PCC. The above - mentioned decoder can be Figure 1The decoder 300 shown. After receiving the compressed bitstreams (i.e., the attribute bitstream and the geometry bitstream) transmitted by the encoder 200, the decoder 300 decodes the geometry bitstream to reconstruct the geometric coordinate information of each point in the point cloud, and decodes the attribute bitstream to reconstruct the attribute information of each point in the point cloud.

[0093] The decoding process executed by the decoder 300 is as follows:

[0094] 1. Entropy Decoding: Entropy decode the geometry bitstream and the attribute bitstream respectively to obtain geometric syntax elements and attribute syntax elements.

[0095] 2. Geometry Decoding: For the AVS-PCC coding framework, geometry decoding includes two modes, namely octree-based geometry decoding and prediction-tree-based geometry decoding. For the G-PCC coding framework, geometry coding includes three modes, namely octree-based geometry decoding, triangle representation (Trisoup)-based geometry decoding, and prediction decoding based on a prediction tree.

[0096] Octree-based Geometry Decoding: Reconstruct an octree based on the geometric syntax elements parsed from the geometry bitstream.

[0097] Prediction-tree-based Geometry Decoding: Reconstruct a prediction tree based on the geometric syntax elements parsed from the geometry bitstream.

[0098] Triangle Representation-based Geometry Decoding: Reconstruct a triangle model based on the geometric syntax elements parsed from the geometry bitstream.

[0099] 3. Geometry Reconstruction: Perform reconstruction to obtain the geometric coordinate information of the points in the point cloud.

[0100] 4. Coordinate Inverse Transformation: Perform an inverse transformation on the reconstructed geometric coordinate information to transform the reconstructed coordinates (positions) of the points in the point cloud from the transformed domain back to the initial domain.

[0101] 5. Inverse Quantization: Perform inverse quantization on the attribute syntax elements.

[0102] 6. Attribute Information Processing: In AVS-PCC, attribute information processing determines the color information of the points in the point cloud through prediction or prediction transformation for the inverse quantized prediction residuals or prediction residual transform coefficients, or determines the color information of the points in the point cloud through transformation for the inverse quantized transform coefficients.

[0103] In MPEG G-PCC, attribute information processing determines the color information of the points in the point cloud through RAHT for the inverse quantized attribute information, or determines the color information of the points in the point cloud through LOD and inverse lifting for the inverse quantized attribute information.

[0104] 7. Color inverse transformation: Transform color information from the YCbCr color space to the RGB color space. In some examples, the color inverse transformation operation may not be performed.

[0105] In some embodiments, local motion estimation may be as follows:

[0106] For non-radar dense point clouds, G-PCC only performs local motion estimation on them. Whether to enable local motion estimation for a certain layer is determined by the localMotionEnabled flag in the Geometry parameter set (GPS) layer. Local motion estimation is based on blocks (prediction units) for inter-frame prediction. First, read the size of the Largest Prediction Unit (LPU) (LPUsize) and the number of layers for block prediction from the configuration parameters, and calculate the size of the minimum LPU (minLPU) (minLPUsize);

[0107] When the current layer node size > LPUsize, there is no motion vector to perform motion compensation on the reference point cloud. Therefore, directly use the occupancy information of the reference point cloud (without motion compensation) as the inter-frame prediction context.

[0108] When the current layer node size = LPUsize, write a recursive prediction unit structure (PU_tree). Each node can continue to be divided downward to use the motion vector of the child node PU to perform motion compensation on the reference point cloud, or directly use the motion vector of the current undivided node to perform motion compensation on the reference point cloud. The PU_tree records the flag bit (split_flag) indicating whether to divide downward, the flag indicating whether to perform compensation at the current layer (isCompensated), and the motion vector set (MVs); if the node is divided to minLPUsize, terminate further division (split_flag = 0), and perform motion compensation (isCompensated = 1). Finally, according to the flag of whether to compensate, decide to select the occupancy information of the reference point cloud or the compensated point cloud as the inter-frame prediction context. Among them, the LPU / PU division can be as Figure 4 shown.

[0109] In some embodiments, the best motion vector selection and coding may be as Figure 5 shown.

[0110] Among them, the motion search motion vector (Motion Vector, MV) (that is, how to obtain the motion vector MV) may be as follows:

[0111] Motion Estimation Criterion: Take the logarithm of the sum of the absolute values of the differences between each point in the predicted block and the block to be encoded (Manhattan distance) as the matching metric

[0112] D(B, P) = ∑ b∈B log2(1 + min p∈P ‖b - p‖1) + ∑ b∈B log2(1 + min p∈P ‖b color - p color ‖1) * k

[0113] where D(B, P) represents the motion estimation result, B (Block) represents the current block, P (Prediction block) represents the prediction block, and k is a coefficient.

[0114] Search Algorithm: Within the search window, starting from the position of the reference node, search for the best 2 motion vectors in 18 surrounding directions. By iteratively searching the step size, continuously narrow the search distance, and finally obtain the best motion vector. In addition, the motion search schematic diagram can be as Figure 6 shown.

[0115] After that, encode the MV.

[0116] Based on rate - distortion optimization (RDO), whether to divide the PU downwards can be as follows:

[0117] The flag indicating whether to divide downwards (split_flag) is determined based on the total cost (Cost) of the distortion between the reference node and the current node, the encoded MV, and the encoded split_flag (when split_flag = 0, the reference node is not compensated, otherwise, the reference node is compensated).

[0118] The calculation process of Cost can be as follows:

[0119] The flag split_flag indicating whether to divide downwards and different PU motion vectors (MVs) are both optional coding parameters. Using a specific set of coding parameters (for example, when not dividing down split_flag = 0 and selecting motion vector MV1), the bitrate and distortion under this condition, that is, the rate - distortion performance (R, D), can be obtained. In order to find the coding parameters with the minimum distortion (D) under a certain bitrate constraint (R), the Lagrange factor is introduced to calculate the following cost:

[0120] C = ∑i[D(B, P(W, Vi)) + λR(Vi)] + λR(split flags) + λR(pop flags)

[0121] Among them, i represents, P(W, Vi) represents, λ represents, R(Vi) represents, R(split flags) represents, and R(popflags) represents.

[0122] By comparing the costs of splitting downwards or not, the optimal motion vector and the split_flag flag of the current PU block are determined.

[0123] It should be noted that local motion is used as an example for illustration, and the specific compensation method of local motion is not limited in the embodiments of the present application.

[0124] The following introduces the point cloud reconstruction method provided by the embodiments of the present application with reference to the accompanying drawings. The point cloud reconstruction method provided by the embodiments of the present application can be executed by an encoding end or a decoding end. For example Figure 1 the encoder 200 shown, for example Figure 1 the decoder 300 described. Among them, the encoding end and the decoding end can be implemented by software, hardware, or a combination thereof. When implemented by hardware, the encoding end can be referred to as an encoding end device or a video encoding device, and the decoding end can be referred to as a decoding end device or a video decoding device.

[0125] Please refer to Figure 7 , Figure 7 which is a flowchart of a point cloud reconstruction method provided by the embodiments of the present application. As Figure 7 shown, it includes the following steps:

[0126] Step 701: Obtain the point cloud information of the node corresponding to the first node in the first frame in the second frame, where the second frame is the reference frame of the first frame;

[0127] Step 702: Determine the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

[0128] The above method can be applied to the encoding end or the decoding end, that is, the above method can be used for point cloud reconstruction during the encoding process, and the above method can also be used for point cloud reconstruction during the decoding process.

[0129] Among them, the above first frame can be understood as the current frame, and the above first node can be understood as the current node.

[0130] The reference frame of the above first frame can be an already encoded frame or an already decoded frame.

[0131] In some embodiments, the reference frame of the above first frame is the previous frame of the first frame, that is, the above second frame is the previous frame of the first frame. The previous frame is the frame before the above first frame and adjacent to the first frame. In this way, the point cloud information of the current frame can be determined based on the point cloud information of the previous frame, thereby improving the point cloud quality.

[0132] The corresponding node of the first node in the second frame may be the node in the second frame that has the same position as the first node and the same size. The first node and the corresponding node of the first node in the second frame are the same node in the first frame and the second frame. The corresponding node of the first node in the second frame may be referred to as a reference node.

[0133] The point cloud information of the corresponding node of the first node in the second frame is the point cloud information obtained by encoding the second frame, or the point cloud information of this node included in the bitstream of the second frame.

[0134] In the embodiments of the present application, a node may be an encoding block, such as a PU.

[0135] The obtaining of the point cloud information of the corresponding node of the first node in the second frame in the first frame may be to copy the node cloud information of this node, and the determining of the point cloud information of the corresponding node of the first node in the second frame as the point cloud information of the first node may be to add the copied point cloud information to the reconstructed point cloud. That is, steps 701 and 702 may be understood as directly copying the point cloud information of the corresponding node of the first node in the second frame into the reconstructed point cloud.

[0136] In the embodiments of the present application, since the point cloud information of the first node is the point cloud information of the corresponding node of the first node in the second frame, there is no need to encode the first node, thereby saving the encoding bitstream of the first node to reduce the bitstream size. In addition, since there is no need to encode the first node, the encoding efficiency can also be improved. In addition, during the decoding process, since the point cloud information of the corresponding node of the first node in the second frame is directly determined as the point cloud information of the first node, the decoding efficiency can also be improved.

[0137] As an alternative embodiment, the method further includes:

[0138] Determining the mode of the first node;

[0139] The obtaining of the point cloud information of the corresponding node of the first node in the second frame in the first frame includes:

[0140] When the mode of the first node is a target mode, obtaining the point cloud information of the corresponding node of the first node in the second frame in the first frame, where the target mode indicates using the point cloud information of the reference frame, or the target mode indicates skipping, or the target mode indicates copying.

[0141] Wherein, the determining of the mode of the first node may be to determine the mode of the first node based on relevant information of the first node or pre-configured information, etc.

[0142] The above target mode may be referred to as a Skip mode or a Copy mode.

[0143] The above-mentioned point cloud information using the reference frame can be understood as determining the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node. The above-mentioned target mode indicating skipping can be understood as that the first node is not encoded in the target mode, and the above-mentioned target mode indicating copying can be understood as copying the point cloud information of the node corresponding to the first node in the second frame to the reconstructed point cloud to obtain the point cloud information of the first node.

[0144] In this embodiment, when the mode of the first node is the target mode, it is possible to obtain the point cloud information of the node corresponding to the first node in the second frame in the first frame, and determine the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node, so as to improve the encoding efficiency and reduce the bitstream size.

[0145] Optionally, when the target condition is satisfied, the mode of the first node is the target mode, and the target condition includes at least one of the following:

[0146] The rate-distortion cost of the first node in the target mode satisfies a preset condition;

[0147] The indication information corresponding to the first node indicates the target mode.

[0148] Among them, the above-mentioned preset condition can be defined in the standard, or determined through negotiation between the encoding end and the decoding end, etc. The rate-distortion cost of the first node in the target mode satisfying the preset condition can mean that the rate-distortion cost of the first node in the target mode is small, and vice versa, the rate-distortion cost is large. Or the rate-distortion cost of the first node in the target mode satisfying the preset condition can mean that the difference between the first frame and the reference frame is small, and vice versa, the difference is large.

[0149] In some embodiments, the preset conditions for different frames can be the same or different. For example, the rate-distortion cost of the first node in the target mode satisfying the preset condition can include at least one of the following:

[0150] The rate-distortion cost of the first node in the target mode is less than a preset threshold, or the rate-distortion cost of the first node in the target mode is within a preset range.

[0151] Among them, the above-mentioned preset threshold or the above-mentioned preset range can be different or the same for different frames. For example, the preset condition includes D < lambda * R, where D represents distortion, R represents the estimated bitstream, and lambda is a constant value in the same frame and can be the same value or different values in different frames.

[0152] In some embodiments, the target mode can be understood as a simplified inter-frame prediction coding mode. In this mode, the point cloud information of the reference frame is directly copied to the reconstructed point cloud, and part of the current frame is skipped to save coding time and resources. For example, the target mode is applicable to scenarios where the difference between the current frame and the reference frame is small, or where there are high requirements for coding latency in real-time dynamic point cloud transmission. That is, in real-time dynamic point cloud transmission with high requirements for coding latency, the coding mode of the first node can also be determined as the above target mode, and the latency requirement is higher than the preset threshold. When encoding, the encoder can evaluate the distortion-rate cost brought by using or not using this mode through rate-distortion cost. If the rate-distortion cost of the above first node (such as a prediction coding block) using the target mode is large, mark that the first node does not use the target mode (such as Skip_Node_flag = 0), and at this time, other modes (such as prediction entropy coding mode) are used to encode the nodes of the current frame layer by layer; if the rate-distortion cost of the first node using the target mode is small, mark that the first node PU uses the target mode (Skip_Node_flag = 1), and at this time, the first node (such as the PU layer node) of the current frame down to the leaf node layer does not need to be encoded, and the reference node or the compensated reference node is directly copied to the reconstructed point cloud.

[0153] The indication information corresponding to the first node may include a flag pre-configured for the above first frame or first node, or a flag determined based on the information of the first node. For example, the flag corresponding to the above first node indicates that the target mode includes at least one of the following:

[0154] The indication information of the sequence where the first frame is located represents the target mode;

[0155] The indication information of the first frame represents the target mode;

[0156] The indication information of the slice where the first node is located represents the target mode;

[0157] The indication information of the layer where the first node is located represents the target mode;

[0158] The indication information of the first node represents the target mode.

[0159] Among them, the indication information of the sequence where the first frame is located can indicate whether the sequence where the first frame is located allows the use of the target mode. For example, the first flag of the sequence where the first frame is located (such as described as Sps.Skip_mode_flag on, where Sps represents the Sequence parameter set, and 1 represents that the tool for inter-frame target mode prediction is enabled for the sequence where the first frame is located, and when this flag is 0, it represents that the inter-frame target mode prediction is disabled.

[0160] The indication information of the above-mentioned first frame may indicate whether the above-mentioned first frame allows the use of the target mode. For example, if the second flag of the first frame (such as described as gps.Skip_mode_flag on) is 1, it means that the first frame enables the tool for inter-frame target mode prediction (such as the Skip mode tool). When this flag is 0, it means that the inter-frame target mode prediction is turned off.

[0161] The indication information of the slice (or called slice point cloud) where the above-mentioned first node is located may indicate whether the slice where the above-mentioned first node is located allows the use of the target mode. For example, if the third flag of the slice where the first node is located (such as described as gdu.Skip_mode_flag on, where gdu represents the geometry data unit) is 1, it means that the slice where the first node is located enables the tool for inter-frame target mode prediction. When this flag is 0, it means that the inter-frame target mode prediction is turned off.

[0162] The indication information of the layer where the above-mentioned first node is located may indicate whether the layer where the above-mentioned first node is located allows the use of the target mode. For example, if the fourth flag (depth_Skip_mode_flag) of the layer where the first node is located is 1, it means that the layer where the first node is located enables the tool for inter-frame target mode prediction. When this flag is 0, it means that the inter-frame target mode prediction is turned off. Among them, the indication information of the layer where the first node is located may be generated by the encoding end when deciding to use the target mode for the layer where the first node is located and added to the bitstream, so that the decoding end can directly determine that the mode of the layer where the first node is located is the target mode based on this indication information. Or, the indication information of the layer where the above-mentioned first node is located may also be pre-configured, and this is not limited.

[0163] The indication information of the above-mentioned first node may indicate whether the above-mentioned first node uses the target mode. For example, if the fifth flag (Skip_Node_flag) of the first node is 1, it means that the first node uses target mode prediction. When this flag is 0, it means that the target mode is not used. Among them, the indication information of the above-mentioned first node may be generated by the encoding end when deciding to use the target mode for the first node and added to the bitstream, so that the decoding end can directly determine that the mode of the first node is the target mode based on this indication information.

[0164] Among them, the semantic meanings of the above-mentioned flag syntax elements and their descriptors can be as shown in Table 1 below:

[0165] Table 1:

[0166]

[0167] In some embodiments, for the encoding end, the above-mentioned target conditions may include:

[0168] The rate - distortion cost of the first node in the target mode is less than a preset threshold, or the rate - distortion cost of the first node in the target mode is within a preset range;

[0169] The indication information corresponding to the first node represents the target mode.

[0170] In some embodiments, for the decoding end, the above - mentioned target conditions may include:

[0171] The indication information corresponding to the first node represents the target mode.

[0172] In some embodiments, for the encoding end, the flag corresponding to the first node representing the target mode includes at least one of the following:

[0173] The indication information of the sequence where the first frame is located represents the target mode;

[0174] The indication information of the first frame represents the target mode;

[0175] The indication information of the slice where the first node is located represents the target mode;

[0176] The indication information of the layer where the first node is located represents the target mode.

[0177] In some embodiments, for the encoding end, the flag corresponding to the first node representing the target mode includes at least one of the following:

[0178] The indication information of the sequence where the first frame is located represents the target mode;

[0179] The indication information of the first frame represents the target mode;

[0180] The indication information of the slice where the first node is located represents the target mode;

[0181] The indication information of the layer where the first node is located represents the target mode.

[0182] The indication information of the first node represents the target mode.

[0183] It should be noted that in some embodiments, it may also be default that at least one of the first frame, the slice where the first node is located, or the layer where the first node is located is allowed to adopt the target mode.

[0184] It should be noted that in some embodiments, the rate - distortion cost of the first node in the target mode may not be considered. For example, in a scenario with high latency requirements, latency requirements may be considered first.

[0185] Optionally, the bitstream corresponding to the first frame includes the indication information corresponding to the first node.

[0186] The bitstream corresponding to the first frame may include at least one of the following:

[0187] The indication information of the sequence where the first frame is located;

[0188] The indication information of the first frame;

[0189] The indication information of the slice where the first node is located;

[0190] The indication information of the layer where the first node is located;

[0191] The indication information of the first node.

[0192] In this embodiment, since the bitstream corresponding to the first frame includes the indication information corresponding to the first node, the decoding end can directly determine to perform point cloud reconstruction on the first node in the target mode based on this.

[0193] It should be noted that, in some embodiments, the mode of the first node may not be determined. For example, it is assumed by default that the first node adopts the target mode.

[0194] In addition, determining the mode of the first node may be to determine whether the first node adopts the target mode. If the target mode is not adopted, other modes may be used for point cloud reconstruction of the first node, such as using the predictive entropy coding mode for point cloud reconstruction. The point cloud reconstruction using the predictive entropy coding mode may be as follows:

[0195] When processing the first node of the first frame, different processing is performed according to the size relationship between the first node and the LPU: when the size of the first node > LPU, since there is no available motion vector for motion compensation of the reference frame point cloud, the reference point cloud is directly used as the prediction information to encode the current frame; when the size of the first node <= LPU, it is divided according to the PU mode. First, it is judged whether the PU needs to continue to be divided downward. If the PU stops dividing downward, the motion-compensated point cloud is used as the prediction information to encode the current frame. On the contrary, if the PU continues to be divided downward, the reference point cloud is directly used as the prediction information to encode the current frame. Finally, the constructed inter-frame context is merged with the intra-frame context information of the first node, and the merged context is input into the entropy encoder to encode the point cloud of the first frame.

[0196] As an optional embodiment, the method further includes:

[0197] Replacing the first node with the node corresponding to the first node in the second frame for the quadtree partitioning of the first frame.

[0198] The above-mentioned multi-way tree division of the first frame by using the node corresponding to the first node in the second frame to replace the first node can be carried out when it is necessary to perform multi-way division on the nodes including the first node. That is, the node corresponding to the first node in the second frame is used to replace the first node for multi-way tree division. For example, the octree of the first node is continuously divided by using the node corresponding to the first node in the second frame.

[0199] In this embodiment, since the node corresponding to the first node in the second frame is used to replace the first node for multi-way tree division of the first frame, it can avoid destroying the neighbor structure and affecting entropy decoding after skipping the first node, because neighbor information is used as context during decoding, thereby improving the encoding and decoding performance.

[0200] As an alternative embodiment, the obtaining of the point cloud information of the node corresponding to the first node in the second frame in the first frame includes:

[0201] Obtaining the compensated point cloud information of the node corresponding to the first node in the second frame.

[0202] Among them, the above compensation can be local motion compensation, specifically, it can be compensated based on the motion vector.

[0203] In this embodiment, since the compensated point cloud information of the node corresponding to the first node in the second frame is obtained, the point cloud information of the first node can be made more accurate.

[0204] It should be noted that in some embodiments, it is also possible to obtain the uncompensated point cloud information of the node corresponding to the first node in the second frame.

[0205] As an alternative embodiment, the point cloud information of the node corresponding to the first node in the second frame includes at least one of the following:

[0206] The point cloud information of the node corresponding to the first node in the second frame itself;

[0207] The point cloud information of the child nodes of the node corresponding to the first node in the second frame;

[0208] The point cloud information of the first node includes at least one of the following:

[0209] The point cloud information of the first node itself;

[0210] The point cloud information of the child nodes of the first node.

[0211] In this embodiment, it is possible to use the point cloud information of the node itself corresponding to the first node in the second frame as the point cloud of the first node itself, or it is also possible to use the point cloud information of the child nodes of the node corresponding to the first node in the second frame as the point cloud information of the child nodes of the first node. In this way, it is not necessary to encode the child nodes of the first node, thereby improving the encoding efficiency and saving the bitstream size.

[0212] As an alternative embodiment, the method further includes:

[0213] Determining the vertices of the target edge based on the points of the second node in the nodes corresponding to the second frame; or

[0214] Determining the vertices of the target edge based on the points of all the nodes corresponding to the target edge in the nodes corresponding to the second frame;

[0215] Wherein, the point cloud information of the second node is the point cloud information in the second frame, the nodes corresponding to the target edge include the second node and the third node, and the vertices of the target edge are used for the point cloud reconstruction of the third node.

[0216] It can be understood that the point cloud information of the second node being the point cloud information in the second frame means that the second node is the node that obtains the point cloud information using the above target mode, or it can be understood that the second node can be the node obtained by dividing the node in the second frame that replaces the above first node during the multi - fork tree division, that is, the node obtained by dividing the nodes in the second frame, and can also be called a Skip node, that is, this node is not encoded.

[0217] Wherein, the above - mentioned second node can include one or more nodes, that is, the above - mentioned target edge can include one or more Skip nodes.

[0218] Wherein, the above - mentioned third node can be a leaf node or a Trisoup node, or it can be understood as a node that does not use the above - mentioned target mode.

[0219] The above - mentioned second node and the third node can also be called co - edge nodes, that is, the second node and the third node share one edge.

[0220] It can be understood that the nodes corresponding to the target edge including the second node and the third node means that the nodes adjacent to the target edge include the second node and the third node.

[0221] The vertices of the above - mentioned target edge can also be understood as the vertices of the above - mentioned third node, and the vertices of the above - mentioned target edge can also be understood as the vertices of the above - mentioned second node.

[0222] The above - mentioned determining the vertices of the target edge based on the points of the second node in the nodes corresponding to the second frame can be to determine the vertices of the target edge only using the points of the second node in the nodes corresponding to the second frame.

[0223] Determining the vertices of the target edge based on the points of all the nodes corresponding to the target edge among the nodes corresponding to the second frame may be determining the vertices of the second node based on the points of all the neighbor nodes of the target edge among the nodes corresponding to the second frame.

[0224] For example: as Figure 8 shown, the nodes corresponding to the target edge include nodes 1, 2, 3, and 4;

[0225] For only node 2 being the above-mentioned second node, there are two ways to determine the vertices in this case:

[0226] Determined only by the points of node 2 among the nodes corresponding to the second frame; or

[0227] Determined by the points of nodes 1, 2, 3, and 4 among the nodes corresponding to the second frame.

[0228] For nodes 2 and 3 being the above-mentioned second nodes, then there are two ways to determine the vertices in this case:

[0229] Determined by the points of nodes 2 and 3 among the nodes corresponding to the second frame;

[0230] Determined by the points of nodes 1, 2, 3, and 4 among the nodes corresponding to the second frame.

[0231] In the above embodiments, since the vertices of the target edge are determined based on the points of the second node among the nodes corresponding to the second frame; or the vertices of the target edge are determined based on the points of all the nodes corresponding to the target edge among the nodes corresponding to the second frame, this can avoid the problem that the neighbor structure of the third node is damaged, resulting in possible inaccuracies in the encoding or decoding of the third node, so as to improve the encoding or decoding performance.

[0232] It should be noted that the embodiments of the present application do not limit the method for determining the vertices based on points. For example: the method for determining the vertices based on points defined in the standard can be adopted.

[0233] In the embodiments of the present application, the point cloud information of the first node in the second frame corresponding to the first frame is obtained, where the second frame is the reference frame of the first frame; the point cloud information of the first node in the second frame corresponding to the first node is determined as the point cloud information of the first node. In this way, since the point cloud information of the first node is the point cloud information of the first node in the second frame corresponding to it, there is no need to encode the first node, thereby saving the encoding bitstream of the first node to reduce the bitstream size. In addition, since there is no need to encode the first node, the encoding efficiency can also be improved.

[0234] Please refer to Figure 9 , Figure 9It is a flowchart of another point cloud reconstruction method provided by an embodiment of the present application. As Figure 9 shown, it includes the following steps:

[0235] Step 901, the decoding end determines the mode of the first node of the first frame based on the bitstream;

[0236] Step 902, the decoding end obtains the point cloud information of the node corresponding to the first node in the second frame based on the mode of the first node, where the second frame is the reference frame of the first frame;

[0237] Step 903, the decoding end determines the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

[0238] The above-mentioned decoding end determines the mode of the first node of the first frame based on the bitstream, which may be that the bitstream includes the mode information of the above-mentioned first node, such as including the indication information corresponding to the first node.

[0239] Optionally, the decoding end obtains the point cloud information of the node corresponding to the first node in the second frame based on the mode of the first node, including:

[0240] When the mode of the first node is the target mode, obtain the point cloud information of the node corresponding to the first node in the second frame, where the target mode indicates using the point cloud information of the reference frame, or the target mode indicates skipping, or the target mode indicates copying.

[0241] Optionally, when the target condition is met, the encoding mode of the first node is the target mode, and the target condition includes:

[0242] The indication information corresponding to the first node indicates the target mode.

[0243] Optionally, the flag corresponding to the first node indicating the target mode includes at least one of the following:

[0244] The indication information of the sequence where the first frame is located indicates the target mode;

[0245] The indication information of the first frame indicates the target mode;

[0246] The indication information of the slice where the first node is located indicates the target mode;

[0247] The indication information of the layer where the first node is located indicates the target mode;

[0248] The indication information of the first node indicates the target mode.

[0249] Optionally, the bitstream corresponding to the first frame includes the indication information corresponding to the first node.

[0250] Optionally, the method further includes:

[0251] The decoding end uses the node corresponding to the first node in the second frame to replace the first node for the multi - fork tree division of the first frame.

[0252] Optionally, obtaining the point cloud information of the node corresponding to the first node in the first frame in the second frame includes:

[0253] Obtaining the compensated point cloud information of the node corresponding to the first node in the second frame.

[0254] Optionally, the reference frame of the first frame is the previous frame of the first frame.

[0255] Optionally, the point cloud information of the node corresponding to the first node in the second frame includes:

[0256] The point cloud information of the node corresponding to the first node in the second frame itself;

[0257] The point cloud information of the child nodes of the node corresponding to the first node in the second frame;

[0258] The point cloud information of the first node includes:

[0259] The point cloud information of the first node itself;

[0260] The point cloud information of the child nodes of the first node.

[0261] Optionally, the method further includes:

[0262] The decoding end determines the vertex of the target edge based on the points of the second node in the nodes corresponding to the second frame; or

[0263] The decoding end determines the vertex of the target edge based on the points of all the nodes corresponding to the target edge in the nodes corresponding to the second frame;

[0264] Wherein, the point cloud information of the second node is the point cloud information in the second frame, the nodes corresponding to the target edge include the second node and the third node, and the vertex of the target edge is used for the point cloud reconstruction of the third node.

[0265] It should be noted that, as the implementation manner of the decoding end corresponding to the Figure 7 shown embodiment, its specific implementation manner can refer to the relevant description of the Figure 7 shown embodiment. To avoid repeated description, this embodiment will not be elaborated here.

[0266] The following uses the above target mode as the Skip mode to illustrate the method provided by the embodiments of the present application through multiple embodiments:

[0267] Embodiment 1:

[0268] This embodiment is mainly about the decoding end, and taking the octree division as an example, when performing inter-frame predictive coding based on a predictive coding block, as Figure 10 shown, the following steps can be taken for operation:

[0269] 1. Encode whether Sps.Skip_mode_flag on of the current frame is 1; if it is 1, it means that the inter-frame prediction Skip mode tool is enabled for the current sequence; if it is 0, it means that the inter-frame prediction Skip mode tool is disabled.

[0270] 2. When Sps.Skip_mode_flag on is 1, encode whether gps.Skip_mode_flag on of the current frame is 1; if it is 1, it means that the inter-frame prediction Skip mode tool is enabled for the current frame point cloud; if it is 0, it means that the inter-frame prediction Skip mode tool is disabled for the current frame point cloud.

[0271] Otherwise, gps.Skip_mode_flag on is 0;

[0272] 3. When gps.Skip_mode_flag on is 1, encode whether gdu.Skip_mode_flag on (geometry data unit) of the current frame is 1; if it is 1, it means that the inter-frame prediction Skip mode tool is enabled for the current slice point cloud; if it is 0, it means that the inter-frame prediction Skip mode tool is disabled for the current slice point cloud.

[0273] Otherwise, gdu.Skip_mode_flag on is 0;

[0274] 4. Determine whether the size of the current depth layer node reaches the size of the node for Trisoup coding. If so, go to step 8 for Trisoup coding; otherwise, perform the following determination.

[0275] 5. When gdu.Skip_mode_flag on is 1, determine and encode whether depth_Skip_mode_flag of the current layer is 1. If it is 1, it means that the inter-frame prediction Skip mode tool is enabled for the current layer; if it is 0, it means that the inter-frame prediction Skip mode tool is disabled for the current layer.

[0276] Otherwise, depth_Skip_mode_flag is 0;

[0277] 6. If the depth_Skip_mode_flag is 1, determine whether the current node is skipped; otherwise, encode the current frame and current layer nodes using the predictive entropy coding mode.

[0278] If both gps.Skip_mode_flag on and depth_Skip_mode_flag are 1, it indicates that only the nodes in the current layer are allowed to perform inter-frame skipping. Before encoding the nodes that meet such conditions, the encoder will evaluate the distortion-rate cost brought by using or not using this mode through rate-distortion cost. If the rate-distortion cost of using the Skip mode for the current node is large, mark and encode that the current node does not use the Skip mode (Skip_Node_flag = 0), and at this time, encode the current frame node layer using the predictive entropy coding mode; if the rate-distortion cost of using the Skip mode for the current node is small, mark and encode that the current node uses the inter-frame Skip mode (Skip_Node_flag = 1). At this time, the current node does not need to be encoded, and the reference block (i.e., the node corresponding to the current node in the reference frame) or the compensated reference block is directly copied to the reconstructed point cloud. However, the reference block of the current node will continue to be divided into multiple branches in place of the current node. The purpose is to avoid directly skipping the current node, which may damage the neighbor structure and thus affect entropy coding because neighbor information is used as context during entropy coding.

[0279] If the depth_Skip_mode_flag is 0, encode the current node using the predictive entropy coding mode at this time.

[0280] 7. If all nodes in the current layer have been traversed, return to step 4; otherwise, return to step 6, that is, traverse the current layer first-in, first-out queue (First Input First Output, FIFO).

[0281] 8. Determine the vertices of all nodes in the current slice (here, the slice is a set of nodes obtained by dividing a frame of point cloud into slices and then further dividing the slices into multiple branches). Calculate the vertices of each node (each edge has a vertex, and each vertex is calculated from the points of the four nodes adjacent to the edge). At this time, determine whether the current node is marked as using the inter-frame Skip mode. If so, use the points in the reference nodes corresponding to the four neighbor nodes to determine the vertex when determining the vertex; otherwise, use the points in the four neighbor nodes to determine the vertex.

[0282] 9. Encode the vertices in the current slice. At this time, determine whether the current node is marked as using the inter-frame Skip mode. If so, the vertex is determined by the points in the reference block and does not need to be encoded; otherwise, encode the vertex.

[0283] 10. Determine and encode the centroid and face vertices of the nodes in the current slice. At this time, determine whether the current node is marked as using the inter-frame Skip mode. If so, the points in the current node are directly copied from the reference block without encoding; otherwise, determine and encode the centroid and face vertices of the current node, and perform surface fitting based on the vertices, centroid, and face vertices to obtain the reconstructed points in the current node.

[0284] 11. Determine whether the current slice is the last slice. If so, end the entire encoding process; otherwise, go back to step 8 to continue encoding.

[0285] Embodiment 2:

[0286] This embodiment describes the decoding end, and takes the octree division as an example. When performing inter-frame prediction decoding based on the predicted coding block, the following steps can be performed:

[0287] 1. Analyze whether the current Sps.Skip_mode_flag on is 1; if it is 1, it means that the inter-frame prediction Skip mode tool is enabled for the current sequence; if it is 0, it means that the inter-frame prediction Skip mode tool is disabled.

[0288] 2. When the current Sps.Skip_mode_flag on is 1, analyze whether the current gps.Skip_mode_flag on is 1; if it is 1, it means that the inter-frame prediction Skip mode tool is enabled for the current frame point cloud; if it is 0, it means that the inter-frame prediction Skip mode tool is disabled for the current frame point cloud.

[0289] Otherwise, gps.Skip_mode_flag on is 0;

[0290] 3. When the gps.Skip_mode_flag on is 1, analyze whether the current gdu.Skip_mode_flag on is 1; if it is 1, it means that the inter-frame prediction Skip mode tool is enabled for the current slice point cloud; if it is 0, it means that the inter-frame prediction Skip mode tool is disabled for the current slice point cloud.

[0291] Otherwise, gdu.Skip_mode_flag on is 0;

[0292] 4. Determine whether the size of the current depth layer node reaches the size of the node encoded by trisoup. If so, go to step 8 for trisoup decoding; otherwise, perform the following judgment.

[0293] 5. If gdu.Skip_mode_flag on is 1, then parse whether depth_Skip_mode_flag of the current layer is 1. If it is 1, it means that the inter-frame prediction Skip mode tool is enabled at the current layer; if it is 0, it means that the inter-frame prediction Skip mode tool is disabled at the current layer.

[0294] Otherwise, depth_Skip_mode_flag is 0;

[0295] 6. If depth_Skip_mode_flag is 1, then parse whether the current node is skipped. Otherwise, use the predictive entropy coding mode to code the nodes of the current frame at the current layer.

[0296] If both gps.Skip_mode_flag on and depth_Skip_mode_flag are 1, it indicates that only the nodes of the current layer are allowed to perform inter-frame skip. Before parsing the nodes that meet such conditions, parse whether Skip_Node_flag (Skip mode) is 0. If it is 0, it means that the rate-distortion cost of using the Skip mode for the current node is large. At this time, use the predictive entropy decoding mode to decode the nodes of the current frame layer; if it is 1, it means that the rate-distortion cost of using the Skip mode for the current node is small. At this time, the current node does not need to be decoded, and the reference block or the compensated reference block is directly copied to the reconstructed point cloud, but the reference node of the current node will replace the current node to continue the multi-way division. The purpose is to avoid directly skipping the current node and then destroying the neighbor structure and affecting the entropy decoding, because the neighbor information is used as context during decoding.

[0297] If depth_Skip_mode_flag is 0, at this time, use the predictive entropy decoding mode to decode the current node.

[0298] 7. If all the nodes of the current layer have been traversed, then return to step 4; otherwise, return to step 6.

[0299] 8. Parse the vertices in the current slice (here, the slice is a group of nodes obtained after a frame of point cloud is divided into slices and the slices are further divided by multi-way division). At this time, judge whether the current node is marked as using the inter-frame Skip mode. If so, use the points in the reference nodes corresponding to the four neighbor nodes to determine the vertices when determining the vertices; otherwise, determine the vertices through parsing.

[0300] 9. Decode the vertices in the current slice. At this time, judge whether the current node is marked as using the inter-frame Skip mode. If so, the vertices are determined by the points in the reference block and do not need to be decoded; otherwise, decode the vertices.

[0301] 10. Decode the centroid and surface vertices of the nodes in the current slice. At this time, determine whether the current node is marked as using the inter-frame Skip mode. If so, the points in the current node are directly copied from the reference block without decoding; otherwise, decode the centroid and surface vertices of the current node, and perform surface fitting based on the vertices, centroid, and surface vertices to obtain the reconstructed points in the current node.

[0302] 11. Determine whether the current slice is the last slice. If so, end the entire decoding process; otherwise, return to step 8 to continue decoding.

[0303] Embodiment 3:

[0304] This embodiment is mainly for the decoding end, and is illustrated by taking the octree division as an example. When performing inter-frame predictive coding based on PUs, as Figure 11 shown, the main difference between this embodiment and the above Embodiment 1 is that:

[0305] A local motion module is added (the Trisoup coding part is omitted here, and for details, see Figure 10 ). The local motion performs motion compensation on the reference block, and when making the Skip determination, the compensated reference block closer to the current block is selected (the uncompensated reference block can also be selected). It includes the following content:

[0306] When it is determined that the current node is an LPU, based on RDO, determine the motion vectors MVs and the split flag split_flag corresponding to the current node LPU or the PUs obtained by dividing the current node;

[0307] When it is determined that the reference node corresponding to the current node in the reference frame needs to be compensated, when terminating further division (split_flag = 0), use the MV to compensate the reference block.

[0308] Among them, the parts that are the same as those in Embodiment 1 will not be elaborated.

[0309] Experiments prove that compared with other coding technologies, the above Embodiment 1 or Embodiment 3 can save 5.6%, 9.2%, or 6.0% of the bitstream size.

[0310] For the point cloud reconstruction method provided in the embodiments of the present application, the execution subject can be a point cloud reconstruction device. In the embodiments of the present application, taking the point cloud reconstruction device as an example to execute the point cloud reconstruction method, the point cloud reconstruction device provided in the embodiments of the present application is described.

[0311] Please refer to Figure 12 , Figure 12 which is the structural diagram of a point cloud reconstruction device provided in the embodiments of the present application. As Figure 12 shown, the point cloud reconstruction device 1200 includes:

[0312] An acquisition module 1201, configured to acquire point cloud information of a node corresponding to a first node in a first frame in a second frame, where the second frame is a reference frame of the first frame;

[0313] A first determination module 1202, configured to determine the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

[0314] Optionally, the apparatus further includes:

[0315] A first determination module, configured to determine a mode of the first node;

[0316] The acquiring the point cloud information of the node corresponding to the first node in the second frame from the first frame includes:

[0317] When the mode of the first node is a target mode, acquiring the point cloud information of the node corresponding to the first node in the second frame from the first frame, where the target mode indicates using the point cloud information of the reference frame, or the target mode indicates skipping, or the target mode indicates copying.

[0318] Optionally, when a target condition is satisfied, the mode of the first node is the target mode, and the target condition includes at least one of the following:

[0319] The rate-distortion cost of the first node in the target mode satisfies a preset condition;

[0320] The indication information corresponding to the first node indicates the target mode.

[0321] Optionally, the flag corresponding to the first node indicating the target mode includes at least one of the following:

[0322] The indication information of the sequence where the first frame is located indicates the target mode;

[0323] The indication information of the first frame indicates the target mode;

[0324] The indication information of the slice where the first node is located indicates the target mode;

[0325] The indication information of the layer where the first node is located indicates the target mode;

[0326] The indication information of the first node indicates the target mode.

[0327] Optionally, the bitstream corresponding to the first frame includes the indication information corresponding to the first node.

[0328] Optionally, the apparatus further includes:

[0329] A partitioning module, configured to use the node corresponding to the first node in the second frame to replace the first node for multi-way tree partitioning of the first frame.

[0330] Optionally, obtaining the point cloud information of the node corresponding to the first node in the first frame in the second frame includes:

[0331] Obtaining the compensated point cloud information of the node corresponding to the first node in the second frame.

[0332] Optionally, the reference frame of the first frame is the previous frame of the first frame.

[0333] Optionally, the point cloud information of the node corresponding to the first node in the second frame includes at least one of the following:

[0334] The point cloud information of the node corresponding to the first node in the second frame itself;

[0335] The point cloud information of the child node of the node corresponding to the first node in the second frame;

[0336] The point cloud information of the first node includes at least one of the following:

[0337] The point cloud information of the first node itself;

[0338] The point cloud information of the child node of the first node.

[0339] Optionally, the apparatus further includes:

[0340] A third determination module, configured to determine the vertex of the target edge based on the points of the node corresponding to the second node in the second frame; or

[0341] A fourth determination module, configured to determine the vertex of the target edge based on the points of all nodes corresponding to the target edge in the second frame;

[0342] Wherein, the point cloud information of the second node is the point cloud information in the second frame, the nodes corresponding to the target edge include the second node and the third node, and the vertex of the target edge is used for point cloud reconstruction of the third node.

[0343] The above point cloud reconstruction apparatus can save the bitstream size.

[0344] In the embodiments of the present application, the point cloud reconstruction device may be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or a chip. For example, the electronic device may be a terminal or other devices other than the terminal. Exemplarily, the terminal may include, but is not limited to, the types of terminals listed in the embodiments of the present application, and other devices may be a server, a Network Attached Storage (NAS), etc., which are not specifically limited in the embodiments of the present application.

[0345] The point cloud reconstruction device provided in the embodiments of the present application can implement Figure 7 each process implemented by the method embodiments shown and achieve the same technical effects. To avoid repetition, details are not described herein again.

[0346] Please refer to Figure 13 , Figure 13 which is a structural diagram of a point cloud reconstruction device provided in the embodiments of the present application. As shown in Figure 13 , the point cloud reconstruction device 1300 includes:

[0347] A first determination module 1301, configured to determine the mode of the first node of the first frame based on the bitstream;

[0348] An acquisition module 1302, configured to acquire the point cloud information of the node corresponding to the first node in the second frame based on the mode of the first node, where the second frame is a reference frame of the first frame;

[0349] A second determination module 1303, configured to determine the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

[0350] Optionally, the acquisition module 1302 is configured to acquire the point cloud information of the node corresponding to the first node in the second frame when the mode of the first node is a target mode, where the target mode indicates using the point cloud information of the reference frame, or the target mode indicates skipping, or the target mode indicates copying.

[0351] Optionally, when a target condition is satisfied, the encoding mode of the first node is the target mode, and the target condition includes:

[0352] The indication information corresponding to the first node indicates the target mode.

[0353] Optionally, that the flag corresponding to the first node indicates the target mode includes at least one of the following:

[0354] The indication information of the sequence where the first frame is located indicates the target mode;

[0355] The indication information of the first frame represents the target mode;

[0356] The indication information of the slice where the first node is located represents the target mode;

[0357] The indication information of the layer where the first node is located represents the target mode;

[0358] The indication information of the first node represents the target mode.

[0359] Optionally, the bitstream corresponding to the first frame includes the indication information corresponding to the first node.

[0360] Optionally, the device further includes:

[0361] A partitioning module, configured to use the node corresponding to the first node in the second frame to replace the first node to perform the multi-way tree partitioning of the first frame.

[0362] Optionally, the obtaining of the point cloud information of the node corresponding to the first node in the second frame in the first frame includes:

[0363] Obtaining the compensated point cloud information of the node corresponding to the first node in the second frame.

[0364] Optionally, the reference frame of the first frame is the previous frame of the first frame.

[0365] Optionally, the point cloud information of the node corresponding to the first node in the second frame includes:

[0366] The point cloud information of the node corresponding to the first node in the second frame itself;

[0367] The point cloud information of the child nodes of the node corresponding to the first node in the second frame;

[0368] The point cloud information of the first node includes:

[0369] The point cloud information of the first node itself;

[0370] The point cloud information of the child nodes of the first node.

[0371] Optionally, the device further includes:

[0372] A third determination module, configured to determine the vertex of the target edge based on the points of the node corresponding to the second node in the second frame; or

[0373] A fourth determination module, configured to determine the vertex of the target edge based on the points of all the nodes corresponding to the target edge in the second frame;

[0374] Among them, the point cloud information of the second node is the point cloud information in the second frame. The nodes corresponding to the target edge include the second node and the third node. The vertices of the target edge are used for the point cloud reconstruction of the third node.

[0375] The above point cloud reconstruction device can save the bitstream size.

[0376] In the embodiments of the present application, the point cloud reconstruction device may be an electronic device, such as an electronic device with an operating system, or a component in an electronic device, such as an integrated circuit or a chip. For example: The electronic device may be a terminal or other devices other than the terminal. Exemplarily, the terminal may include, but is not limited to, the types of terminals listed in the embodiments of the present application. Other devices may be servers, NAS, etc., which are not specifically limited in the embodiments of the present application.

[0377] The point cloud reconstruction device provided by the embodiments of the present application can achieve Figure 9 each process implemented by the method embodiment shown and achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0378] As Figure 14 shown, the embodiments of the present application further provide an electronic device 1400, including a processor 1401 and a memory 1402. A program or instruction that can run on the processor 1401 is stored on the memory 1402. For example, when the electronic device 1400 is an encoding end device, when the program or instruction is executed by the processor 1401, each step of the above point cloud reconstruction method embodiment is implemented and the same technical effect can be achieved. When the electronic device 1400 is a decoding end device, when the program or instruction is executed by the processor 1401, each step of the above point cloud reconstruction method embodiment is implemented and the same technical effect can be achieved. To avoid repetition, it will not be elaborated here. Optionally, the memory 1402 may be Figure 1 the memory 102 or the memory 113 in the embodiment shown, and the processor 1401 can implement Figure 1 the functions of the encoder 200 or the decoder 300 in the embodiment shown in -3.

[0379] The embodiments of the present application further provide an electronic device, including: a memory configured to store video data; and a processing circuit configured to implement each step of the above point cloud reconstruction method embodiment. Optionally, the memory may be Figure 1 the memory 102 or the memory 113 in the embodiment shown, and the processing circuit can implement Figure 1 the functions of the encoder 200 or the decoder 300 in the embodiment shown in -3.

[0380] An embodiment of the present application further provides an electronic device, including a processor and a communication interface, the communication interface is coupled to the processor, and the processor is configured to run a program or an instruction to implement as Figure 7 or the steps in the method embodiment shown in FIG. 9. This device embodiment corresponds to the above method embodiment, and all implementation processes and implementation manners of the above method embodiment can be applied to this terminal embodiment and can achieve the same technical effect.

[0381] The above electronic device can be a terminal or other devices other than the terminal, such as a server, a Network Attached Storage (NAS), etc.

[0382] Among them, the terminal can be a mobile phone, a tablet personal computer, a laptop computer, a notebook computer, a personal digital assistant (PDA), a handheld computer, a netbook, an ultra-mobile personal computer (UMPC), a mobile Internet device (MID), an augmented reality (AR), a virtual reality (VR) device, a mixed reality (MR) device, a robot, a wearable device, a flight vehicle, a vehicle user equipment (VUE), a shipborne device, a pedestrian user equipment (PUE), a smart home (home devices with wireless communication functions, such as refrigerators, TVs, washing machines or furniture, etc.), a game console, a personal computer (PC), a teller machine or a self-service machine, etc. Wearable devices include: smart watches, smart bracelets, smart earphones, smart glasses, smart jewelry (smart bracelets, smart bracelets, smart rings, smart necklaces, smart anklets, smart ankle chains, etc.), smart wristbands, smart clothing, etc. Among them, the vehicle-mounted device can also be called a vehicle-mounted terminal, a vehicle-mounted controller, a vehicle-mounted module, a vehicle-mounted component, a vehicle-mounted chip or a vehicle-mounted unit, etc. It should be noted that the specific type of the terminal is not limited in the embodiment of the present application.

[0383] The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server, which can provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), or cloud computing services based on big data and artificial intelligence platforms, etc.

[0384] Exemplarily, the above-mentioned electronic device may include, but is not limited to Figure 1 the types of the source device 100 or the destination device 110 shown.

[0385] Taking the electronic device as a terminal as an example, Figure 15 is a schematic diagram of the hardware structure of a terminal for implementing an embodiment of the present application.

[0386] The terminal 1500 includes, but is not limited to: at least some components such as a radio frequency unit 1501, a network module 1502, an audio output unit 1503, an input unit 1504, a sensor 1505, a display unit 1506, a user input unit 1507, an interface unit 1508, a memory 1509, and a processor 1510.

[0387] Those skilled in the art can understand that the terminal 1500 may further include a power source (such as a battery) for powering each component, and the power source can be logically connected to the processor 1510 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. Figure 15 The terminal structure shown in does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0388] It should be understood that in the embodiments of the present application, the input unit 1504 may include a Graphics Processing Unit (GPU) 15041 and a microphone 15042. The graphics processor 15041 processes the image data of static pictures or videos obtained by an image acquisition device (such as a camera) in the video acquisition mode or the image acquisition mode, or may process the obtained point cloud data. The display unit 1506 may include a display panel 15061, and the display panel 15061 may be configured in the form of, for example, a liquid crystal display, an organic light emitting diode, etc. The user input unit 1507 includes at least one of a touch panel 15071 and other input devices 15072. The touch panel 15071 is also referred to as a touch screen. The touch panel 15071 may include two parts: a touch detection device and a touch controller. The other input devices 15072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, which will not be elaborated herein.

[0389] In the embodiments of the present application, after receiving downlink data from a network-side device, the radio frequency unit 1501 may transmit it to the processor 1510 for processing; in addition, the radio frequency unit 1501 may send uplink data to the network-side device. Generally, the radio frequency unit 1501 includes, but is not limited to, an antenna, an amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, etc.

[0390] The memory 1509 can be used to store software programs or instructions as well as various data. The memory 1509 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data. Among them, the first storage area may store an operating system, application programs or instructions required for at least one function (such as a sound playback function, an image playback function, etc.). In addition, the memory 1509 may include a volatile memory or a non-volatile memory. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDR SDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM), and a direct rambus random access memory (DRRAM). The memory 1509 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0391] The processor 1510 may include one or more processing units; optionally, the processor 1510 integrates an application processor and a modem processor. Among them, the application processor mainly processes operations related to the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above modem processor may not be integrated into the processor 1510.

[0392] In one embodiment:

[0393] The processor 1510 is configured to obtain the point cloud information of the node corresponding to the first node in the first frame in the second frame, where the second frame is the reference frame of the first frame; and determine the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

[0394] Optionally, the processor 1510 is further configured to:

[0395] Determine the mode of the first node;

[0396] The obtaining of the point cloud information of the node corresponding to the first node in the first frame in the second frame includes:

[0397] When the mode of the first node is the target mode, obtain the point cloud information of the node corresponding to the first node in the first frame in the second frame, where the target mode indicates using the point cloud information of the reference frame, or the target mode indicates skipping, or the target mode indicates copying.

[0398] Optionally, when the target condition is satisfied, the mode of the first node is the target mode, and the target condition includes at least one of the following:

[0399] The rate-distortion cost of the first node under the target mode satisfies a preset condition;

[0400] The indication information corresponding to the first node indicates the target mode.

[0401] Optionally, the flag corresponding to the first node indicating the target mode includes at least one of the following:

[0402] The indication information of the sequence where the first frame is located indicates the target mode;

[0403] The indication information of the first frame indicates the target mode;

[0404] The indication information of the slice where the first node is located indicates the target mode;

[0405] The indication information of the layer where the first node is located indicates the target mode;

[0406] The indication information of the first node indicates the target mode.

[0407] Optionally, the bitstream corresponding to the first frame includes the indication information corresponding to the first node.

[0408] Optionally, the processor 1510 is further configured to use the node corresponding to the first node in the second frame to replace the first node for the multi-way tree division of the first frame.

[0409] Optionally, the obtaining of the point cloud information of the node corresponding to the first node in the first frame in the second frame includes:

[0410] Obtain the compensated point cloud information of the node corresponding to the first node in the second frame.

[0411] Optionally, the reference frame of the first frame is the previous frame of the first frame.

[0412] Optionally, the point cloud information of the node corresponding to the first node in the second frame includes at least one of the following:

[0413] The point cloud information of the node itself corresponding to the first node in the second frame;

[0414] The point cloud information of the child nodes of the node corresponding to the first node in the second frame;

[0415] The point cloud information of the first node includes at least one of the following:

[0416] The point cloud information of the first node itself;

[0417] The point cloud information of the child nodes of the first node.

[0418] Optionally, the processor 1510 is further configured to:

[0419] Determine the vertices of the target edge based on the points of the node corresponding to the second node in the second frame; or

[0420] Determine the vertices of the target edge based on the points of all the nodes corresponding to the target edge in the second frame;

[0421] Wherein, the point cloud information of the second node is the point cloud information in the second frame, the nodes corresponding to the target edge include the second node and the third node, and the vertices of the target edge are used for the point cloud reconstruction of the third node.

[0422] In another embodiment:

[0423] The processor 1510 is configured to determine the mode of the first node of the first frame based on the bitstream; based on the mode of the first node, obtain the point cloud information of the node corresponding to the first node in the second frame, where the second frame is the reference frame of the first frame; and determine the point cloud information of the first node as the point cloud information of the node corresponding to the first node in the second frame.

[0424] The above decoding end determines the mode of the first node of the first frame based on the bitstream, which may be that the bitstream includes the mode information of the above first node, such as including the indication information corresponding to the first node.

[0425] Optionally, the decoding end obtains the point cloud information of the node corresponding to the first node in the second frame based on the mode of the first node, including:

[0426] When the mode of the first node is the target mode, obtain the point cloud information of the node corresponding to the first node in the second frame, where the target mode indicates using the point cloud information of the reference frame, or the target mode indicates skipping, or the target mode indicates copying.

[0427] Optionally, when the target condition is satisfied, the encoding mode of the first node is the target mode, where the target condition includes:

[0428] The indication information corresponding to the first node indicates the target mode.

[0429] Optionally, the flag corresponding to the first node indicating the target mode includes at least one of the following:

[0430] The indication information of the sequence where the first frame is located indicates the target mode;

[0431] The indication information of the first frame indicates the target mode;

[0432] The indication information of the slice where the first node is located indicates the target mode;

[0433] The indication information of the layer where the first node is located indicates the target mode;

[0434] The indication information of the first node indicates the target mode.

[0435] Optionally, the bitstream corresponding to the first frame includes the indication information corresponding to the first node.

[0436] Optionally, the processor 1510 is further configured to:

[0437] Use the node corresponding to the first node in the second frame to replace the first node for the multi-way tree division of the first frame.

[0438] Optionally, the obtaining the point cloud information of the node corresponding to the first node in the first frame in the second frame includes:

[0439] Obtain the compensated point cloud information of the node corresponding to the first node in the second frame.

[0440] Optionally, the reference frame of the first frame is the previous frame of the first frame.

[0441] Optionally, the point cloud information of the node corresponding to the first node in the second frame includes:

[0442] The point cloud information of the node corresponding to the first node in the second frame itself;

[0443] The point cloud information of the child nodes of the node corresponding to the first node in the second frame;

[0444] The point cloud information of the first node includes:

[0445] The point cloud information of the first node itself;

[0446] The point cloud information of the child nodes of the first node.

[0447] Optionally, the processor 1510 is further configured to:

[0448] Determine the vertices of the target edge based on the points of the second node in the nodes corresponding to the second frame; or

[0449] Determine the vertices of the target edge based on the points of all the nodes corresponding to the target edge in the nodes corresponding to the second frame;

[0450] Wherein, the point cloud information of the second node is the point cloud information in the second frame, the nodes corresponding to the target edge include the second node and the third node, and the vertices of the target edge are used for the point cloud reconstruction of the third node.

[0451] The above terminal can save the bitstream size.

[0452] It can be understood that the implementation processes of the various implementation manners mentioned in this embodiment can refer to the relevant descriptions of the method embodiment XXX, and achieve the same or corresponding technical effects. To avoid repetition, they will not be elaborated here.

[0453] The embodiment of the present application further provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, it implements each process of the above-mentioned point cloud reconstruction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0454] Wherein, the processor is the processor in the terminal described in the above embodiment. The readable storage medium includes computer-readable storage media, such as ROM, RAM, magnetic disks or optical discs, etc. In some examples, the readable storage medium can be a non-transitory readable storage medium.

[0455] The embodiment of the present application further provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement each process of the above-mentioned point cloud reconstruction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0456] It should be understood that the chip mentioned in the embodiment of the present application may include a system-on-chip (also referred to as a system chip, chip system or system-on-chip), or may also include an independent display chip, etc.

[0457] The embodiment of the present application further provides a computer program / program product, which is stored in a storage medium. The computer program / program product is executed by at least one processor to implement each process of the above-mentioned point cloud reconstruction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be elaborated here.

[0458] An embodiment of the present application further provides an encoding and decoding system, including: an encoding end device and a decoding end device. The encoding end device can be used to execute the steps of the point cloud reconstruction method as described above Figure 7 shown, and the decoding end device can be used to execute the steps of the point cloud reconstruction method as described above Figure 9 shown.

[0459] It should be noted that in this document, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device including a series of elements includes not only those elements but also other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0460] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of a computer software product plus a necessary general hardware platform, and of course, can also be implemented by hardware. This computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes several instructions for causing a terminal or a network-side device to execute the methods described in various embodiments of the present application.

[0461] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms of embodiments without departing from the purpose of the present application and the scope protected by the claims. These embodiments are all within the protection scope of the present application.

Claims

1. A point cloud reconstruction method, characterized in that, Including: Obtain the point cloud information of the node corresponding to the first node in the first frame in the second frame, where the second frame is the reference frame of the first frame; Determine the point cloud information of the first node as the point cloud information of the node corresponding to the first node in the second frame.

2. The method according to claim 1, characterized in that, The method further includes: Determine the mode of the first node; The obtaining of the point cloud information of the node corresponding to the first node in the first frame in the second frame includes: When the mode of the first node is the target mode, obtain the point cloud information of the node corresponding to the first node in the first frame in the second frame, where the target mode indicates using the point cloud information of the reference frame, or the target mode indicates skipping, or the target mode indicates copying.

3. The method according to claim 2, characterized in that, When the target condition is satisfied, the mode of the first node is the target mode, and the target condition includes at least one of the following: The rate distortion cost of the first node in the target mode satisfies a preset condition; The indication information corresponding to the first node indicates the target mode.

4. The method according to claim 3, characterized in that The flag corresponding to the first node indicating the target mode includes at least one of the following: The indication information of the sequence where the first frame is located indicates the target mode; The indication information of the first frame indicates the target mode; The indication information of the slice where the first node is located indicates the target mode; The indication information of the layer where the first node is located indicates the target mode; The indication information of the first node indicates the target mode.

5. The method according to claim 3 or 4, characterized in that, The bitstream corresponding to the first frame includes the indication information corresponding to the first node.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Use the node corresponding to the first node in the second frame to replace the first node for the multi-way tree division of the first frame.

7. The method according to any one of claims 1 to 6, characterized in that The obtaining of the point cloud information of the node corresponding to the first node in the first frame in the second frame includes: Obtain the compensated point cloud information of the node corresponding to the first node in the second frame.

8. The method according to any one of claims 1 to 7, characterized in that, The reference frame of the first frame is the previous frame of the first frame.

9. The method according to any one of claims 1 to 8, characterized in that, The point cloud information of the node corresponding to the first node in the second frame includes at least one of the following: The point cloud information of the node corresponding to the first node in the second frame itself; The point cloud information of the child nodes of the node corresponding to the first node in the second frame; The point cloud information of the first node includes at least one of the following: The point cloud information of the first node itself; The point cloud information of the child nodes of the first node.

10. The method according to any one of claims 1 to 9, characterized in that, The method further includes: Determine the vertex of the target edge based on the point of the node corresponding to the second node in the second frame; or Determine the vertex of the target edge based on the points of all the nodes corresponding to the target edge in the second frame; Wherein, the point cloud information of the second node is the point cloud information in the second frame, the nodes corresponding to the target edge include the second node and the third node, and the vertex of the target edge is used for the point cloud reconstruction of the third node.

11. A point cloud reconstruction method, characterized in that, Including: The decoding end determines the mode of the first node of the first frame based on the bitstream; The decoding end obtains the point cloud information of the node corresponding to the first node in the second frame based on the mode of the first node, where the second frame is the reference frame of the first frame; The decoding end determines the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

12. The method according to claim 11, wherein The decoding end obtains the point cloud information of the node corresponding to the first node in the second frame based on the mode of the first node, including: When the mode of the first node is the target mode, obtaining the point cloud information of the node corresponding to the first node in the second frame, where the target mode indicates using the point cloud information of the reference frame, or the target mode indicates skipping, or the target mode indicates copying.

13. The method according to claim 12, wherein When the target condition is satisfied, the mode of the first node is the target mode, and the target condition includes: The indication information corresponding to the first node indicates the target mode.

14. The method according to any one of claims 11 to 13, characterized in that The method further includes: The decoding end uses the node corresponding to the first node in the second frame to replace the first node for the multi-way tree division of the first frame.

15. The method according to any one of claims 11 to 14, characterized in that The method further includes: The decoding end determines the vertex of the target edge based on the points of the node corresponding to the second node in the second frame; or The decoding end determines the vertex of the target edge based on the points of all the nodes corresponding to the target edge in the second frame; Wherein, the point cloud information of the second node is the point cloud information in the second frame, the nodes corresponding to the target edge include the second node and the third node, and the vertex of the target edge is used for the point cloud reconstruction of the third node.

16. A point cloud reconstruction device, characterized in that, Including: An acquisition module, configured to acquire the point cloud information of the node corresponding to the first node in the first frame in the second frame, where the second frame is the reference frame of the first frame; A first determination module, configured to determine the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

17. The device according to claim 16, wherein The apparatus further includes: A first determination module, configured to determine the mode of the first node; The acquisition of the point cloud information of the node corresponding to the first node in the first frame in the second frame includes: When the mode of the first node is the target mode, acquiring the point cloud information of the node corresponding to the first node in the first frame in the second frame, where the target mode indicates using the point cloud information of the reference frame, or the target mode indicates skipping, or the target mode indicates copying.

18. The device according to claim 17, wherein When the target condition is satisfied, the mode of the first node is the target mode, and the target condition includes at least one of the following: The rate-distortion cost of the first node in the target mode satisfies a preset condition; The indication information corresponding to the first node indicates the target mode.

19. The device according to any one of claims 16 to 17, characterized in that, The apparatus further includes: A division module, configured to use the node corresponding to the first node in the second frame to replace the first node for the multi-way tree division of the first frame.

20. The device according to any one of claims 16 to 19, characterized in that The apparatus further includes: A third determination module, configured to determine the vertex of the target edge based on the points of the node corresponding to the second node in the second frame; or A fourth determination module, configured to determine the vertex of the target edge based on the points of all the nodes corresponding to the target edge in the second frame; Among them, the point cloud information of the second node is the point cloud information in the second frame. The nodes corresponding to the target edge include the second node and the third node. The vertices of the target edge are used for the point cloud reconstruction of the third node.

21. A point cloud reconstruction device, characterized in that, It includes: A first determination module, configured to determine the mode of the first node in the first frame based on the bitstream; An acquisition module, configured to acquire the point cloud information of the node corresponding to the first node in the second frame based on the mode of the first node, where the second frame is the reference frame of the first frame; A second determination module, configured to determine the point cloud information of the node corresponding to the first node in the second frame as the point cloud information of the first node.

22. The device according to claim 21, characterized in that, The acquisition module is configured to acquire the point cloud information of the node corresponding to the first node in the second frame when the mode of the first node is a target mode, where the target mode indicates using the point cloud information of the reference frame, or the target mode indicates skipping, or the target mode indicates copying.

23. The device according to claim 22, characterized in that, When a target condition is satisfied, the mode of the first node is the target mode, and the target condition includes: The indication information corresponding to the first node indicates the target mode.

24. An electronic device, characterized in that, It includes a processor and a memory. The memory stores a program or instruction that can run on the processor. When the program or instruction is executed by the processor, it implements the steps of the point cloud reconstruction method according to any one of claims 1 to 10, or implements the steps of the method according to any one of claims 11 to 15.

25. A readable storage medium, characterized in that, A program or instruction is stored on the readable storage medium. When the program or instruction is executed by the processor, it implements the steps of the point cloud reconstruction method according to any one of claims 1 to 10, or implements the steps of the method according to any one of claims 11 to 15.

26. A chip, characterized in that, The chip includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run a program or instruction to implement the steps of the point cloud reconstruction method according to any one of claims 1 to 10, or implement the steps of the method according to any one of claims 11 to 15.

Images