Method and device for encoding and decoding point cloud information

Abstract

The present application provides a method and apparatus for encoding and decoding point cloud information. The method includes: obtaining a point cloud, the point cloud comprising a set of points in a three-dimensional space; segmenting the point cloud into a plurality of objects, and generating occupancy information for each of the plurality of objects; and encoding the occupancy information by considering the distances between the plurality of objects.

Description

Background Art

[0001] Point cloud modeling has been widely used in recent years. For example, point clouds can be used for object detection and localization in autonomous vehicles, for mapping in geographic information systems (GIS), and for visualizing and archiving cultural heritage objects and collections in cultural heritage projects.

[0002] A point cloud consists of a set of high-dimensional points (usually three-dimensional (3D)), each of which includes 3D location information and additional attributes such as color and reflectivity. High-dimensional point sets can be captured using Light Detection and Ranging (LIDAR) using multiple cameras and depth sensors in various deployments and can consist of thousands to billions of points to realistically represent the original scene.

[0003] Compression techniques are needed to reduce the amount of data required to represent point clouds for faster transmission and / or reduced storage. As explained in Non-Patent Documents 1 and 2, the Moving Picture Experts Group (MPEG) has created a Joint Technical Committee (JTC 1 / SC 29 / WG 11) and an ad hoc group (MPEG-PCC) to standardize compression techniques for static and / or dynamic point clouds. MPEG is a working group of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC).

[0004] Non-Patent Document 1: Use Cases for Point Cloud Compression, JTC1 / SC29 WG11 document number N16331, published in Chinese in Geneva in June 2016.

[0005] Non-Patent Document 2: Requirements for Point Cloud Compression, JTC1 / SC29 WG11 document number N16330, published in Chinese in Geneva in June 2016. Summary of the Invention

[0006] This application relates to point cloud coding (PCC). As described herein, for example, a context modeling method for occupancy information coding and decoding is described in detail for point cloud compression.

[0007] According to one aspect of the present application, a method for encoding and decoding point cloud information can be performed by at least one processor and can include: obtaining a point cloud, wherein the point cloud includes a set of points in a three-dimensional space; segmenting the point cloud into multiple objects and generating occupancy information for each of the multiple objects; and encoding the occupancy information by considering the distance between the multiple objects.

[0008] According to one aspect of the present application, the method may be implemented by encoding the occupancy information object by object, so that the next occupancy information to be encoded is encoded based on the classification of surrounding objects among the multiple objects, and the classification of the surrounding objects is based on the distance between the multiple objects.

[0009] According to another aspect of the present application, the method may be performed by encoding the occupancy information using a context index, wherein the context index is defined by a lookup table using the following formula: LUT{ctxIdx}{ctxIdxChild}, wherein LUT is a lookup table, ctxIdx is a node in the multiple objects, and ctxIdxChild is a child node in the multiple objects.

[0010] According to another aspect of the present application, the method may be performed by segmenting the point cloud using a flexible tree segmentation method.

[0011] According to another aspect of the present application, the method may be performed in such a way that the point cloud is segmented using a depth-first segmentation method.

[0012] According to another aspect of the present application, the method may be performed by encoding the occupancy information for the objects one by one, thereby encoding the next item of occupancy information to be encoded based on the classification of surrounding objects among the multiple objects, wherein the surrounding objects have been encoded and classified based on the distance between the multiple objects.

[0013] According to another aspect of the present application, the method may be performed by encoding the occupancy information of the objects one by one, thereby encoding the next occupancy information to be encoded based on the classification of surrounding objects among the multiple objects, at least some of the surrounding objects have been encoded or partially encoded, and are classified based on the distance between the multiple objects.

[0014] According to one aspect of the present application, a device for encoding and decoding information of a point cloud may include: at least one memory for storing computer program code; and at least one processor for accessing the at least one memory and operating according to the computer program code; the computer program code may include: an acquisition code for acquiring a point cloud, the point cloud comprising a set of points in a three-dimensional space; a segmentation code for segmenting the point cloud into multiple objects and generating occupancy information for each of the multiple objects; and an encoding code for encoding the occupancy information by considering the distance between the multiple objects.

[0015] According to one aspect of the present application, the encoding code can be used to encode the occupancy information on an object-by-object basis, thereby encoding the next item of occupancy information to be encoded based on the classification of surrounding objects among the multiple objects, wherein the surrounding objects are classified based on the distance between the multiple objects.

[0016] According to another aspect of the present application, an encoding code may be used to encode the occupancy information using a context index defined by a lookup table using the following formula: LUT{ctxIdx}{ctxIdxChild}, where LUT is a lookup table, ctxIdx is a node in multiple objects, and ctxIdxChild is a child node in multiple objects.

[0017] According to another aspect of the present application, a segmentation code can be used to segment a point cloud using a flexible tree segmentation scheme.

[0018] According to another aspect of the present application, the segmentation code can be used to segment the point cloud using a depth-first segmentation scheme.

[0019] According to another aspect of the present application, the encoding code can be used to encode the occupancy information object by object, so that the next occupancy information to be encoded is encoded based on the classification of surrounding objects among the multiple objects, and the surrounding objects have been encoded and classified based on the distance between the multiple objects.

[0020] According to one aspect of the present application, a non-transitory computer-readable storage medium storing instructions may be provided, the instructions causing at least one processor to: obtain a point cloud comprising a set of points in a three-dimensional space; segment the point cloud into a plurality of objects and generate occupancy information for each of the plurality of objects; and encode the occupancy information by taking into account the distance between the plurality of objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the octree partitioning method in three-dimensional space.

[0022] Figure 2 This is a schematic diagram of octree partitioning and tree structure.

[0023] Figure 3 It is a schematic diagram of the classification of adjacent nodes.

[0024] Figure 4 It is a schematic diagram of the classification of adjacent nodes.

[0025] Figure 5 is a schematic diagram of the adjacent nodes of a non-cube node.

[0026] Figure 6 is a flow chart of an example method for encoding and decoding information of a point cloud.

[0027] Figure 7 It is a schematic diagram of an example of an implementation environment of the system and / or method of the present application.

[0028] Figure 8 yes Figure 7 A schematic diagram of exemplary components of at least one device. DETAILED DESCRIPTION

[0029] Point Cloud Compression in MPEG Test Model 13 (TMC13)

[0030] As described in Non-Patent Document 3 and Non-Patent Document 4, in the Test Model 13 (TMC13) codec published by MPEG, the geometric information and related attributes, such as color or reflectivity, of each point in the point cloud are compressed separately. The geometric information (e.g., the 3D coordinates of the point cloud) and its occupancy information are encoded using an octree (OT) segmentation method. Based on the reconstructed geometric structure, the attributes are then compressed using prediction, lifting, and region-adaptive hierarchical transformation techniques. The octree segmentation method and the occupancy information encoding process are described in detail below.

[0031] Non-Patent Document 3: G-PCC Codec Description, ISO / IEC JTC1 / SC29 / WG11 Document No. N18891, October 2019.

[0032] Non-Patent Document 4: ISO / IEC CD 23090-9, Text for Geometry-Based Point Cloud Compression (G-PCC Codec description), ISO / IEC JTC1 / SC29 WG11 Document No. N18478, July 2019, Geneva.

[0033] Octree partitioning method

[0034] In TMC13, if the octree geometry codec is used, the geometry encoding is performed as follows. First, the bounding box B aligned with the cube axis consists of two points (0,0,0) and (2 M-1 ,2 M-1 ,2 M-1 ) is limited, where 2 M-1 The size of B is defined, and M is given in the bitstream. B is then further divided in a recursive manner to build an octree structure. At each stage, a cube is further divided into 8 sub-cubes. An 8-bit code, the occupancy code, is then generated by associating each sub-cube with a 1-bit value to indicate whether it contains a point (i.e., whether it is full and has a value of 1) or does not contain a point (i.e., whether it is empty and has a value of 0). Only full sub-cubes (i.e., non-volume elements) with a size greater than 1 are further subdivided. Figure 1 Schematic diagram of the octree partitioning method in three-dimensional space.

[0035] Figure 2 An example of a two-level octree partitioning method and the corresponding occupancy code is shown in , where dark cubes and nodes indicate that they are occupied by points.

[0036] Encoding occupancy codes

[0037] The occupancy code of each node is then compressed by an arithmetic encoder. The occupancy code can be represented as S, where S is an 8-bit integer, and each bit in S indicates the occupancy status of each child node. TMC13 has two occupancy code encoding methods: bit-by-bit encoding and byte-by-byte encoding. Bit-by-bit encoding is enabled by default. In both methods, arithmetic coding with context modeling is used to encode the occupancy code, where the context state is initialized at the beginning of the entire encoding process and updated during the encoding process.

[0038] For bit-by-bit encoding, the eight binary numbers in S are encoded in a certain order, where each binary number is encoded by referring to the occupancy status of the neighboring nodes and the occupancy status of the children of the neighboring nodes, where the neighboring nodes are at the same level as the current node.

[0039] For byte-by-byte encoding, S is encoded by referring to the following information:

[0040] An adaptive lookup table (A-LUT) that records the N (e.g., 32) most frequently used occupied codes, and

[0041] A cache that records the most recently observed M (e.g., 16) different occupied codes.

[0042] A binary flag indicating whether S is in the A-LUT is encoded. If S is in the A-LUT, the index into the A-LUT is encoded using a binary arithmetic encoder. If S is not in the A-LUT, a binary flag indicating whether S is in the cache is encoded. If S is in the cache, the binary representation of the index of S is encoded using a binary arithmetic encoder. Otherwise, if S is not in the cache, the binary representation of S is encoded using a binary arithmetic encoder.

[0043] The decoding process starts by parsing the bitstream for the dimensions of the bounding box B. The same octree structure is then built by subdividing B according to the decoded occupancy code.

[0044] However, in the codec schemes discussed above, the representation based on the octree partitioning method may not be flexible enough for the different characteristics of point clouds serving different purposes. That is, the compression efficiency may be limited. The use of quadtree (QT) and binary tree (BT) and other flexible partitioning schemes have been mentioned in non-patent literature 5, and the encoding and decoding of point clouds are discussed below. As described in this article, using only one traversal order of the tree structure (breadth-first or depth-first) may not be suitable for all application scenarios.

[0045] Non-patent document 5: Implicit geometric segmentation scheme for point cloud encoding and decoding, US provisional application 523568US.

[0046] Therefore, a hybrid traversal order, briefly discussed in Non-Patent Document 6, is proposed. The context modeling approach for occupancy information encoding and decoding can be modified to fully utilize adjacent coded nodes. This improves coding efficiency and accommodates flexible segmentation schemes and traversal orders.

[0047] Non-patent document 6: Hybrid encoding and decoding order for point cloud encoding and decoding, U.S. Provisional Application 527115US.

[0048] The methods and apparatuses presented herein may be used individually or in any combination. Furthermore, each of the methods (or embodiments), encoders, and decoders may be implemented by processing circuitry (e.g., at least one processor or at least one integrated circuit). In one example, at least one processor executes a program stored on a non-transitory computer-readable medium. Furthermore, the disclosure herein is not limited to use with TMC13 software or the MPEG-PCC standard.

[0049] Binarization of occupancy information

[0050] According to various embodiments, if a node is divided into N sub-nodes, an N-bit occupation code is generated, that is, b N- 1b N-2 ...b0, where each bit b i Indicates whether the i-th child node is occupied or empty. Usually, b i is equal to 1, indicating that the i-th child node is occupied by at least one point, and b i Equal to 0 means the i-th child node is empty.

[0051] In another embodiment, b i is equal to 0, indicating that the i-th child node is occupied by at least one point, and b i Equal to 1 means that the i-th child node is empty. For those child nodes that are occupied, these child nodes can be further split until a leaf node is reached.

[0052] According to various embodiments, the encoding and decoding order of the bits in the occupied code can be defined in any order. For an N-bit code, the maximum number of encoding and decoding orders is To express the encoding and decoding order of different granularities, you can specify the following:

[0053] Apply the predefined codec order to all nodes and all point clouds, without requiring extra bits to specify the codec order;

[0054] Apply a specific codec order to a specific point cloud. The codec order can be specified in the sequence header of the bitstream.

[0055] Apply a specific codec order to each frame of the point cloud. The codec order can be specified in the frame header of the bitstream.

[0056] Apply a specific codec order to each slice of the point cloud. The codec order can be specified in the slice header of the bitstream.

[0057] Apply a specific codec order to the nodes of each segmentation depth of the point cloud. The codec order can be specified at the beginning of each segmentation depth. This mode is suitable for traversing the tree structure in breadth-first order.

[0058] Apply a specific encoding and decoding order to a node and all its descendant nodes. You can specify the encoding and decoding order before encoding the node. This mode is suitable for traversing the tree structure in depth-first order.

[0059] Apply a specific codec order to each node of the point cloud. You can specify the codec order before encoding each node.

[0060] Apply a specific codec order to nodes in a point cloud when a certain eligibility condition is met. The codec order can be specified at each segmentation depth for breadth-first order, or at an ancestor node for depth-first order, or in a slice header, frame header, or sequence header, or a default codec order can be applied to the remaining nodes in the point cloud that do not meet the eligibility condition.

[0061] According to various embodiments, a preferred codec order can be selected by selecting a candidate codec order requiring the fewest bits from a set of candidate codec orders. An index can be signaled to indicate a particular codec order. The index can be encoded using bypass coding or entropy coding with probability and context updating.

[0062] Bypass coding of occupation codes

[0063] According to various embodiments, the busy code may be represented with fewer bits and bypass coding may be used accordingly.

[0064] In one embodiment, if the current node has only one occupied child node, it is acceptable to encode the position of the occupied node. First, a flag is specified to indicate whether the current node includes only one occupied child node. If so, a 3-bit index is encoded for the octree partitioning scheme, a 2-bit index is encoded for the quadtree partitioning scheme, and a 1-bit index is encoded for the binary tree partitioning scheme. Otherwise, the occupancy code of the current node is encoded normally. Whether the mode is applicable can be determined by the number of occupied adjacent nodes or other conditions; the flag and index are only encoded when the mode is applicable. For example, a threshold value can be defined. The mode is only applicable when the number of occupied adjacent nodes is less than the threshold value.

[0065] In another embodiment, if the current node has only one unoccupied child node, it is equivalent to encoding the position of the unoccupied node. A flag can be first specified to indicate whether the current node includes only one unoccupied child node. If so, a 3-bit index of the octree partitioning scheme is encoded, a 2-bit index of the quadtree partitioning scheme is encoded, and a 1-bit index of the binary tree partitioning scheme is encoded. Otherwise, the occupancy code of the current node is encoded normally. Whether the mode is applicable can be determined by the number of occupied adjacent nodes or other conditions; the flag and index are encoded when the mode is applicable. For example, a threshold value can be defined, and the mode is only applicable when the number of occupied adjacent nodes is greater than the threshold value.

[0066] Context modeling in occupancy information encoding

[0067] According to various embodiments, all previously encoded nodes may be used in the context modeling process for occupancy information encoding of the present application before encoding the current node. To simplify context modeling, only the previously encoded nodes closest to the current node are used. As explained herein, a variety of methods may be used to simplify the context.

[0068] Distance-based context modeling

[0069] According to various embodiments, when encoding the occupancy code of the current node, the neighboring encoded nodes may be classified according to their distances to the current node. Figure 3 and Figure 4 A 2D diagram of distance-based classification is shown in FIG, where the distance between two nodes is measured by the distance l2 and l1 respectively. Encoded nodes with the same distance to the current node are classified as the same type. Then, different classes (i.e., C1, C2, ..., C K ) are sorted, where C i is a set of encoded nodes ranked by proximity to the current node, and K is the number of groups. The context index can be calculated as:

[0070] ctxIdx=N1·(M2M3…M K )+N2·(M3M4…M K )+…+N K ,

[0071] Among them, N i represents the number of occupied nodes in group i, and M i Indicates the maximum number of nodes in group i.

[0072] Similarly, according to various embodiments, when encoding and decoding the occupancy information bits of each child node, the same method can be used to obtain the context index of each child node, i.e., ctxIdxChild. Then, the occupancy code of the current node can be encoded by entropy coding based on the derived ctxIdx and ctxIdxChild. For example, the final context index, i.e., idx, can be obtained by the lookup table LUT as follows:

[0073] idx=LUT[ctxIdx][ctxIdxChild].

[0074] In another embodiment, a technique is used in which for each C i Define the threshold T i If N i ≥T i , then add positive bias scores as follows.

[0075]

[0076] in, The final score is used as the context index of the current node.

[0077] According to various embodiments, if a flexible tree partitioning scheme is allowed, where symmetric and asymmetric octree partitioning, quadtree partitioning, and binary tree partitioning are enabled, the current node may not be a cube and may have different sizes in some dimensions. In these cases, the neighboring nodes of the current node may be defined as the encoded neighboring nodes with the same size as the current node, such as Figure 5 In distance-based context modeling, the distance order of adjacent nodes can be changed accordingly. For example, in Figure 5 In the network, the two adjacent nodes in the vertical direction are closer to the central node than the two adjacent nodes in the horizontal direction.

[0078] According to various embodiments, if a depth-first traversal order is used, only some of the neighboring nodes of the current node and some of the neighboring nodes of the parent node of the current node are encoded before the current node, so only partial context information can be used for context modeling of the current node. However, for those neighboring nodes that have been encoded, complete occupancy information from the current segmentation depth to the leaf node can be used for context modeling of the current node.

[0079] Figure 6 is a flow chart of an example method 600 for encoding and decoding information of a point cloud. In some embodiments, Figure 6 At least one step of the block may be performed by the user device 710. In some embodiments, Figure 6 At least one step block may be performed by another device or a group of devices independent of or including the user device 710 , such as the platform 720 .

[0080] like Figure 6 As shown, method 600 may include acquiring a point cloud (block 610 ), the point cloud comprising a set of points in three-dimensional space.

[0081] The point cloud may be a set of points in a 3-dimensional space, each point having associated attributes, such as color, material properties, etc.

[0082] like Figure 6 As further shown, method 600 may include segmenting the point cloud into a plurality of objects (block 620 ), generating occupancy information for each of the plurality of objects (block 630 ), and encoding the occupancy information by considering distances between the plurality of objects (block 640 ).

[0083] Figure 7 is a schematic diagram of an example communication system 700 in which the systems and / or methods described herein may be implemented. Figure 7 As shown, communication system 700 may include user equipment 710, platform 720, and network 730. The devices of communication system 700 may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0084] Communication system 700 can support unidirectional data transmission. For example, a first user device 710 can encode video data at a local location for transmission to a second user device 710 via network 730. Second user device 710 can receive the encoded video data from first user device 710 from network 730, decode the encoded data, and display the recovered video data. Unidirectional data transmission is common in media service applications, etc.

[0085] Communication system 700 can support bidirectional transmission of data. For example, communication system 700 can support bidirectional transmission of encoded video, such as might occur during a video conference. For bidirectional transmission of data, each user device 710 can encode video data captured at a local location for transmission to another user device 710 via network 730. Each user device 710 can also receive encoded video data transmitted by another user device 710, decode the encoded data, and display the recovered video data on a local display device.

[0086] User device 710 includes one or more devices that can receive, generate, store, process, and / or provide information associated with platform 720. For example, user device 710 can include a computing device (e.g., a desktop computer, a laptop computer, a tablet computer, a handheld computer, a smart speaker, a server, etc.), a mobile phone (e.g., a smartphone, a wireless phone, etc.), a wearable device (e.g., smart glasses or a smart watch), or similar devices. In some embodiments, user device 710 can receive information from platform 720 and / or send information to platform 720.

[0087] The platform 720 includes at least one device as described elsewhere herein that can provide information to the user device 710. In some embodiments, the platform 720 can include a cloud server or a group of cloud servers. In some embodiments, the platform 720 can employ a modular design so that software components can be swapped in and out based on specific needs. This allows the platform 720 to be easily and / or quickly reconfigured for different uses.

[0088] In some embodiments, as shown, platform 720 can be hosted in a cloud computing environment 722. It is worth noting that although the embodiments described herein describe platform 720 as being hosted in a cloud computing environment 722, in some embodiments, platform 720 may not be cloud-based (i.e., may be implemented outside of a cloud computing environment) or may be partially cloud-based.

[0089] Cloud computing environment 722 includes an environment hosting platform 720. Cloud computing environment 722 can provide computing, software, data access, storage, and other services without requiring end users (e.g., user devices 710) to be aware of the physical location and configuration of the systems and / or devices hosting platform 720. As shown, cloud computing environment 722 can include a set of computing resources 724 (collectively, "computing resources 724" and individually, "computing resource 724").

[0090] Computing resources 724 include one or more personal computers, workstation computers, server devices, or other types of computing and / or communication devices. In some embodiments, computing resources 724 can host platform 720. Cloud resources can include computing instances executed in computing resources 724, storage devices provided in computing resources 724, data transmission devices provided by computing resources 724, etc. In some embodiments, computing resources 724 can communicate with other computing resources 724 via wired connections, wireless connections, or a combination of wired and wireless connections.

[0091] Further Figure 7 As shown, the computing resources 724 include a set of cloud resources, such as one or more applications (APP) 724-1, one or more virtual machines (VM) 724-2, virtualized storage (VS) 724-3, one or more hypervisors (HYP) 724-4, etc.

[0092] Applications 724-1 include one or more software applications that can be provided to or accessed by user device 710 and / or platform 720. Applications 724-1 eliminate the need for user device 710 to install and execute software applications. For example, applications 724-1 may include software associated with platform 720 and / or any other software that can be provided via cloud computing environment 722. In some embodiments, one application 724-1 can send and receive information to and from one or more other applications 724-1 via virtual machine 724-2.

[0093] Virtual machine 724-2 comprises a software implementation of a machine (e.g., a computer) that executes programs like a physical machine. Virtual machine 724-2 may be a system virtual machine or a process virtual machine, depending on the use and degree of correspondence of virtual machine 724-2 to any real machine. A system virtual machine may provide a complete system platform that supports the execution of a complete operating system ("OS"). A process virtual machine may execute a single program and may support a single process. In some embodiments, virtual machine 724-2 may execute on behalf of a user (e.g., user device 710) and may manage the infrastructure of cloud computing environment 722, such as data management, synchronization, or long-term data transfer.

[0094] Virtualized storage 724-3 includes one or more storage systems and / or one or more devices that use virtualization technology within the storage system or device of the computing resource 724. In some embodiments, within the context of the storage system, the types of virtualization may include block virtualization and file virtualization. Block virtualization may refer to abstracting (or separating) logical storage from physical storage, thereby allowing access to the storage system without considering the physical storage or heterogeneous structure. The above separation may allow administrators of the storage system to flexibly manage the storage of end users. File virtualization may eliminate the dependency between data accessed at the file level and the location of the physical storage file. This may optimize the performance of storage usage, server consolidation, and / or non-disruptive file migration.

[0095] Hypervisor 724-4 can provide hardware virtualization technology that allows multiple operating systems (e.g., "guest operating systems") to execute simultaneously on a host computer such as computing resource 724. Hypervisor 724-4 can provide a virtual operating platform to the guest operating systems and can manage the execution of the guest operating systems. Multiple instances of various operating systems can share virtualized hardware resources.

[0096] The network 730 includes one or more wired and / or wireless networks. For example, the network 730 may include a cellular network (e.g., a fifth generation (5G) network, a Long-Term Evolution (LTE) network, a third generation (3G) network, a Code Division Multiple Access (CDMA) network, etc.), a public land mobile network (PLMN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a telephone network (e.g., a public switched telephone network (PSTN)), a private network, an ad hoc network, an intranet, the Internet, a fiber-optic-based network, etc., and / or a combination of these or other types of networks.

[0097] Figure 7 The number and arrangement of devices and networks shown are provided as an example. Figure 7 There may be more devices and / or networks, fewer devices and / or networks, different devices and / or networks, or differently arranged devices and / or networks than those shown. Figure 7 Two or more of the devices shown may be implemented in a single device, or Figure 7 The single device shown may be implemented by multiple distributed devices. Additionally or alternatively, one or more devices of environment 700 may perform one or more functions described as being performed by another set of devices of environment 200.

[0098] Figure 8 8 is a diagram of example components of device 800. Device 900 may correspond to user device 710 and / or platform 720. Figure 8 As shown, device 800 may include a bus 810 , a processor 820 , a memory 830 , a storage component 840 , an input component 850 , an output component 860 , and a communication interface 870 .

[0099] The bus 810 includes components that allow communication between components of the device 800. The processor 820 is implemented in hardware, firmware, or a combination of hardware and software. The processor 820 is a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or another type of processing component. In some embodiments, the processor 820 includes one or more processors that can be programmed to perform functions. The memory 830 includes a random access memory (RAM), a read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic memory, and / or optical memory) that stores information and / or instructions for use by the processor 820.

[0100] The storage component 840 stores information and / or software related to the operation and use of the device 800. For example, the storage component 940 may include a hard disk (e.g., a magnetic disk, an optical disk, a magneto-optical disk, and / or a solid-state disk), a compact disk (CD), a digital versatile disk (DVD), a floppy disk, a cassette, a magnetic tape, and / or another type of non-volatile computer-readable medium, and corresponding drives.

[0101] Input components 850 include components that allow device 800 to receive information, such as through user input, such as a touch screen display, a keyboard, a keypad, a mouse, buttons, switches, and / or a microphone. Additionally or alternatively, input components 850 may include sensors for sensing information (e.g., a global positioning system (GPS) component, an accelerometer, a gyroscope, and / or an actuator). Output components 860 include components that provide output information from device 800, such as a display, a speaker, and / or one or more light emitting diodes (LEDs).

[0102] The communication interface 870 includes a transceiver-like component (e.g., a transceiver and / or a separate receiver and transmitter) that enables the device 800 to communicate with other devices, for example, via a wired connection, a wireless connection, or a combination of wired and wireless connections. The communication interface 870 can allow the device 800 to receive information from another device and / or provide information to another device. For example, the communication interface 870 can include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency (RF) interface, a universal serial bus (USB) interface, a Wi-Fi interface, a cellular network interface, etc.

[0103] Device 800 can perform one or more processes described herein. Device 800 can perform these processes in response to processor 820 executing software instructions stored by non-volatile computer-readable media (e.g., memory 830 and / or storage component 840). Computer-readable media is defined herein as non-volatile memory devices. Memory devices include storage space within a single physical storage device or storage space distributed across multiple physical storage devices.

[0104] The software instructions may be read into the memory 830 and / or storage component 840 from another computer-readable medium or from another device via the communication interface 870. When executed, the software instructions stored in the memory 830 and / or storage component 840 may cause the processor 820 to perform one or more of the processes described herein. Additionally or alternatively, hardwired circuitry may be used in place of or in combination with software instructions to perform one or more of the processes described herein. Accordingly, the embodiments described herein are not limited to any specific combination of hardware circuitry and software.

[0105] Figure 8 The number and arrangement of components shown are provided as an example. Figure 8 The device 800 may include more components, fewer components, different components, or components arranged differently than those shown. Additionally or alternatively, one or more components of the device 800 may perform one or more functions described as being performed by another group of components of the device 800.

[0106] The foregoing disclosure provides illustration and description, but is not intended to be exhaustive or to limit the implementations to the precise form disclosed. Modifications and variations are possible in light of the above disclosure or may be acquired from practice of the implementations.

[0107] As used herein, the term component is intended to be broadly interpreted as hardware, firmware, or a combination of hardware and software.

[0108] Obviously, the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not limited to these implementations. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code - it should be understood that software and hardware can be designed based on the description herein to implement the systems and / or methods.

[0109] Although particular combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways not specifically recited in the claims and / or disclosed in the specification. Although each dependent claim listed below may be directly dependent on only one claim, the disclosure of possible implementations includes each dependent claim in combination with every other claim in the claim group.

[0110] Elements, actions or instructions used herein should not be interpreted as critical or necessary unless explicitly described as such. In addition, as used herein, the articles "one" and "an" are intended to include one or more projects and can be used interchangeably with "one or more". In addition, as used herein, the term "set" is intended to include one or more projects (e.g., related projects, unrelated projects, a combination of related projects and unrelated projects, etc.), and can be used interchangeably with "one or more". When only one project is meant, the term "one" or similar language is used. In addition, as used herein, the terms "has", "have", "having" etc. are intended to be open terms. Further, the phrase "based on" is intended to mean "based at least in part on", unless otherwise explicitly stated.

Claims

1. A method for encoding and decoding point cloud information, characterized in that: include: Acquire a point cloud, the point cloud comprising a set of points in a three-dimensional space; segmenting the point cloud into a plurality of objects and generating occupancy information for each of the plurality of objects; and encoding the occupancy information by using arithmetic coding with context modeling by considering distances between the plurality of objects and using coded nodes in the plurality of objects; The performing context modeling by considering the distances between the multiple objects and using the encoded nodes in the multiple objects includes: The encoded nodes having the same distance to the current node in the multiple objects are classified into the same type, each type including a group of encoded nodes, the context index of the current node is calculated using the number of occupied nodes in each group and the maximum number of nodes in each group, and the occupancy code of the current node is encoded based on the context index of the current node.

2. The method according to claim 1, characterized in that Encoding the occupancy information includes: The occupancy information is encoded object by object so that a next item of occupancy information to be encoded is encoded based on classification of surrounding objects among the plurality of objects, the surrounding objects being classified based on distances between the plurality of objects.

3. The method according to claim 1, characterized in that The context modeling using the encoded nodes in the plurality of objects by considering the distances between the plurality of objects includes: When encoding the occupancy information of a child node of a node in the plurality of objects, determining the final context index corresponding to the child node , where LUT is a lookup table, is the context index of the node, is the context index of the child node.

4. The method according to claim 1 or 2, characterized in that Segmenting the point cloud into multiple objects includes: The point cloud is segmented using a flexible tree segmentation scheme.

5. The method according to claim 4, characterized in that The plurality of objects include objects having different sizes.

6. The method according to claim 1 or 2, characterized in that Segmenting the point cloud into multiple objects includes: The point cloud is segmented using a depth-first segmentation scheme.

7. The method according to claim 5, characterized in that Encoding the occupancy information includes: The occupancy information is encoded by considering the size of the object among the plurality of objects.

8. The method according to claim 1 or 2, characterized in that Calculating the context index using the number of occupied nodes in each group and the maximum number of nodes in each group includes: Different classes are sorted based on their distance from the node. Sort by, is a set of coded objects that are ranked i-th in terms of proximity to the current object, and is the number of groups; Calculate the context index ,in, Representation Group The number of occupied nodes in , and Representation Group The maximum number of nodes in the.

9. The method according to claim 1 or 2, characterized in that Encoding the occupancy information includes: The occupancy information is encoded object by object so that a next item of occupancy information to be encoded is encoded based on the classification of surrounding objects among the plurality of objects, at least some of which have been encoded or partially encoded and are classified based on distances between the plurality of objects.

10. A device for encoding and decoding point cloud information, characterized in that: include: at least one memory for storing computer program code; as well as at least one processor configured to access the at least one memory and operate according to the computer program code, the computer program code comprising: Acquisition code for acquiring a point cloud, the point cloud comprising a set of points in a three-dimensional space; Segmentation code for segmenting the point cloud into a plurality of objects and generating occupancy information for each of the plurality of objects; and an encoding code for encoding the occupancy information by considering distances between the plurality of objects, using coded nodes in the plurality of objects for context modeling, and adopting arithmetic coding with the context modeling; Wherein, the encoding code is used for: The encoded nodes having the same distance to the current node in the multiple objects are classified into the same type, each type including a group of encoded nodes, the context index of the current node is calculated using the number of occupied nodes in each group and the maximum number of nodes in each group, and the occupancy code of the current node is encoded based on the context index of the current node.

11. The device according to claim 10, characterized in that The encoding code is used to encode the occupancy information object by object, so that the next item of occupancy information to be encoded is encoded based on the classification of surrounding objects among the plurality of objects, the surrounding objects being classified based on the distances between the plurality of objects.

12. The device according to claim 10 or 11, characterized in that The encoding code is used to: When encoding the occupancy information of a child node of a node in the plurality of objects, determining the final context index corresponding to the child node , where LUT is a lookup table, is the context index of the node, is the context index of the child node.

13. The device according to claim 10 or 11, characterized in that The segmentation code is used to segment the point cloud using a flexible tree segmentation scheme.

14. The device according to claim 13, characterized in that The plurality of objects include objects of different sizes.

15. The device according to claim 10 or 11, characterized in that The segmentation code is used to segment the point cloud using a depth-first segmentation scheme.

16. The device according to claim 14, characterized in that The encoding code is for encoding the occupancy information by considering the size of the object among the plurality of objects.

17. The device according to claim 10 or 11, characterized in that The encoding code is used to encode the occupancy information on an object-by-object basis, so that a next item of occupancy information to be encoded is encoded based on the classification of surrounding objects among the multiple objects, at least some of the surrounding objects have been encoded or partially encoded, and are classified based on the distances between the multiple objects.

18. A non-volatile computer-readable storage medium storing instructions, characterized in that: The instructions cause at least one processor to execute the method according to any one of claims 1-9.

Images