Method, computer system and storage medium for point cloud encoding and decoding

Abstract

A method, computer program, and computer system for point cloud encoding and decoding are provided. Data corresponding to a point cloud is received. Hash elements corresponding to attribute values ​​associated with the received data are reconstructed. The size of the hash table can be reduced based on deleting one or more hash elements in the hash elements, and one or more hash elements in the hash elements correspond to non-boundary areas associated with the attribute values. Data corresponding to the point cloud is decoded based on the reconstructed hash elements.

Description

[0001] Cross-reference related applications

[0002] This application claims the benefit of priority to U.S. patent application No. 17 / 317,053, filed on May 11, 2021, which claims priority based on U.S. Provisional Application No. 63 / 066,583, filed on August 17, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure generally relates to the field of data processing, and more particularly to point cloud encoding and decoding. Background Art

[0004] Point clouds have been widely used in recent years. For example, point clouds can be used for object detection and object positioning in autonomous vehicles; point clouds can also be used for mapping in Geographic Information Systems (GIS), and point clouds can also be used in cultural heritage to visualize and archive cultural heritage objects and collections, etc. A point cloud contains a set of high-dimensional points, usually three-dimensional (3D) points, each of which includes 3D location information and additional attributes such as color, reflectivity, etc. Point clouds can be captured using multiple cameras and depth sensors or lidar in various settings, and can consist of thousands to billions of points to truly represent the original scene. In order to transmit faster or reduce storage, compression technology is needed to reduce the amount of data required to represent the point cloud. Summary of the invention

[0005] Embodiments relate to methods, systems, and computer-readable media for point cloud encoding and decoding. According to one aspect, a method for point cloud encoding and decoding is provided. The method may include receiving data corresponding to a point cloud. Reconstructing a hash element corresponding to an attribute value associated with the received data. Decoding the data corresponding to the point cloud based on the reconstructed hash element.

[0006] According to another aspect, a computer system for point cloud encoding and decoding is provided. The computer system may include one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices, and program instructions stored on at least one of the one or more storage devices, and the program instructions are executed by at least one processor of the one or more processors through at least one memory of the one or more memories, so that the computer system can perform a method. The method may include: receiving data corresponding to a point cloud; reconstructing a hash element corresponding to an attribute value associated with the received data; and decoding the data corresponding to the point cloud based on the reconstructed hash element.

[0007] According to another aspect, a computer-readable medium for point cloud encoding and decoding is provided. The computer-readable medium may include one or more computer-readable storage devices and program instructions stored on at least one tangible storage device of the one or more tangible storage devices, and the program instructions may be executed by a processor. The program instructions may be executed by the processor to perform a method, which may include: receiving data corresponding to a point cloud; reconstructing a hash element corresponding to an attribute value associated with the received data; and decoding the data corresponding to the point cloud based on the reconstructed hash element. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] These and other objects, features and advantages will become apparent from the following detailed description of illustrative embodiments read in conjunction with the accompanying drawings. The various features of the drawings are not drawn to scale because the illustrations are intended to be clear and easy to understand by those skilled in the art in conjunction with the detailed description. In the drawings:

[0009] Figure 1 illustrates a networked computer environment according to at least one embodiment;

[0010] Figure 2A-2B is a block diagram of a point cloud update system according to at least one embodiment;

[0011] Figure 2C is a schematic diagram of a region adaptive hierarchical transform (RAHT) according to at least one embodiment;

[0012] Figure 2D is a schematic diagram of performing hash shrinking while maintaining boundary elements according to at least one embodiment;

[0013] Figure 2E is a table of syntax elements for predicting a residual according to at least one embodiment;

[0014] Figure 3 is an operational flow chart illustrating steps performed by a program for point cloud encoding and decoding according to at least one embodiment;

[0015] Figure 4 According to at least one embodiment Figure 1 block diagrams of internal and external components of computers and servers depicted in;

[0016] Figure 5 According to at least one embodiment, Figure 1 A block diagram of an illustrative cloud computing environment of a computer system depicted in; and

[0017] Figure 6 According to at least one embodiment Figure 5 A block diagram of the functional layers of an illustrative cloud computing environment. DETAILED DESCRIPTION

[0018] Detailed embodiments of the claimed structures and methods are disclosed herein; however, it is understood that the disclosed embodiments are merely illustrative of the claimed structures and methods, which may be implemented in various forms. However, these structures and methods may be implemented in many different forms and should not be construed as limited to the exemplary embodiments set forth herein. Instead, these exemplary embodiments are provided so that the disclosure will be thorough and complete and fully convey the scope to those skilled in the art. In the description, details of well-known features and techniques may be omitted to avoid unnecessarily obscuring the presented embodiments.

[0019] Embodiments generally relate to the field of data processing, and more specifically to point cloud encoding and decoding. The exemplary embodiments described below provide a system, method, and computer program in which a hash-based approach is used to encode and decode point cloud attribute data. Therefore, some embodiments can improve the field of computing by making point cloud encoding and decoding more efficient without the need to find the nearest neighbors of the point cloud data.

[0020] As mentioned above, point clouds have been widely used in recent years. For example, point clouds can be used for object detection and object positioning in autonomous driving vehicles; point clouds can also be used for mapping in Geographic Information Systems (GIS), and point clouds can be used in cultural heritage for visualization and archiving of cultural heritage objects and collections. A point cloud contains a set of high-dimensional points, usually three-dimensional (3D) points, each of which includes 3D position information and additional attributes such as color, reflectivity, etc. Point clouds can be captured using multiple cameras and depth sensors or lidar in various settings, and can consist of thousands to billions of points to truly represent the original scene. In order to transmit faster or reduce storage, compression technology is needed to reduce the amount of data required to represent the point cloud. In prediction-based attribute encoding and decoding, the attributes of the current point are predicted from the already encoded points close to the current point.

[0021] In the TMC13 model, geometric information and related attributes such as color or reflectivity are compressed separately. The geometric information, i.e. the 3D coordinates of the point cloud, is encoded and decoded with its occupancy information by octree partitioning. The attributes are then compressed based on the reconstructed geometry using prediction, lifting and region adaptive hierarchical transform techniques. However, finding the nearest point in 3D space for each point can be expensive. In addition, for lossless and near-lossless coding of multiple channel attributes (e.g., RGB color), multiple channels are processed directly without color space conversion and encoded and decoded independently. However, especially in the RGB domain, there is often a strong correlation between different color channels. Not exploiting this correlation may result in performance losses in terms of codec efficiency. Therefore, it may be advantageous to use a hash-based prediction scheme and inter-component correlations.

[0022] Various aspects are described herein with reference to flowchart illustrations and / or block diagrams of methods, devices (systems) and computer-readable media according to various embodiments. It should be understood that each block of the flowchart illustrations and / or block diagrams and the combination of blocks in the flowchart illustrations and / or block diagrams can be implemented by computer-readable program instructions.

[0023] Reference now Figure 1 , showing a functional block diagram of a networked computer environment of a point cloud encoding and decoding system 100 (hereinafter referred to as a “system”) for encoding and decoding cloud data. It should be understood that Figure 1 This is merely an illustration of one implementation and does not imply any limitations with respect to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made based on design and implementation requirements.

[0024] System 100 may include computer 102 and server computer 114. Computer 102 may communicate with server computer 114 via communication network 110 (hereinafter referred to as "network"). Computer 102 may include processor 104 and software program 108 stored on data storage device 106, and computer 102 may interact with a user and communicate with server computer 114. Figure 4 As discussed, computer 102 may include internal components 800A and external components 900A, respectively, and server computer 114 may include internal components 800B and external components 900B, respectively. Computer 102 may be, for example, a mobile device, a phone, a personal digital assistant, a netbook, a laptop, a tablet computer, a desktop computer, or any type of computing device capable of running programs, accessing a network, and accessing a database.

[0025] As referenced below Figure 5 and Figure 6As discussed, the server computer 114 may also operate in a cloud computing service model, such as Software as a Service (SaaS), Platform as a Service (PaaS), or Infrastructure as a Service (IaaS). The server computer 114 may also be located in a cloud computing deployment model, such as a private cloud, a community cloud, a public cloud, or a hybrid cloud.

[0026] The server computer 114 can be used for point cloud encoding and decoding, and can run a point cloud encoding and decoding program 116 (hereinafter referred to as "program") that can interact with the database 112. Figure 3 The point cloud codec program method is described in more detail. In one embodiment, the computer 102 can operate as an input device including a user interface, and the program 116 can be primarily run on the server computer 114. In an alternative embodiment, the program 116 can be primarily run on one or more computers 102, and the server computer 114 can be used to process and store data used by the program 116. It should be noted that the program 116 can be a stand-alone program, or can be integrated into a larger point cloud codec program.

[0027] However, it should be noted that in some cases, the processing of program 116 may be shared in any proportion between computer 102 and server computer 114. In another embodiment, program 116 may be executed on more than one computer, server computer, or some combination of computers and server computers, such as, for example, on multiple computers 102 communicating with a single server computer 114 via network 110. In another embodiment, for example, program 116 may be executed on multiple server computers 114 communicating with multiple client computers via network 110. Alternatively, the program may be executed on a network server that communicates with the server and multiple client computers via the network.

[0028] The network 110 may include a wired connection, a wireless connection, a fiber optic connection, or some combination of these connections. In general, the network 110 may be any combination of connections and protocols that will support communication between the computer 102 and the server computer 114. The network 110 may include various types of networks, such as a local area network (LAN), a wide area network (WAN) such as the Internet, a telecommunication network such as a public switched telephone network (PSTN), a wireless network, a public switched network, a satellite network, a cellular network (e.g., a fifth generation (5G) network, a long-term evolution (LTE) network, a third generation (3G) network, a code division multiple access (CDMA) network, etc.), a public land mobile network (PLMN), a metropolitan area network (MAN), a private network, an ad hoc network, an intranet, a fiber-based network, etc., and / or a combination of these networks or other types of networks.

[0029] Figure 1 The number and arrangement of devices and networks shown are provided as examples. In practice, there may be additional devices and / or networks, fewer devices and / or networks, different devices and / or networks, or different Figure 1 The devices and / or networks shown are arranged differently. Figure 1 Two or more of the devices shown may be implemented in a single device, or Figure 1 The single device shown may be implemented as multiple distributed devices. Additionally or alternatively, one set of devices (eg, one or more devices) of system 100 may perform one or more functions described as being performed by another set of devices of system 100.

[0030] Reference now Figure 2A and 2B , block diagrams 200A and 200B of a point cloud update system are shown. Block diagram 200A may depict a direct or forward transformation in a lifting scheme. Block diagram 200B may depict an inverse transformation in a lifting scheme.

[0031] For prediction-based attribute encoding and decoding, let (P i ) i＝1…N is the set of locations associated with the point cloud points, and let (M i ) i＝1…NFor (P i ) i＝1…N First, sort the points in ascending order according to their associated Morton codes. Let I be the array of point indices sorted according to this process. The encoder / decoder compresses / decompresses the points respectively according to the order defined by I. In each iteration i, a point P is selected i . Analysis of P i distance to s (e.g., s=64) previous points, and select P i k (e.g., k=3) nearest neighbors of point i are used for prediction. More precisely, the attribute value (a) is predicted by using a linear interpolation process based on the distances of the nearest neighbors of point i. i ) i∈0…k-1 .set up is the set of k nearest neighbors of the current point i, and let are their decoded / reconstructed property values ​​and is their distance to the current point. Predicted attribute value Given by:

[0032]

[0033] The lifting-based attribute codec is built on the prediction-based codec. The main difference compared to the prediction-based scheme is the introduction of two additional steps. The first is the introduction of the update operator. The second is the use of an adaptive quantization strategy.

[0034] Reference now Figure 2C , shows a schematic diagram 200C of a region adaptive hierarchical transform (RAHT). The block diagram 200C may depict a forward transform 202A and a reverse transform 202B. For RAHT encoding, and w0 is the input coefficient F l+1,2n The sign of the weight, and w1 for F l+1,2n+1 The same is true.

[0035] Reference now Figure 2D , a schematic diagram 200D is shown of performing hash shrinking while maintaining boundary elements. The hash shrinking may include hash elements 206A-C, boundary hash elements 208A-C, and current node 210. According to one or more embodiments, the encoded geometric occupancy information may be saved in a cache memory because a later encoded node needs to use the information as context.

[0036] The encoded and decoded attribute information can be saved in a hash table, and the later encoded and decoded nodes can retrieve them as predictors. The hash table is used to store the reconstructed attribute values. Specifically, a hash table H is maintained, where the key value is the Morton code of the 3D coordinates of the point, i.e., M i =Morton(x i ,y i ,z i ), where (x i ,y i ,z i ) is the 3D coordinate of the i-th point. Use Morton encoding M i As a key value, you can directly access its attribute value reconstructed in the hash table H. If H(M i ) is NULL, it means that the position (x i ,y i ,z i ) is unoccupied and has no attribute value associated with it, or is not associated with the position (x i ,y i ,z i )The associated attributes have not yet been encoded.

[0037] When encoding / decoding the attribute value of the current point, the previously encoded and decoded attributes of the neighboring points are obtained from the hash table H and used as predictors. After encoding / decoding the attribute value of the current node, the reconstructed attributes of the current node are stored in H.

[0038] Assume that the current point has i ,y i ,z i ), then find the geometric position at (x i -Δ x ,y i -Δ y ,z i -Δ z ) is negligible, where Δ x , Δ y and Δ z is the search step size with small values. However, if the search range is large, searching the space is time consuming because the search space increases cubically to the search step size.

[0039] To speed up the search, a hierarchical structure with multiple hash tables can be constructed. Each hash table contains the attribute values ​​at a specific octree partition level. For each octree partition depth d, a hash table H is maintained. d , where the key value is the Morton code of the octree node at depth d, i.e. in, is the 3D coordinate of the octree node at depth d. Using Morton encoding As a key value, you can access the hash table H d The attribute value in H. d The attribute value in may be the average attribute of all points in the corresponding octree node, or the median attribute value of all points in the corresponding octree node, or the weighted average attribute value of all points in the corresponding octree node. Multiple hash tables may be maintained from the leaf node level to some level above.

[0040] When searching, either bottom-up or top-down search can be applied until enough prediction candidates are found.

[0041] To optimize / speed up hash implementations, keeping the hash table as small as possible can allow for less expensive storage of information and faster searches. In one embodiment, a maximum hash table size is defined. Note that the maximum size can be fixed for all cases, or can be configured differently on a case-by-case basis and sent in the bitstream as part of a high-level syntax, such as a sequence parameter set, geometry parameter set, or slice header. When the hash table reaches maximum capacity, the hash table will shrink itself by removing some of its elements. The rules for which elements will be removed may be different in multiple cases. In one embodiment, the hash table will remove elements that are not at block boundaries. Block boundaries can be defined based on the position coordinates of the current node. Typically, block sizes are defined as powers of 2. For example, the block size can be (2 in 3D space) or (2 M ,2 M ,2 M ). Then, we have iM -1 (i = 1, 2, 3, ...) is a boundary. Therefore, all hash elements at the boundary are kept, and all remaining elements can be deleted from the table. Elements that do not belong to the boundary can be removed. Note that the boundary size M can be fixed for all cases, or can be configured differently on a case-by-case basis and sent in the bitstream as part of a high-level syntax, such as a sequence parameter set, a geometry parameter set, or a slice header.

[0042] A node-based encoding and decoding method is proposed, in which the entire point cloud can be divided into several child nodes by breadth-first octree partitioning, and each child node can be encoded and decoded differently or in parallel. In this scheme, each child node can maintain a separate hash table, and each hash table from different child nodes is independent of each other. In this case, since each child node is part of the original point cloud, the maximum size of the hash table is also reduced. However, this may introduce some performance loss in terms of encoding and decoding efficiency because context information is not shared across node boundaries. To compensate for the loss, in another embodiment, the elements at the node boundaries are maintained and can be used for different child nodes.

[0043] In order to exploit the correlation between different components, prediction schemes can be applied in different attribute coding methods, such as prediction-based attribute coding, lifting-based attribute coding and RAHT-based attribute coding. Cross-component prediction can be applied to the original signal or prediction residual or transform coefficients (if any transform technique, such as RAHT, is applied). In the following embodiments, prediction-based attribute coding is used as an example, and the cross-component prediction scheme is applied to the prediction residuals of different channels (components).

[0044] Different models can be established to predict the signal cross-component, such as linear models, quadratic models and other polynomial models. In the following embodiments, a linear model is used as an example.

[0045] Cross-component prediction is not limited to RGB color attributes, but can be used for any other attributes as long as there is more than one type of attribute to be encoded and decoded and there is a correlation between them. In the following embodiments, RGB color attributes will be used as examples.

[0046] Assume that the prediction residuals of the R, G and B components are r0, r1, r2 respectively, and assume that these signals are linearly dependent. These components can be encoded and decoded in any order. The encoding and decoding order can be fixed for all cases, or can be configured differently for each case and specified in the bitstream as part of the high-level syntax, such as a sequence parameter set, a geometry parameter set, or a slice header. In the following embodiments, it is assumed that the encoding and decoding order is R first, followed by G and B. Therefore, when encoding and decoding the prediction residual of G, the reconstructed prediction residual of R can be used as a prediction; when encoding and decoding the prediction residual of B, the reconstructed prediction residuals of R and G can be used for prediction.

[0047] More specifically, assume that the prediction residual of the R component, i.e., r0, is first encoded and decoded, and its reconstructed value after quantization / dequantization is R0. Then, the prediction residual of the G component, i.e., r1, can be predicted by R0 as follows:

[0048] p1=α0R0+β0

[0049] where α0 and β0 are the linear model parameters, and p1 is the predicted residual of G. Therefore, instead of directly

[0050] Then, the second-order residual of G, i.e. r1, is encoded, and its reconstructed value after quantization / dequantization is R1′.

[0051] And the reconstructed prediction residual of the G component is R1=R1′+p1. Finally, the prediction residual of the B component, i.e. r2, can be predicted by R0 and R1.

[0052] p2=α1R0+β1R1+γ1,

[0053] Among them, α1, β1 and γ1 are linear model parameters, and p2 is the prediction residual of the prediction of B. Therefore, instead of directly encoding the prediction residual, i.e., r2, the second-order residual between r2 and p2 can be encoded. Then, the second-order residual of B, i.e., r2′, is encoded, and its reconstructed value after quantization / dequantization is R2′, and the reconstructed prediction residual of the B component is R2=R′2+p2.

[0054] As shown below, the linear model can be simplified in many ways. However, it is noted that the simplification method can be any other similar form. In one embodiment, the linear model is simplified as follows:

[0055] p1=α0R0, p2=α1R0+β1R1,

[0056] Where β0 and γ1 are fixed to zero. In another embodiment, the linear model can be simplified as follows,

[0057] p1=α0R0,p2=α1R0,

[0058] Where β0, γ1 and β1 are fixed to zero. In another embodiment, the linear model can be simplified as follows,

[0059] p1=α0R0,p2=β1R1,

[0060] Where β0, γ1 and α1 are fixed to zero. In another embodiment, the linear model can be simplified as follows,

[0061] p1=α0R0,p2=α0R0,

[0062] Where β0 and γ1 and β1 are fixed to zero. And In another embodiment, the linear model can be simplified as follows,

[0063] p1=α0R0,p2=α0R1

[0064] Among them, β0, γ1 and α1 are fixed to zero, and β1 = α0.

[0065] In the above linear model, the model parameters are fixed for the encoder and decoder and for all cases. In one embodiment, the prediction can be derived as:

[0066] p1=R0,p2=R0;

[0067] p1=R0, p2=R1; or

[0068] p1=R0,

[0069] However, the model parameters may be different for different point clouds, and these parameters may be signaled in a high-level syntax, such as in a sequence parameter set, an attribute parameter set, or a slice header, etc.

[0070] In other embodiments, the model parameters may also be signaled as "low-level syntax", which means that each codec unit (node) may have a different set of parameters.

[0071] In another embodiment, in order to avoid explicitly signaling the model parameters, the parameters can be adaptively learned in real time from the reconstructed prediction residuals. The encoder and decoder can perform the same training phase by minimizing the mean square error while encoding and decoding.

[0072] Reference now Figure 2E , Table 200E showing the syntax elements for prediction residuals. In order to more efficiently encode the prediction residuals, different attribute components can use separate contexts. In one embodiment, the R, G, and B components all use different context models. prediction_residual_is_zero[i] specifies whether the prediction residual of the i-th component is equal to 0. prediction_residual_is_one[i] specifies whether the prediction residual of the i-th component is equal to 1. prediction_residual_is_two[i] specifies whether the prediction residual of the i-th component is equal to 2. prediction_residual_minus_three[i] specifies that the prediction residual of the i-th component is prediction_residual_minus_three[i]+3. In another embodiment, the R component uses a separate context model, while the G and B components use the same context model.

[0073] Reference now Figure 3 , showing an operational flow chart of the steps of method 300 performed by a program for point cloud encoding and decoding.

[0074] At 302 , method 300 may include receiving data corresponding to a point cloud.

[0075] At 304, method 300 may include reconstructing a hash element corresponding to a property value associated with the received data.

[0076] At 306 , method 300 may include decoding data corresponding to the point cloud based on the reconstructed hash elements.

[0077] Understandably, Figure 3 This is merely an illustration of one implementation and does not imply any limitations regarding how different embodiments may be implemented. Many modifications to the depicted environments may be made based on design and implementation requirements.

[0078] In one implementation, a size of a hash table is reduced based on deleting one or more hash elements, the one or more hash elements corresponding to non-boundary regions associated with attribute values, wherein the hash table includes a plurality of hash elements.

[0079] In one embodiment, a maximum size of the hash table is defined for data corresponding to the point cloud.

[0080] In one embodiment, the maximum size of the hash table is signaled in the sequence parameter set, geometry parameter set or slice header.

[0081] In one embodiment, the hash table is reduced by removing some elements from the hash table based on the hash table reaching a maximum size.

[0082] In one embodiment, the hash table is reduced based on a Morton order associated with the hash elements.

[0083] In one embodiment, in a sequence parameter set, a geometry parameter set, or a slice header, a boundary size associated with one or more neighboring nodes is signaled, where the nodes are obtained by partitioning a point cloud by a breadth-first octree, and the block size of the point cloud is defined as a power of 2, and the size of the block in 3D space is (2 M ,2 M ,2 M ), with a value equal to 2 iM One or more neighboring nodes of the position coordinate of -1 are boundaries, where i=1, 2, 3, ..., and M is the boundary size.

[0084] Figure 4 According to an illustrative embodiment Figure 1 A block diagram 400 of the internal and external components of a computer is depicted. It should be understood that Figure 4 This is merely an illustration of one implementation and does not imply any limitations with respect to the environments in which different embodiments may be implemented. Many modifications to the depicted environment may be made based on design and implementation requirements.

[0085] like Figure 5 As shown, the computer 102 ( Figure 1 ) and server computer 114 ( Figure 1 ) may include a set of internal components 800A, 800B and external components 900A, 900B, respectively. Each set of internal components 800 includes one or more processors 820, one or more computer-readable RAMs 822 and one or more computer-readable ROMs 824, one or more operating systems 828, and one or more computer-readable tangible storage devices 830 located on one or more buses 826.

[0086] The processor 820 is implemented in hardware, firmware, or a combination of hardware and software. The processor 820 is a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or other types of processing components. In some implementations, the processor 820 includes one or more processors that can be programmed to perform functions. The bus 826 includes components that allow communication between internal components 800A and 800B.

[0087] Located on server computer 114 ( Figure 1 ) on one or more operating systems 828, software programs 108 ( Figure 1 ) and point cloud codec 116 ( Figure 1 ) are stored on one or more of the corresponding computer-readable tangible storage devices 830 for execution by one or more of the corresponding processors 820 through one or more of the corresponding RAMs 822 (typically including cache). Figure 4In the illustrated embodiment, each computer-readable tangible storage device 830 is a magnetic disk storage device of an internal hard drive. Alternatively, each computer-readable tangible storage device 830 is a semiconductor storage device such as ROM 824, EPROM, flash memory, optical disk, magneto-optical disk, solid state disk, compact disk (CD), digital versatile disk (DVD), floppy disk, cassette, magnetic tape, and / or other types of non-transitory computer-readable tangible storage devices that can store computer programs and digital information.

[0088] Each set of internal components 800A, 800B also includes an R / W drive or interface 832 to read from and write to one or more portable computer readable tangible storage devices 936 (e.g., CD-ROM, DVD, memory stick, tape, disk, optical disk, or semiconductor storage device). Figure 1 ) and point cloud codec 116 ( Figure 1 ) can be stored on one or more of the corresponding portable computer-readable tangible storage devices 936, read via the corresponding R / W drive or interface 832 and loaded into the corresponding hard disk drive 830.

[0089] Each set of internal components 800A, 800B also includes a network adapter or interface 836, such as a TCP / IP adapter card; a wireless Wi-Fi interface card; or a 3G, 4G or 5G wireless interface card or other wired or wireless communication link. Server computer 114 ( Figure 1 ) on the software program 108 ( Figure 1 ) and point cloud codec 116 ( Figure 1 ) can be downloaded from an external computer to the computer 102 via a network (eg, the Internet, a local area network, or other wide area network) and a corresponding network adapter or interface 836 ( Figure 1 ) and the server computer 114. The software program 108 and the point cloud codec program 116 on the server computer 114 are loaded into the corresponding hard disk drive 830 through the network adapter or interface 836. The network may include copper wires, optical fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers.

[0090] Each set of external components 900A, 900B may include a computer display 920, a keyboard 930, and a computer mouse 934. The external components 900A, 900B may also include a touch screen, a virtual keyboard, a touch pad, a pointing device, and other human-computer interface devices. Each set of internal components 800A, 800B also includes a device driver 840 to interface with the computer display 920, the keyboard 930, and the computer mouse 934. The device driver 840, the R / W driver or interface 832, and the network adapter or interface 836 include hardware and software (stored in the storage device 830 and / or ROM 824).

[0091] It should be understood in advance that although the present disclosure includes a detailed description of cloud computing, the implementation of the teachings described herein is not limited to a cloud computing environment. Instead, some embodiments can be implemented in conjunction with any other type of computing environment now known or later developed.

[0092] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be quickly provisioned and released with minimal management effort or through interaction with a service provider. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0093] Features are as follows:

[0094] On-demand self-service: Cloud consumers can automatically and unilaterally provision computing capacity (such as server time and network storage) as needed without the need for manual interaction with the service provider.

[0095] Broad network access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0096] Resource pooling: Pooling of a provider's computing resources to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. There is a sense of location independence because consumers generally do not control or know the exact location of the provided resources, but are able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0097] Rapid elasticity: Capacity can be provisioned quickly and elastically, in some cases automatically, to scale up quickly and release quickly to scale down quickly. To the consumer, the available capacity often appears to be unlimited and can be purchased at any time and in any quantity.

[0098] Measured services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the utilized services.

[0099] The service model is as follows:

[0100] Software as a Service (SaaS): The capability provided to the consumer is to use the provider's applications running on the cloud infrastructure. The applications are accessible from a variety of client devices through a thin client interface, such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0101] Platform as a Service (PaaS): The capability provided to consumers is to deploy applications created or acquired by consumers onto cloud infrastructure, where the applications are created using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure including networks, servers, operating systems or storage, but rather control the deployed applications and possible configuration of the application hosting environment.

[0102] Infrastructure as a Service (IaaS): The capabilities provided to consumers are the provision of processing, storage, networking, and other basic computing resources, where consumers can deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but control the operating system, storage, deployed applications, and may have limited control over selected networking components (e.g., host firewalls).

[0103] The deployment model is as follows:

[0104] Private cloud: Cloud infrastructure is run for the sole use of one organization. The cloud infrastructure can be managed by that organization or a third party and can exist on-premises or off-premises.

[0105] Community cloud: Cloud infrastructure is shared by several organizations and supports a specific community with common concerns (e.g., mission, security requirements, policy, and compliance considerations). Community clouds can be managed by these organizations or by a third party and can exist on-premises or off-premises.

[0106] Public cloud: Cloud infrastructure is available to the general public or large industrial groups and is owned by an organization that sells cloud services.

[0107] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain distinct entities but are tied together by standardized technologies or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0108] The cloud computing environment is service-oriented, with an emphasis on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is the infrastructure consisting of a network of interconnected nodes.

[0109] refer to Figure 5 , depicts an illustrative cloud computing environment 500. As shown, the cloud computing environment 500 includes one or more cloud computing nodes 10, and local computing devices used by cloud consumers can communicate with the cloud computing nodes 10, such as personal digital assistants (PDA) or cellular phones 54A, desktop computers 54B, laptop computers 54C and / or automobile computer systems 54N. The cloud computing nodes 10 are communicative with each other. The cloud computing nodes 10 can be physically or virtually grouped in one or more networks (not shown), such as a private cloud, community cloud, public cloud, or hybrid cloud as described above, or a combination thereof. This enables the cloud computing environment 500 to provide infrastructure, platforms, and / or software as services that do not require cloud consumers to maintain resources on local computing devices. It should be understood that Figure 5 The types of computing devices 54A-N shown are for illustration only, and cloud computing node 10 and cloud computing environment 500 may communicate with any type of computerized device over any type of network and / or network addressable connection (eg, using a web browser).

[0110] Reference Figure 6 , showing a cloud computing environment 500 ( Figure 5 ) provides a set of functional abstraction layers 600. It should be understood in advance that Figure 6 The components, layers and functions shown in are for illustration only, and the embodiments are not limited thereto. As depicted in the figure, the following layers and corresponding functions are provided:

[0111] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: host 61; server 62 based on Reduced Instruction Set Computer (RISC) architecture; server 63; blade server 64; storage device 65; and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0112] Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities may be provided: virtual servers 71 ; virtual storage 72 ; virtual networks 73 , including virtual private networks; virtual applications and operating systems 74 ; and virtual clients 75 .

[0113] In one example, the management layer 80 may provide the functionality described below. Resource provisioning 81 provides dynamic acquisition of computing resources and other resources for performing tasks in a cloud computing environment. Metering and pricing 82 provides cost traceability when resources are used in a cloud computing environment, as well as billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides consumers and system administrators with access to the cloud computing environment. Service level management 84 provides cloud computing resource allocation and management so that the required service levels are met. Service Level Agreement (SLA) planning and implementation 85 provides pre-scheduling and acquisition of cloud computing resources, for which future demand is anticipated based on the SLA.

[0114] The workload layer 90 provides examples of functions that can use the cloud computing environment. Examples of workloads and functions that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analysis processing 94; transaction processing 95; and point cloud encoding and decoding 96. Point cloud encoding and decoding 96 can decode point cloud data based on hash-based attribute encoding.

[0115] Some embodiments may involve systems, methods, and / or computer-readable media incorporating any possible level of technical detail. A computer-readable medium may include a computer-readable non-transitory storage medium (or medium) having computer-readable program instructions thereon that cause a processor to perform operations.

[0116] A computer-readable storage medium may be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above storage devices. A non-exhaustive list of more specific examples of computer-readable storage media includes the following items: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device (e.g., a raised structure in a groove or a punched card with instructions recorded thereon), and any suitable combination of the above items. Computer-readable storage media as used herein should not be construed as being a transient signal in itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or an electrical signal transmitted through a wire.

[0117] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device or to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium in the corresponding computing / processing device.

[0118] The computer readable program code / instructions for performing operations may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, configuration data of an integrated circuit, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(e.g., SmallTalk, C++, etc.) and procedural programming languages ​​(e.g., "C" programming language or similar programming languages). The computer readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, which may be partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or a connection may be established with an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), can execute computer-readable program instructions by utilizing state information of the computer-readable program instructions to personalize the electronic circuits to perform various aspects or operations.

[0119] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a way to implement the functions / actions specified in the flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium, which may direct the computer, programmable data processing device, and / or other device to operate in a specific manner, so that the computer-readable storage medium storing the instructions includes an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified in the flowchart and / or block diagram.

[0120] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing device, or other device, so that a series of operating steps are executed on the computer, other programmable device, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable device, or other device implement the functions / actions specified in the flowchart and / or block diagram.

[0121] The flowchart and block diagram in the figure show the architecture, function and operation of the possible implementation of the system, method and computer-readable medium according to various embodiments.At this point, each frame in the flowchart or block diagram may represent a module, section or part of an instruction, which includes one or more executable instructions for implementing a specified logical function.Compared with those frames depicted in the figure, the method, computer system and computer-readable medium may include additional frames, fewer frames, different frames or frames of different arrangements.In some optional implementations, the function noted in the frame may not occur in the order noted in the figure.For example, the two frames shown in succession can actually be executed simultaneously or substantially simultaneously, or sometimes the frame can be executed in the opposite order, depending on the function involved.It should also be noted that each frame illustrated in the block diagram and / or the flowchart and the combination of the frames illustrated in the block diagram and / or the flowchart can be implemented by a system based on special-purpose hardware, which performs a specified function or action or realizes a combination of special-purpose hardware and computer instructions.

[0122] Obviously, the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or a combination of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, the operation and behavior of the systems and / or methods are described herein without reference to specific software code - it should be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0123] Unless otherwise clearly described, the elements, actions or instructions used herein should not be understood as critical or necessary. In addition, as used herein, the articles "one" and "an" are intended to include one or more projects, and can be used interchangeably with "one or more". In addition, as used herein, the term "set" is intended to include one or more projects (e.g., related projects, unrelated projects, combinations of related projects and unrelated projects, etc.), and can be used interchangeably with "one or more". In the case of only one project being meant, the term "one" or similar language is used. In addition, as used herein, the term "have", "have", "contain" or similar terms are intended to be open terms. In addition, the phrase "based on" means "based at least in part on", unless otherwise clearly stated.

[0124] Descriptions of various aspects and embodiments have been given for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Even if combinations of features are described in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of possible implementations. In fact, many of these features can be combined in ways that are not specifically stated in the claims and / or disclosed in the specification. Although each dependent claim listed below may be directly subordinate to only one claim, the disclosure of possible implementations includes the combination of each dependent claim with every other claim in the claim set. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements found in the market, or to enable those of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A point cloud encoding and decoding method, comprising: receiving data corresponding to the point cloud; reconstructing, based on hash elements corresponding to attribute values ​​of octree partition levels, hash elements corresponding to attribute values ​​associated with the received data; and The data corresponding to the point cloud is decoded based on the reconstructed hash elements.

2. The method according to claim 1, further comprising: The size of the hash table is reduced based on deleting one or more of the hash elements, the one or more hash elements corresponding to the non-boundary regions associated with the attribute value, wherein the hash table includes a plurality of hash elements.

3. The method according to claim 2, wherein: A maximum size of the hash table is defined for the data corresponding to the point cloud.

4. The method according to claim 3, wherein: The maximum size of the hash table is signaled in a sequence parameter set, geometry parameter set or slice header.

5. The method according to claim 3, wherein: Based on the hash table reaching the maximum size, the hash table is reduced by removing some elements from the hash table.

6. The method according to claim 2, wherein: The hash table is reduced based on a Morton order associated with the hash elements.

7. The method according to claim 1, wherein: In a sequence parameter set, a geometry parameter set or a slice header, a boundary size associated with one or more neighboring nodes is signaled, wherein the nodes are obtained by partitioning the point cloud by a breadth-first octree, the block size of the point cloud is defined as a power of 2, the size of the block is (2^M, 2^M, 2^M) in 3D space, and the one or more neighboring nodes having position coordinates equal to 2^iM-1 are boundaries, where i=1, 2, 3, ..., and M is the boundary size.

8. A computer system for point cloud encoding and decoding, the computer system comprising: one or more computer-readable non-transitory storage media configured to store computer program code; as well as One or more computer processors configured to access the computer program code and operate according to the instructions of the computer program code, the computer program code comprising: receiving code configured to cause the one or more computer processors to receive data corresponding to a point cloud; reconstruction code configured to cause the one or more computer processors to reconstruct hash elements corresponding to attribute values ​​associated with the received data based on hash elements corresponding to attribute values ​​of octree partition levels; and A decoding code is configured to cause the one or more computer processors to decode the data corresponding to the point cloud based on the reconstructed hash element.

9. The computer system of claim 8, further comprising reduction code configured to reduce a size of the hash table based on deleting one or more of the hash elements, the one or more hash elements corresponding to non-boundary regions associated with the attribute value, wherein The hash table includes a plurality of hash elements.

10. The computer system according to claim 9, wherein: A maximum size of the hash table is defined for the data corresponding to the point cloud.

11. The computer system according to claim 10, wherein: The maximum size of the hash table is signaled in a sequence parameter set, geometry parameter set or slice header.

12. The computer system according to claim 10, wherein: Based on the hash table reaching the maximum size, the hash table is reduced by removing some elements from the hash table.

13. The computer system according to claim 9, wherein: The hash table is reduced based on a Morton order associated with the hash elements.

14. The computer system according to claim 8, wherein: In a sequence parameter set, a geometry parameter set or a slice header, a boundary size associated with one or more neighboring nodes is signaled, wherein the nodes are obtained by partitioning the point cloud by a breadth-first octree, the block size of the point cloud is defined as a power of 2, the size of the block is (2^M, 2^M, 2^M) in 3D space, and the one or more neighboring nodes having position coordinates equal to 2^iM-1 are boundaries, where i=1, 2, 3, ..., and M is the boundary size.

15. A non-transitory computer-readable medium having stored thereon a computer program for point cloud encoding and decoding, wherein the computer program is configured to cause one or more computer processors to execute the method according to any one of claims 1 to 7.

16. A method for processing point cloud data, characterized in that: The point cloud data is generated by encoding and decoding based on the method described in any one of claims 1 to 7.

Images