Point cloud data processing method and device, storage medium and electronic equipment

Abstract

The invention discloses a point cloud data processing method and device, a storage medium and electronic equipment. The method comprises the following steps: acquiring initial point cloud data to be sent; dividing the initial point cloud data into a group of point cloud blocks; performing graphical representation on the basis of the point cloud data in each point cloud block in the group of point cloud blocks to generate a group of graph structure data, one graph structure data in the group of graph structure data corresponding to one point cloud block in the group of point cloud blocks, and the graph structure data being used for representing features of the corresponding point cloud block; and encoding the group of graph structure data to obtain a group of data packets, and sending the group of data packets to a target terminal so as to perform decoding reconstruction at the target terminal to obtain target point cloud data. The technical problems that the processing efficiency of the point cloud data is low and the effect cannot be quickly achieved in the related technology are solved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular, to a method and device for processing point cloud data, a storage medium, and an electronic device. Background Art

[0002] As the collection and application of large-scale point cloud data become more and more common. However, traditional point cloud processing and transmission methods have many technical problems and challenges when facing large-scale point cloud data. Traditional point cloud processing methods usually operate directly on the original point coordinates and attributes. Although this method is simple and intuitive, it is difficult to fully utilize the geometric structure and topological information of point cloud data, resulting in low efficiency when processing complex geometric relationships. At the same time, the lack of a comprehensive description of the intrinsic structure of the data limits the accuracy and efficiency of data processing and analysis. With the continuous increase in the scale of point cloud data, traditional point cloud processing methods face huge computing and storage challenges. Processing the entire point cloud data as a whole can easily lead to bottlenecks in computing resources and storage space, low processing efficiency, and difficulty in meeting the needs of real-time processing.

[0003] Traditional point cloud processing and transmission methods are difficult to meet the needs of real-time user interaction and real-time data update. When users need to interact with point cloud data (such as rotation, scaling, local magnification, etc.) or when point cloud data changes, traditional methods cannot respond quickly, resulting in poor user experience.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method and device for processing point cloud data, a storage medium, and an electronic device, so as to at least solve the technical problem in the related art that the processing efficiency of point cloud data is low and the effect cannot be quickly achieved.

[0006] According to one aspect of an embodiment of the present application, a method for processing point cloud data is provided, including: obtaining initial point cloud data to be sent; dividing the initial point cloud data into a group of point cloud blocks; generating a group of graph structure data based on a graphical representation of the point cloud data in each point cloud block in the group of point cloud blocks, wherein a graph structure data in the group of graph structure data corresponds to a point cloud block in the group of point cloud blocks, and the graph structure data is used to represent the characteristics of the corresponding point cloud block; encoding the group of graph structure data to obtain a group of data packets, and sending the group of data packets to a target terminal for decoding and reconstruction at the target terminal to obtain target point cloud data.

[0007] According to another aspect of an embodiment of the present application, a point cloud data processing device is also provided, including: an acquisition module, used to acquire initial point cloud data to be sent; a division module, used to divide the initial point cloud data into a group of point cloud blocks; a generation module, used to generate a group of graph structure data based on a graphical representation of the point cloud data in each point cloud block in the group of point cloud blocks, wherein a graph structure data in the group of graph structure data corresponds to a point cloud block in the group of point cloud blocks, and the graph structure data is used to represent the characteristics of the corresponding point cloud block; a processing module, used to encode the group of graph structure data to obtain a group of data packets, and send the group of data packets to a target terminal for decoding and reconstruction at the target terminal to obtain target point cloud data.

[0008] In an exemplary embodiment, the device is used to generate a set of graph structure data by graphically representing the point cloud data in each point cloud block in the set of point cloud blocks in at least one of the following ways: performing a feature extraction operation on the target point cloud block to determine the target local features corresponding to the target point cloud block; calculating the topological relationship for at least two points in the target point cloud block to determine the target topological relationship corresponding to the target point cloud block; generating target graph structure data based on the target local features and the target topological relationship, wherein the node attributes of each node in the target graph structure data are determined by the target local features, and the connection relationship of each node in the target graph structure data is determined by the target topological relationship.

[0009] In an exemplary embodiment, the device is used to perform a feature extraction operation on the target point cloud block to determine the target local features corresponding to the target point cloud block in at least one of the following ways: performing a normal vector feature extraction operation on each point in the target point cloud block to determine the normal vector features corresponding to the target point cloud block, wherein the normal vector features are used to represent the direction of the surface where the corresponding points are located; performing a curvature feature extraction operation on each point in the target point cloud block to determine the curvature features corresponding to the target point cloud block, wherein the curvature features are used to represent the degree of curvature of the surface where the corresponding points are located; performing principal component analysis on the target point cloud block to obtain eigenvalues ​​and eigenvectors, wherein the eigenvalues ​​and the eigenvectors are used to represent the distribution direction and shape of the target point cloud block.

[0010] In an exemplary embodiment, the device is used to calculate the topological relationship of at least two points in the target point cloud block to determine the target topological relationship corresponding to the target point cloud block in at least one of the following ways: calculating the topological relationship of two points in the target point cloud block to determine whether the two points are adjacent; calculating the topological relationship of three points in the target point cloud block to determine whether the three points constitute a target triangle, wherein the three vertices of the target triangle are adjacent to each other; calculating the topological relationship of two target triangles in the target point cloud block to determine whether there is a shared edge, wherein the shared edge indicates that there are two common vertices between the two target triangles, and the three vertices of the target triangle are adjacent to each other.

[0011] In an exemplary embodiment, the device is used to generate target graph structure data based on the target local features and the target topological relationship in at least one of the following ways: generating a vertex graph based on the target local features and the target topological relationship, wherein the target graph structure data includes the vertex graph, a vertex of the vertex graph represents a point in the target point cloud block, and an edge of the vertex graph represents a connection relationship between two points in the target point cloud block to indicate the similarity between the two points; generating an edge graph based on the target local features and the target topological relationship, wherein the target graph structure data includes the edge graph, a node of the edge graph represents a pair of points in the target point cloud block whose similarity satisfies a first condition, and an edge of the edge graph represents that the similarity between two pairs of points in the target point cloud block satisfies a second condition; generating a hypergraph based on the target local features and the target topological relationship, wherein the target graph structure data includes the hypergraph, the hypergraph includes hyperedges and edges of the hypergraph, the hyperedges represent at least two points whose similarity of high-order features satisfies a third condition, and the edges of the hypergraph represent that the similarity between two of the hyperedges satisfies a fourth condition.

[0012] In an exemplary embodiment, the device is used to divide the initial point cloud data into a group of point cloud blocks in the following manner: determine an initial blocking method based on the distribution characteristics of the initial point cloud data; block the initial point cloud data according to the initial blocking method to obtain a group of initial point cloud blocks; determine an initial distribution quality parameter corresponding to the group of initial point cloud blocks, determine a target blocking method based on the initial distribution quality parameter, block the group of initial point cloud blocks according to the target blocking method to obtain a group of intermediate point cloud blocks; repeat the following steps until the intermediate distribution quality parameter meets a preset condition, and determine the group of intermediate point cloud blocks as the group of point cloud blocks, wherein the intermediate distribution quality parameter corresponds to the group of intermediate point cloud blocks: update the target blocking method based on the intermediate distribution quality parameter, re-block according to the updated target blocking method, and determine the updated intermediate distribution quality parameter.

[0013] In an exemplary embodiment, the device is also used for: after dividing the initial point cloud data into a group of point cloud blocks, merging the point cloud blocks based on the degree of similarity between the respective point cloud blocks in the group of point cloud blocks to obtain multi-level point cloud blocks including different levels, wherein the group of point cloud blocks includes the multi-level point cloud blocks; generating multi-level graph structure data based on a graphical representation of the point cloud data in each point cloud block in the multi-level point cloud blocks, wherein the group of graph structure data includes the multi-level graph structure data; encoding the multi-level graph structure data layer by layer to obtain a multi-level data packet; and sending the multi-level data packet to the target terminal layer by layer according to the network bandwidth and display capability of the target terminal, so as to be decoded and reconstructed at the target terminal to obtain the target point cloud data.

[0014] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned point cloud data processing method when running.

[0015] According to another aspect of the embodiment of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above point cloud data processing method.

[0016] According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the point cloud data processing method through the computer program.

[0017] In an embodiment of the present application, a method of obtaining initial point cloud data and dividing it into point cloud blocks is adopted. First, the initial point cloud data is divided into point cloud blocks that are easy to manage and transmit. Then, graph structure data is used to represent the characteristics of each point cloud block, and the point cloud block information is efficiently encoded in a graphical manner. Finally, a group of data packets obtained by encoding are decoded at the target terminal to reconstruct the target point cloud data. By generating and encoding graph structure data based on point cloud blocks, the purpose of effectively representing and transmitting point cloud data is achieved, the integrity and accuracy of point cloud data is guaranteed, and the transmission efficiency is improved, thereby achieving the technical effect of efficient transmission and reconstruction of point cloud data, and then solving the technical problem of low processing efficiency of point cloud data and inability to quickly achieve effects in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a schematic diagram of an application environment of an optional point cloud data processing method according to an embodiment of the present application;

[0020] Figure 2 is a flowchart of an optional method for processing point cloud data according to an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of an optional method for processing point cloud data according to an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of another optional method for processing point cloud data according to an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of another optional method for processing point cloud data according to an embodiment of the present application;

[0024] Figure 6 is a schematic structural diagram of an optional point cloud data processing device according to an embodiment of the present application;

[0025] Figure 7 is a schematic structural diagram of an optional point cloud data processing product according to an embodiment of the present application;

[0026] Figure 8 It is a schematic diagram of the structure of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretation:

[0030] Point cloud data is a data structure that represents a collection of points in three-dimensional space. Each point contains its coordinates in space (usually X, Y, Z coordinates), and sometimes also includes additional information such as color, intensity, normal, etc. Point cloud data can be used to describe the shape, surface features or geometric structure of an object.

[0031] The present application is described below in conjunction with embodiments:

[0032] According to one aspect of an embodiment of the present application, a method for processing point cloud data is provided. Optionally, in this embodiment, the method for processing point cloud data can be applied to Figure 1 In the hardware environment composed of the server 101 and the terminal device 103 shown in FIG. Figure 1As shown, the server 101 is connected to the terminal 103 via a network, and can be used to provide services for the terminal device or an application installed on the terminal device. The application can be a video application, an instant messaging application, a browser application, an educational application, a game application, etc. A database 105 may be set up on the server or independently of the server to provide data storage services for the server 101, for example, a game data storage server. The above-mentioned network may include, but is not limited to, a wired network and a wireless network, wherein the wired network includes, a local area network, a metropolitan area network and a wide area network; the wireless network includes, Bluetooth, WIFI and other networks that implement wireless communication; the terminal device 103 may be a terminal configured with an application, and may include, but is not limited to, at least one of the following: a mobile phone (such as an Android phone, an iOS phone, etc.), a laptop computer, a tablet computer, a PDA, a MID (Mobile Internet Devices), a PAD, a desktop computer, a smart TV, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, a virtual reality (VR) terminal, an augmented reality (AR) terminal, a mixed reality (MR) terminal and other computer devices; the above-mentioned server may be a single server, a server cluster consisting of multiple servers, or a cloud server.

[0033] Combination Figure 1 As shown, the above-mentioned point cloud data processing method can be executed by an electronic device, which can be a terminal device or a server. The above-mentioned point cloud data processing method can be implemented by the terminal device or the server respectively, or by the terminal device and the server together.

[0034] The above is only an example and is not specifically limited in this embodiment.

[0035] Optionally, as an optional implementation, as Figure 2 As shown, the method for processing the above point cloud data includes:

[0036] S202, obtaining initial point cloud data to be sent;

[0037] S024, dividing the initial point cloud data into a group of point cloud blocks;

[0038] S206, generating a set of graph structure data based on the point cloud data in each point cloud block in the set of point cloud blocks by graphically representing the point cloud data, wherein one graph structure data in the set of graph structure data corresponds to one point cloud block in the set of point cloud blocks, and the graph structure data is used to represent the features of the corresponding point cloud block;

[0039] S208, encode a group of graph structure data to obtain a group of data packets, and send the group of data packets to a target terminal for decoding and reconstruction at the target terminal to obtain target point cloud data.

[0040] Optionally, in an embodiment of the present application, the above-mentioned initial point cloud data refers to a set of original points obtained by three-dimensional scanning technology, including but not limited to point clouds obtained by building scanning, topographic mapping, and industrial parts measurement. The above-mentioned point cloud blocks refer to small blocks into which the initial point cloud data is divided, each block containing a certain number of points, including but not limited to point cloud blocks divided according to spatial position, point cloud density or specific shape features. The above-mentioned graph structure data refers to a graphical data structure generated based on the point cloud data in the point cloud block, including but not limited to adjacency matrices, edge lists, graph nodes, etc., which are used to describe the spatial relationship and characteristics between each point in the point cloud block. The above-mentioned data packet refers to a series of data units obtained after encoding the graph structure data, including but not limited to compressed binary data, serialized objects, etc., for network transmission or storage.

[0041] Exemplarily, the implementation process is as follows:

[0042] S1, in the field of architectural surveying and mapping, a laser scanner is used to scan a building to obtain initial point cloud data containing the three-dimensional coordinate information of the building.

[0043] S2, according to the structural characteristics of the building and the scanning range of the scanner, divides the point cloud data of the entire building into several small blocks, each of which corresponds to a part of the building, such as a floor or a wall.

[0044] S3, for each point cloud block, extract the spatial relationship and features of its internal point cloud and construct graph structure data. For example, a graph can be constructed in which nodes represent points in the point cloud, edges represent the spatial connection relationship between points, and the weight of the edge can represent the distance between points or other geometric properties.

[0045] S4, converting each graph structure data into a format suitable for network transmission, such as serializing the adjacency matrix or edge list of the graph into binary data. This step may include data compression to reduce the bandwidth required for transmission.

[0046] S5, packing the encoded graph structure data into data packets and sending them to a target terminal, such as a remote server or a mobile device, via a wireless network or a wired network.

[0047] S6, decoding and reconstruction are performed at the target terminal to obtain the target point cloud data. At the receiving end, the received data packet is decoded to restore the original graph structure data, and then the corresponding point cloud block is reconstructed according to the graph structure data. In this way, the complete building point cloud data can be reconstructed at the target terminal for subsequent analysis, display or other applications.

[0048] It should be noted that the methods and devices for obtaining the initial point cloud data to be sent can be diversified, and can be a laser scanner, a depth camera, or any other device capable of capturing three-dimensional spatial information, which is not limited in this application. When the initial point cloud data is divided into a group of point cloud blocks, the basis for the division can be spatial position, point cloud density, shape characteristics, or any other suitable standard, which is not limited in this application. When generating graph structure data based on point cloud blocks, the specific method of graphical representation can be to construct an adjacency matrix, an edge list, or any other graph representation method, which is not limited in this application. When encoding graph structure data, different encoding algorithms can be used, such as compression encoding, serialization encoding, or any other encoding technology, which is not limited in this application. The data packet sent to the target terminal can be any form of data transmission format, such as TCP / IP, UDP, or other protocols, which is not limited in this application. When the target terminal performs decoding and reconstruction, the decoding technology and algorithm corresponding to the encoding can be used to ensure the integrity and accuracy of the data, which is not limited in this application.

[0049] Through the above steps, efficient transmission and accurate reconstruction of point cloud data can be achieved, which is suitable for scenarios that require remote processing and analysis of three-dimensional spatial data, such as virtual reality, augmented reality, urban planning, and building information modeling.

[0050] In an exemplary embodiment, taking the application scenario of an autonomous vehicle as an example, the autonomous vehicle needs to process and analyze the three-dimensional data of the surrounding environment in real time to ensure safe driving. The laser radar (LiDAR) system carried by the autonomous vehicle continuously scans the surrounding environment and obtains initial point cloud data. These data include three-dimensional coordinate information of buildings, pedestrians, other vehicles, etc. around the vehicle. The system divides these initial point cloud data into multiple point cloud blocks, each of which represents the environment within a certain angle range around the car. Based on the point cloud data in each point cloud block, the system performs a graphical representation and generates graph structure data. For example, the system can construct a graph in which nodes represent points in the point cloud, edges represent the spatial connection relationship between points, and the weight of the edge can represent the distance or other geometric properties between points. The system encodes these graph structure data and converts each graph structure data into a format suitable for network transmission, such as serializing the adjacency matrix or edge list of the graph into binary data, and compressing the data to reduce the bandwidth required for transmission. The encoded data packet is sent to the central processing unit of the vehicle, and the central processing unit decodes the received data packet, restores the original graph structure data, and reconstructs the point cloud environment around the vehicle based on these data. The reconstructed point cloud data is used in the vehicle’s navigation and obstacle avoidance systems, ensuring that autonomous vehicles can safely navigate complex environments.

[0051] In another exemplary embodiment, taking the application scenario of an indoor navigation system as an example, the indoor navigation system needs to accurately construct and update the three-dimensional model of the indoor environment to provide accurate navigation services. Multiple depth sensors deployed in the indoor environment collect initial point cloud data, which include three-dimensional coordinate information of indoor walls, furniture, decorations, etc. The system divides these initial point cloud data into multiple point cloud blocks, each of which represents an area indoors, such as a room or a corridor. Based on the point cloud data in each point cloud block, the system performs a graphical representation and generates graph structure data. The system can construct a graph in which nodes represent points in the point cloud, edges represent the spatial connection relationship between points, and the weight of the edge can represent the distance or other geometric properties between points. The system encodes these graph structure data, converts each graph structure data into a format suitable for network transmission, and performs data compression to reduce the bandwidth required for transmission. The encoded data packet is sent to the central server, and the server decodes the received data packet, restores the original graph structure data, and reconstructs the point cloud model of the indoor environment based on these data. The reconstructed point cloud data is used in indoor navigation systems to provide users with accurate indoor positioning and navigation services, helping users find their destinations in complex indoor environments.

[0052] Through this embodiment, a method of obtaining initial point cloud data and dividing it into point cloud blocks is adopted. First, the initial point cloud data is divided into point cloud blocks that are easy to manage and transmit. Then, the characteristics of each point cloud block are represented by graph structure data, and the point cloud block information is efficiently encoded in a graphical manner. Finally, a group of data packets obtained by encoding are decoded at the target terminal to reconstruct the target point cloud data. Through the technical solution of generating graph structure data based on point cloud blocks and encoding, the purpose of effectively representing and transmitting point cloud data is achieved, the integrity and accuracy of point cloud data is guaranteed, and the transmission efficiency is improved at the same time, thereby realizing the technical effect of efficient transmission and reconstruction of point cloud data, and further solving the technical problem of low processing efficiency of point cloud data and inability to quickly achieve effect in related technologies.

[0053] As an optional solution, a set of graph structure data is generated based on a graphical representation of the point cloud data in each point cloud block in a set of point cloud blocks, including:

[0054] Perform feature extraction on the target point cloud block to determine the target local features corresponding to the target point cloud block;

[0055] Calculating a topological relationship for at least two points in the target point cloud block to determine a target topological relationship corresponding to the target point cloud block;

[0056] Target graph structure data is generated based on target local features and target topological relationships, wherein the node attributes of each node in the target graph structure data are determined by the target local features, and the connection relationships of each node in the target graph structure data are determined by the target topological relationships.

[0057] Optionally, in this embodiment, the target point cloud block can be understood as any point cloud block in the above group of point cloud blocks. The feature extraction operation refers to the operation of identifying and extracting from the target point cloud block the operation that helps to describe the characteristics of the point cloud block, including but not limited to edge detection, corner point detection, plane detection and other methods. The target local features refer to the features that describe the local geometric and physical properties of the target point cloud block, including but not limited to the normal, curvature, color, intensity and other properties of the point. The topological relationship refers to the spatial connection relationship between points in the target point cloud block, including but not limited to adjacency, inclusion, intersection and the like. The target topological relationship refers to the connection relationship between points in the point cloud block determined based on the principle of topology, including but not limited to the distance threshold between points, connection angle and other conditions. The target graph structure data refers to the graph data structure constructed according to the characteristics and topological relationship of the target point cloud block, including but not limited to adjacency matrix, edge list, graph node, etc., used to represent the geometric and topological information of the point cloud block. The node attributes refer to the attributes of each node in the target graph structure data, which are determined by the target local features, including but not limited to the geometric features, physical properties or statistical information of the node. The above connection relationship refers to the connection mode between each node in the target graph structure data, which is determined by the target topological relationship, including but not limited to the edges, weights or other parameters representing the connection strength between nodes.

[0058] For example, in the field of intelligent manufacturing, the application scenario of quality inspection of a mechanical part can be implemented as follows:

[0059] S1, perform feature extraction on the target point cloud block. First, use a high-precision 3D scanner to scan the mechanical part to obtain its point cloud data. Then, through feature extraction algorithms, such as convolutional neural networks based on deep learning, extract local features of the part surface, including but not limited to key quality indicators such as surface roughness, edge sharpness, and geometric shape.

[0060] S2, calculate the topological relationship of at least two points in the target point cloud block. Then, analyze the spatial position relationship of each point in the point cloud data, calculate the topological properties such as the distance and angle between them, and determine the connection relationship of the internal structure of the part, such as the connection between the hole and the body, the relative position between different features, etc.

[0061] S3, generate target graph structure data based on target local features and target topological relationships. Finally, construct graph structure data, where nodes represent points in the point cloud, and node attributes include local features obtained from the feature extraction step, such as surface roughness value, edge sharpness, etc. The connection relationship between nodes is determined based on the topological relationship, such as using edges to represent the spatial connection and structural relationship between points, and the weight of the edge can represent the distance between points or the strength of the connection.

[0062] Through the above steps, a graph structure data that accurately describes the geometric and topological characteristics of mechanical parts can be generated. This data can be used for subsequent quality inspection and analysis, such as further analyzing defects and deformation of parts through graph neural networks, thereby realizing automated quality control. This method not only improves the efficiency of inspection, but also enhances the accuracy of inspection, which is of great significance for improving the automation and intelligence level of the manufacturing industry.

[0063] As an optional solution, performing a feature extraction operation on the target point cloud block to determine the target local features corresponding to the target point cloud block includes at least one of the following:

[0064] Perform a normal vector feature extraction operation on each point in the target point cloud block to determine the normal vector feature corresponding to the target point cloud block, wherein the normal vector feature is used to represent the direction of the surface where the corresponding point is located;

[0065] Performing a curvature feature extraction operation on each point in the target point cloud block to determine the curvature feature corresponding to the target point cloud block, wherein the curvature feature is used to represent the curvature degree of the surface where the corresponding point is located;

[0066] The principal component analysis is performed on the target point cloud block to obtain the eigenvalues ​​and eigenvectors, wherein the eigenvalues ​​and eigenvectors are used to represent the distribution direction and shape of the target point cloud block.

[0067] Optionally, in an embodiment of the present application, the above-mentioned normal vector feature extraction operation refers to the process of calculating the surface normal of each point in the target point cloud block, including but not limited to methods such as local plane fitting or gradient calculation, which are used to represent the direction of the surface where the corresponding point is located. The above-mentioned curvature feature extraction operation refers to the process of calculating the surface curvature of each point in the target point cloud block, including but not limited to methods such as second-order derivatives or curvature estimation algorithms, which are used to represent the degree of curvature of the surface where the corresponding point is located. The above-mentioned principal component analysis refers to the process of performing statistical analysis on the target point cloud block to determine its main change direction and shape, and the obtained eigenvalues ​​and eigenvectors include but are not limited to those obtained by decomposing the covariance matrix, which are used to represent the distribution direction and shape of the target point cloud block.

[0068] For example, in the field of cultural relics protection and digitization, the application scenario of high-precision digital archiving of an ancient sculpture can be implemented as follows:

[0069] S1, perform normal vector feature extraction operation for each point in the target point cloud block. First, use high-precision 3D scanning equipment to fully scan the ancient sculpture and obtain its point cloud data. Then, calculate the normal vector of each point in the point cloud block, which can be achieved by solving the best fitting plane of the local neighborhood around each point. The normal vector feature reflects the direction of the surface where each point is located.

[0070] S2, performs a curvature feature extraction operation on each point in the target point cloud block. Next, the normal of each point and the relative position of the neighboring points are analyzed to calculate the curvature feature of each point. The curvature features can be determined by the principal curvature or high-order derivative method. These features reflect the curvature of the surface where each point is located, which is crucial for identifying the details and texture features of the sculpture.

[0071] S3, perform principal component analysis on the target point cloud block. Finally, perform principal component analysis (PCA) on the entire point cloud block, which is a statistical method used to determine the most significant direction of change in the data. Through PCA, eigenvalues ​​and eigenvectors describing the distribution direction and shape of the point cloud block can be obtained. This information is very important for understanding the overall geometric structure and local characteristics of the sculpture.

[0072] Through the above steps, a digital model containing rich geometric and physical features can be generated. This model can not only accurately reproduce the shape of the sculpture, but also preserve the microscopic features of its surface, which is of great value for the protection, research and educational display of cultural relics. In addition, these features can also be used for further analysis, such as crack detection, structural stability assessment, etc., to provide a scientific basis for cultural relic protection work.

[0073] It should be noted that feature extraction is an important step in the graphical representation of point clouds, which converts the geometric and attribute information of point clouds into the attributes of the graph structure. The specific implementation process is as follows:

[0074] S1, for each point in the block, extract its local features. Common features include: Normal vector: indicates the direction of the surface where the point is located. Curvature: indicates the degree of curvature of the surface where the point is located, including mean curvature, Gaussian curvature, etc. Eigenvalue: Perform principal component analysis (PCA) on the point cloud to obtain eigenvalues ​​and eigenvectors, which represent the main distribution direction and shape of the point cloud.

[0075] S2, for each pair of points or each group of points, calculate the topological relationship between them. Common relationships include: Adjacency relationship: whether two points are adjacent, that is, whether they are connected by an edge. Triangle relationship: whether the three points form a triangle, that is, whether they are adjacent to each other. Shared edge relationship: whether two triangles share an edge, that is, whether they have two common vertices.

[0076] S3, based on the extracted features and topological relationships, constructs a graphical representation of the point cloud, taking the features as the node attributes of the graph and the topological relationships as the edges of the graph.

[0077] S4, post-processes and optimizes the constructed graph, such as denoising, smoothing, simplification, etc., to improve the robustness and expressiveness of the graph.

[0078] S5, the advantage of feature extraction and graph construction is that it can comprehensively utilize the geometric and topological information of the point cloud to construct a more complete and effective graphical representation. However, the computational complexity of feature extraction and topological analysis is large, and it is necessary to balance the selection of features and computational efficiency.

[0079] It should be noted that the graphical representation of point clouds can be applied to many aspects, such as point cloud compression, feature analysis, semantic understanding, etc. The following are some specific application examples:

[0080] Point cloud compression: Using graphical representation, point clouds can be converted into more compact data structures, such as adjacency matrices and edge lists. Point cloud compression can be achieved by encoding and quantizing the graph structure.

[0081] Feature analysis: By analyzing the node attributes and edge weights in the graphical representation, various features of the point cloud can be extracted, such as local shape descriptors, global structural features, etc., which can be used for tasks such as point cloud classification, segmentation, and retrieval.

[0082] Semantic understanding: By introducing semantic attributes such as object categories and component labels in graphical representations, semantic segmentation and understanding of point clouds can be achieved, and points in the point cloud can be given a higher level of meaning for applications such as scene understanding and object detection.

[0083] Point cloud completion: By utilizing the topological relationships in the graphical representation, incomplete or missing point clouds can be repaired and completed. By using methods such as graph node interpolation and edge prediction, the positions and attributes of missing points can be estimated to improve the integrity of the point cloud.

[0084] Point cloud generation: By learning the graphical representation of point clouds, we can build generative models, such as graph convolutional neural networks, graph variational autoencoders, etc., starting from the graph structure, to generate realistic point cloud data, which can be applied to tasks such as point cloud synthesis, enhancement, and editing.

[0085] In other words, the application of graphical representation demonstrates the advantages of converting point clouds into graph structures. Through the analysis and processing of graphs, the features and semantics of point clouds can be better explored, and efficient compression, robust representation and intelligent analysis of point clouds can be achieved. However, the application of graphical representation is also limited by the complexity and computational efficiency of graphs, and it is necessary to balance the accuracy of representation and the computational overhead in specific tasks.

[0086] As an optional solution, calculating the topological relationship of at least two points in the target point cloud block and determining the target topological relationship corresponding to the target point cloud block includes at least one of the following:

[0087] Calculate the topological relationship between two points in the target point cloud block to determine whether the two points are adjacent;

[0088] Calculate the topological relationship of the three points in the target point cloud block to determine whether the three points form a target triangle, where the three vertices of the target triangle are adjacent to each other;

[0089] The topological relationship between the two target triangles in the target point cloud block is calculated to determine whether there is a shared edge, wherein the shared edge indicates that there are two common vertices between the two target triangles, and the three vertices of the target triangles are adjacent to each other.

[0090] Optionally, in an embodiment of the present application, the above-mentioned topological relationship refers to the spatial connection state between points in the point cloud data, including but not limited to the adjacency between points, the connection order, etc., which is used to determine the relative position relationship of points in the point cloud. Whether the above-mentioned two points are adjacent refers to determining whether the two points are directly connected in space by calculating parameters such as the distance or angle between the two points, including but not limited to the case of sharing a common face or edge. The above-mentioned target triangle refers to a geometric figure composed of three points in the point cloud data, which are not collinear and constitute a closed plane figure, including but not limited to a triangular face formed by three adjacent points. Whether the above-mentioned three points constitute a target triangle refers to determining whether they can form a triangle by checking the spatial position relationship of the three points, including but not limited to checking whether the three points meet the geometric conditions of the triangle, such as whether the three points are not collinear. The above-mentioned shared edge refers to the existence of one or more common edges between two target triangles, which are defined by two common vertices, including but not limited to the case where two adjacent triangles share a common edge.

[0091] For example, in the application scenarios of 3D modeling and building information modeling (BIM), the topological relationship calculation process of the target point cloud block can be implemented as follows:

[0092] S1, calculate the topological relationship between two points in the target point cloud block. First, obtain the point cloud data of the building structure through 3D scanning technology. Then, for any two points in the point cloud block, calculate the distance and angle between them to determine whether the two points are adjacent in space. This calculation can be judged by a preset threshold. For example, if the distance between two points is less than a certain value, they are considered to be adjacent.

[0093] S2, calculate the topological relationship of the three points in the target point cloud block. Next, select any three points from the point cloud data and check whether they form a target triangle. This can be achieved by verifying whether the three points are adjacent to each other and whether they meet the geometric conditions of the triangle (such as the three points are not collinear). If the three points meet the conditions for forming a triangle, then they are considered to form a target triangle.

[0094] S3, calculates the topological relationship between the two target triangles in the target point cloud block. Finally, for the two target triangles that have been determined, check whether there is a shared edge between them. This can be achieved by comparing the edges and vertices of the two triangles. If two triangles have two identical sides, they share an edge, which means that there are two common vertices between them.

[0095] It should be noted that the method for calculating whether two points are adjacent can be diversified, including judgment based on distance threshold, angle threshold or geometric feature, which is not limited in this application. The conditions for determining whether three points constitute a target triangle may include but are not limited to geometric properties such as the three points are not collinear and the sum of the interior angles of the triangle is 180 degrees, which is not limited in this application. The judgment of whether two target triangles have shared edges can be based on a variety of methods such as vertex matching of edges, consistency of vector directions of edges, or similarity of geometric lengths of edges, which is not limited in this application. In addition, the definition of shared edges may also vary according to different application scenarios and precision requirements. For example, it may be completely overlapping edges or approximately overlapping edges within a certain error range, which is not limited in this application.

[0096] Through the above steps, a topological diagram of the building structure can be constructed, which is crucial for subsequent building analysis, structural optimization and maintenance. This topological diagram can help architects and engineers understand how the building structure is connected, identify weaknesses in the structure, and make more reasonable decisions during the design and construction process. In addition, this topological diagram can also be used in virtual reality and augmented reality technology to provide users with a more realistic building interaction experience.

[0097] As an optional solution, generating target graph structure data based on target local features and target topological relationships includes at least one of the following:

[0098] Generate a vertex graph based on the local features of the target and the target topological relationship, wherein the target graph structure data includes the vertex graph, a vertex of the vertex graph represents a point in the target point cloud block, and an edge of the vertex graph represents a connection relationship between two points in the target point cloud block to indicate the similarity between the two points;

[0099] Generate an edge graph based on the local features of the target and the target topological relationship, wherein the target graph structure data includes the edge graph, a node of the edge graph represents a pair of points in the target point cloud block whose similarity satisfies a first condition, and an edge of the edge graph represents that the similarity between two pairs of points in the target point cloud block satisfies a second condition;

[0100] A hypergraph is generated based on target local features and target topological relationships, wherein the target graph structure data includes a hypergraph, the hypergraph includes a hyperedge and an edge of the hypergraph, the hyperedge represents at least two points whose high-order feature similarity satisfies a third condition, and the edge of the hypergraph represents that the similarity between two hyperedges satisfies a fourth condition.

[0101] Optionally, in an embodiment of the present application, the above-mentioned vertex graph refers to a graph structure data, in which a vertex represents a point in a target point cloud block, and an edge of the vertex graph represents a connection relationship between two points in the target point cloud block to indicate the similarity between the two points, including but not limited to a connection relationship determined by a distance metric, directional consistency or other feature similarity indicators. The above-mentioned edge graph refers to a graph structure data, in which a node represents a pair of points in the target point cloud block whose similarity satisfies a first condition, and an edge of the edge graph indicates that the similarity between two pairs of points in the target point cloud block satisfies a second condition, including but not limited to the similarity determined by comparing the feature differences or relative positional relationships between the two pairs of points. The above-mentioned hypergraph refers to a graph structure data, including hyperedges and edges of the hypergraph, the hyperedges representing at least two points whose similarity of high-order features satisfies a third condition, and the edges of the hypergraph representing that the similarity between two hyperedges satisfies a fourth condition, including but not limited to high-order feature similarity determined by cluster analysis, pattern recognition or machine learning methods.

[0102] For example, in the application scenarios of urban planning and traffic management, the process of generating target graph structure data based on target local features and target topological relationships is as follows:

[0103] S1, generate a vertex graph based on the local features of the target and the target topological relationship. First, obtain the point cloud data of the urban traffic network, including roads, intersections, traffic signs, etc., through high-precision 3D scanning technology. Then, construct a vertex graph, where each vertex represents a point in the point cloud, such as a traffic sign or an intersection. The edge of the vertex graph represents the connection relationship between two points, such as road connection, and the weight of the edge can indicate the similarity based on the distance between the two points or the traffic flow.

[0104] S2, generating an edge graph based on the local features of the target and the target topological relationship. Next, identifying point pairs whose similarity meets the first condition from the point cloud data, for example, pairing similar intersections according to traffic flow or geographical location. Constructing an edge graph, where each node represents such a pair of points, and the edges of the edge graph represent the similarity between these point pairs, such as the similarity of their traffic patterns or geographical layouts, which meets the second condition.

[0105] S3, a hypergraph is generated based on the target local features and the target topological relationship. Finally, point groups whose similarity of high-order features satisfies the third condition are identified, for example, multiple intersections are grouped according to the traffic flow pattern during peak traffic hours. A hypergraph is constructed, where hyperedges represent these point groups, and the edges of the hypergraph represent the similarity between different hyperedges, such as their functional similarity in the traffic network or their relative positions in the city, which satisfies the fourth condition.

[0106] Through the above steps, multi-level graph data reflecting the structure and characteristics of the urban traffic network can be created, which is of great significance for traffic flow prediction, urban planning and management. These graph data can help planners identify traffic bottlenecks, optimize traffic signal control, design more efficient traffic routes, and provide decision support for the sustainable development of urban transportation systems.

[0107] As an optional solution, the initial point cloud data is divided into a set of point cloud blocks, including:

[0108] Determine the initial block division method based on the distribution characteristics of the initial point cloud data;

[0109] Divide the initial point cloud data into blocks according to the initial block division method to obtain a set of initial point cloud blocks;

[0110] Determine an initial distribution quality parameter corresponding to a set of initial point cloud blocks, determine a target block division method based on the initial distribution quality parameter, divide the set of initial point cloud blocks according to the target block division method, and obtain a set of intermediate point cloud blocks;

[0111] Repeat the following steps until the intermediate distribution quality parameter meets the preset condition, and determine a group of intermediate point cloud blocks as a group of point cloud blocks, wherein the intermediate distribution quality parameter corresponds to a group of intermediate point cloud blocks:

[0112] The target block division method is updated based on the intermediate distribution quality parameter, and the blocks are re-divided according to the updated target block division method to determine the updated intermediate distribution quality parameter.

[0113] Optionally, in an embodiment of the present application, the above-mentioned initial point cloud data refers to the original point cloud information obtained by three-dimensional scanning or other data acquisition technology, including but not limited to the three-dimensional coordinate set of buildings, terrains, industrial parts, etc. The above-mentioned initial block method refers to the segmentation strategy adopted according to the spatial distribution characteristics of the initial point cloud data, including but not limited to block based on point cloud density, block based on point cloud area, or block based on specific shape features. The above-mentioned initial point cloud block refers to a subset of point cloud data obtained after processing according to the initial block method, including but not limited to a set of points with similar spatial positions or features. The above-mentioned initial distribution quality parameters refer to a set of parameters for evaluating the uniformity or rationality of the distribution of the initial point cloud block, including but not limited to the size, shape, density or uniformity of the distribution of the point cloud block. The above-mentioned target block method refers to a block strategy optimized based on the initial distribution quality parameters, including but not limited to adjusting the size and shape of the point cloud block or redefining the boundary of the point cloud block. The above-mentioned intermediate point cloud block refers to a new subset of point cloud data obtained after processing according to the target block method, including but not limited to the optimized point cloud block, whose distribution quality parameters are closer to the preset ideal state. The above-mentioned intermediate distribution quality parameters refer to parameters used to evaluate and adjust the block strategy during the block segmentation process, including but not limited to the uniformity, compactness or other quality indicators related to the target application of the intermediate point cloud blocks. The above-mentioned preset conditions refer to a set of criteria used to determine whether to stop iteration during the block segmentation process, including but not limited to a specific threshold reached by the intermediate distribution quality parameters or a parameter change below a certain value in consecutive iterations.

[0114] For example, in the application scenario of 3D modeling of urban buildings, block processing and optimization based on point cloud data can be achieved according to the following process:

[0115] S1, determine the initial block division method based on the distribution characteristics of the initial point cloud data. First, use the laser scanner carried by the drone to scan the urban buildings to obtain point cloud data containing the three-dimensional coordinates of the buildings. Analyze the distribution characteristics of these data, such as the density and uniformity of the point cloud and the structural characteristics of the building, and determine the appropriate initial block division method. For example, the building can be divided according to different floors or different facades.

[0116] S2, the initial point cloud data is divided into blocks according to the initial block division method to obtain a set of initial point cloud blocks. According to the determined block division method, the point cloud data of the entire city building is divided into a number of small blocks, each of which contains adjacent point cloud data, and these small blocks can represent different parts of the building, such as a single floor or a single facade.

[0117] S3, determine a set of initial distribution quality parameters corresponding to the initial point cloud blocks. Calculate the parameters of each initial point cloud block, such as the density of the point cloud in the block, the size and shape of the block, etc. These parameters will be used to evaluate the quality of the block and serve as the basis for subsequent block optimization.

[0118] S4, determining a target block division method based on the initial distribution quality parameters. The initial distribution quality parameters are analyzed to determine whether the block division method needs to be adjusted to optimize the quality of the point cloud blocks. For example, if the point cloud density of some blocks is found to be too high, further subdivision may be required; if some blocks are too sparse, merging may be required.

[0119] S5, a group of initial point cloud blocks are divided into blocks according to the target block division method to obtain a group of intermediate point cloud blocks. According to the adjusted target block division method, the initial point cloud blocks are further subdivided or merged to generate new intermediate point cloud blocks, which will better meet the needs of subsequent processing and modeling.

[0120] S6, repeat the following steps until the intermediate distribution quality parameters meet the preset conditions. The distribution quality parameters of the intermediate point cloud blocks are continuously monitored, and the target block method is updated according to these parameters, and then the blocks are re-blocked until the preset quality conditions are met, for example, the block size is uniform and the point cloud density is moderate, which can meet the optimization requirements of the 3D modeling software.

[0121] S7, determining the intermediate point cloud blocks that meet the conditions as the final point cloud blocks. Once the intermediate distribution quality parameters meet the preset conditions, these intermediate point cloud blocks will be determined as the final point cloud blocks for subsequent 3D modeling and analysis.

[0122] Through the above process, it is possible to achieve effective block processing of large-scale urban building point cloud data, optimize the data processing process, improve modeling efficiency and quality, and reduce the consumption of computing resources.

[0123] As an optional solution, after dividing the initial point cloud data into a group of point cloud blocks, the method further includes:

[0124] Merging the point cloud blocks in a group of point cloud blocks based on the similarity between the point cloud blocks to obtain multi-level point cloud blocks including different levels, wherein the group of point cloud blocks includes multi-level point cloud blocks;

[0125] Based on the graphical representation of the point cloud data in each point cloud block in the multi-level point cloud block, a multi-level graph structure data is generated, wherein a set of graph structure data includes the multi-level graph structure data;

[0126] Encode the multi-level graph structure data layer by layer to obtain a multi-level data packet;

[0127] According to the network bandwidth and display capability of the target terminal, multi-layer data packets are sent to the target terminal layer by layer, so as to be decoded and reconstructed at the target terminal to obtain the target point cloud data.

[0128] Optionally, in an embodiment of the present application, the multi-level point cloud block refers to a set of point cloud blocks with different levels obtained after the initial point cloud data is divided and merged, including but not limited to point cloud blocks hierarchically organized according to the spatial proximity, geometric similarity or topological relationship of the point cloud data. The multi-level graph structure data refers to a data structure generated after a graphical representation of each point cloud block in the multi-level point cloud block, including but not limited to vertex graphs, edge graphs or hypergraphs, etc. These graph structure data describe the characteristics and mutual relationships of the point cloud blocks at different levels. The multi-level data packet refers to a series of data packets obtained after encoding the multi-level graph structure data layer by layer, including but not limited to data units arranged in the hierarchical order of the graph structure data, which can be used for network transmission and decoding and reconstruction at the target terminal. The target terminal refers to a device that receives and processes multi-level data packets, including but not limited to personal computers, mobile devices, virtual reality helmets or other intelligent terminals with graphics processing and display capabilities. These terminals receive and render point cloud data according to their network bandwidth and display capabilities to provide user interaction and visual experience.

[0129] For example, in the application scenario of digital protection of cultural heritage, the processing and transmission of point cloud data can be achieved according to the following process:

[0130] S1 divides the initial point cloud data into a set of point cloud blocks. First, a high-precision laser scanner is used to fully scan the cultural heritage to obtain its 3D point cloud data. Then, these data are divided into multiple small blocks based on spatial position and geometric features, each of which represents a part of the cultural heritage, such as a part of a sculpture or a corner of a building.

[0131] S2, based on the similarity between each point cloud block in a group of point cloud blocks, merges them to obtain multi-level point cloud blocks including different levels. By analyzing the shape, texture and spatial relationship of the point cloud blocks, similar or adjacent blocks are merged into larger blocks to form a multi-level structure, where each level represents a different level of detail and abstraction.

[0132] S3, based on the graphical representation of the point cloud data in each point cloud block in the multi-level point cloud block, generates multi-level graph structure data. A graphical model, such as a vertex graph or an edge graph, is constructed for each point cloud block to convert the point cloud data into graph structure data, where each node represents a point and the edge represents the spatial relationship between the points.

[0133] S4, encodes the multi-level graph structure data layer by layer to obtain a multi-level data packet. Using graph coding technology, such as graph Fourier transform or graph wavelet transform, the graph structure data of each layer is compressed and encoded to generate a series of data packets, each of which contains a level of graph structure information.

[0134] S5, sends multi-level data packets to the target terminal layer by layer according to the network bandwidth and display capability of the target terminal. According to the user's network conditions and device performance, the transmission data packet layer is dynamically selected, and the low-level rough model is transmitted first. As the network conditions improve, the higher-level detailed models are gradually transmitted.

[0135] S6, decoding and reconstruction are performed at the target terminal to obtain the target point cloud data. The user device decodes the received data packet and reconstructs a multi-level point cloud model. The user can choose to view different levels of details as needed, from rough outlines to fine textures.

[0136] Through the above process, cultural heritage point cloud data can be effectively organized, compressed and transmitted, allowing users to view three-dimensional models at different levels of detail on different devices according to network conditions, providing an efficient solution for the digital protection and research of cultural heritage.

[0137] The following is a further explanation of this application with reference to specific examples:

[0138] This application maps point cloud data into a graph data structure, uses nodes to represent points, and uses edges to represent the topological relationship between points, making full use of the characteristics of the graph to characterize the intrinsic structure and constraints of the point cloud. On the basis of graphical representation, the point cloud is further divided into multiple small blocks, which are processed and transmitted separately to improve the processing efficiency and parallelism of large-scale point cloud data. At the same time, through graph pyramid, graph wavelet and other technologies, a multi-level and multi-resolution representation of the point cloud is constructed to adapt to different network environments and display devices, and support progressive loading and interactive exploration of point clouds. During the transmission process, the content and order of transmission are dynamically adjusted according to network bandwidth and user behavior, and important and interesting point cloud areas are transmitted first, providing a real-time and adaptive point cloud browsing experience.

[0139] The point cloud data is divided into multiple blocks, and the features of each block are represented by a graphical method, so that efficient point cloud data transmission can be achieved through the transmission graph. In an exemplary embodiment, Figure 3 is a schematic diagram of an optional method for processing point cloud data according to an embodiment of the present application, and the implementation process is as follows: Figure 3 As shown:

[0140] S302, the large-scale point cloud data is divided into multiple small blocks according to the spatial position, each block contains a part of the point cloud data, and the block division method can adopt a spatial index structure such as an octree, a kd tree, or a uniform grid, a surface fitting method, etc.:

[0141] Octree: Recursively divide the point cloud space into eight subspaces, each subspace as a block.

[0142] kd Tree: Recursively splits based on the coordinate values ​​of the points, selecting one coordinate axis for division each time.

[0143] Uniform Grid: Divide the point cloud space into grids of equal size, with each grid as a block.

[0144] Surface fitting: Fit and segment the point cloud according to its geometric features, dividing points with similar features into a block.

[0145] Block size: It is necessary to find a balance between the size and number of data blocks to avoid excessive changes in the point cloud within a block or too many blocks affecting processing efficiency.

[0146] S304, graphical representation within the block:

[0147] Vertex Graph: Each point is a vertex of the graph, and the edges represent the connection relationship between points, such as distance or proximity.

[0148] Edge Graph: Each edge is a node in the graph, representing the distance or attribute difference between two points. The weight of the edge can represent the geometric or attribute relationship between the points.

[0149] Hypergraph: allows one edge to connect multiple vertices, representing higher-order relationships between multiple points, such as coplanar relationships or feature collinearity.

[0150] Feature extraction: Describe the main features of the point cloud through geometric features (such as normal vectors, curvature) and topological features (such as adjacency relationships).

[0151] S306, Graph Coding and Transmission:

[0152] Encoding method:

[0153] Graph Fourier Transform: Perform frequency domain analysis on graph signals and encode the graph spectrum coefficients as features.

[0154] Graph Wavelet Transform: Perform wavelet transform on the graph to extract multi-scale features for encoding.

[0155] Adjacency matrix compression: compress the adjacency matrix of a graph to reduce the amount of data stored and transmitted.

[0156] Attribute quantization: Quantize the attributes of nodes and edges, reducing data precision in exchange for compression rate.

[0157] Transmission process: The structural information of the graph (such as vertex list and edge list) is transmitted first, and then the attribute information of the node is transmitted to achieve efficient transmission.

[0158] S308, image decoding and point cloud reconstruction:

[0159] Decoding process: The receiving end reconstructs the graph structure and node attributes based on the transmitted graph data.

[0160] Point cloud reconstruction: Use the decoded image data to reconstruct the point cloud data of each block, and obtain a complete point cloud model by splicing the data of each block.

[0161] Deep learning technology: Deep learning methods such as graph convolutional neural networks (GCNs) can be used to improve the quality and efficiency of point cloud reconstruction.

[0162] S310, Progressive Transmission and Update:

[0163] Progressive loading: First transmit the low-level representation of the graph, and gradually transmit high-level details to achieve progressive loading of the point cloud.

[0164] Local update: When a point cloud undergoes local changes, only the graph representation of the corresponding block needs to be transmitted, reducing the amount of data transmission.

[0165] Detail control: According to user needs and network conditions, select different levels of point cloud representation to achieve multi-level detail control.

[0166] Advantages and application prospects: This method makes full use of graph data structures and signal processing technology to efficiently compress and transmit large-scale point cloud data. Compared with directly transmitting the original point cloud coordinates and attributes, this method significantly reduces the amount of data while retaining the main features of the point cloud. Through block division and progressive transmission, it also supports hierarchical loading and local updates of point clouds.

[0167] By designing efficient graphical representation and encoding schemes, we can balance compression rate, accuracy, computational complexity, etc. At present, some studies have explored different graphical representation and encoding methods, such as octree-based graph representation and encoding based on spectral graph theory. With the development of technologies such as graph neural networks, graphical point cloud transmission methods are expected to be further improved and applied.

[0168] Among them, the implementation process of point cloud segmentation is as follows:

[0169] Octree partitioning: Octree partitioning is an efficient method based on recursive partitioning of space, which is particularly suitable for the partitioning and management of 3D point cloud data. This method starts with the bounding box of the entire point cloud and recursively divides it into eight subspaces. Each subspace is called an octree node, and the node is divided according to the spatial position of the point. The specific process is as follows:

[0170] S1, initial partitioning: First determine the bounding box of the point cloud data, that is, the smallest cube that encloses the point cloud. Divide the cube into eight sub-cubes of equal size, each of which represents an octree node.

[0171] S2, recursive segmentation: divide the point cloud data in each sub-cube into eight equal parts again, and continue recursive segmentation until the stopping condition is met. The stopping condition can be that the number of points in the sub-cube is lower than a certain threshold, or the size of the sub-cube is smaller than a certain minimum value.

[0172] S3, tree structure construction: During the recursive segmentation process, the tree structure of the octree is constructed. Each node records the pointer of its child node and the point cloud data it contains. The leaf node represents the final block.

[0173] S4, Data Storage and Retrieval: The octree partitioning method is not only used for data segmentation, but also for efficient data storage and retrieval. Through the octree structure, point cloud data in a specific spatial area can be quickly located and accessed, supporting efficient neighborhood queries and collision detection.

[0174] The advantages of octree segmentation are its adaptability and efficiency. It can dynamically adjust the segmentation depth according to the distribution of point cloud data, avoiding the data imbalance problem that may be caused by uniform segmentation. However, building and maintaining the octree structure requires high computational and storage overhead, especially for very dense or sparse point cloud data.

[0175] KD Tree Blocking: KD Tree is a partitioning structure used for multi-dimensional spatial indexing, especially suitable for point cloud data partitioning based on coordinates. The partitioning process of KD Tree is based on the coordinate values ​​of the points and recursively partitions them, selecting a coordinate axis for partitioning each time. Figure 7 is a schematic diagram of another optional method for processing point cloud data according to an embodiment of the present application;

[0176] The specific process is as follows:

[0177] S1, initial construction: select a coordinate axis (such as the x-axis) of the point cloud data, and divide the point cloud into two parts according to the median value of the axis to form two child nodes of the root node. The selection of the median value can be achieved by quick sorting or median selection algorithm.

[0178] S2, recursive segmentation: For the point cloud data in each child node, select the next coordinate axis (such as the y axis) to perform the same segmentation. This process is performed recursively, and the selected coordinate axis is rotated in a fixed order each time (such as x, y, z in turn).

[0179] S3, tree structure construction: During the recursive partitioning process, the tree structure of the kd tree is constructed. Each node records its partitioning coordinate axis and partitioning value, as well as the pointer of its child node.

[0180] S4, Data storage and retrieval: The kd-tree partitioning method performs well in data storage and retrieval. Through the kd-tree structure, range queries and nearest neighbor searches can be performed efficiently, which is suitable for various geometric query tasks.

[0181] The advantage of kd-tree partitioning is its simplicity and efficiency, which can be quickly constructed and queried. Compared with octree, kd-tree is suitable for a wider range of data distribution situations, but it may face the dimensionality disaster problem when processing high-dimensional data, which needs to be improved through optimization algorithms.

[0182] Uniform Grid Blocking: Uniform grid blocking is a simple and effective method based on fixed-size grids, which is suitable for regular segmentation of point cloud data. This method divides the point cloud space into grids of equal size, and each grid is a data block. The specific process is as follows:

[0183] S1, initial division: determine the bounding box of the point cloud data and select the size of the grid (such as 1m x 1m x 1m). According to the bounding box and grid size, divide the entire point cloud space into multiple grid cells of equal size.

[0184] S2, point cloud allocation: traverse the point cloud data and assign each point to the corresponding grid cell according to its spatial coordinates. Each grid cell contains all the points within its range.

[0185] S3, grid index construction: Create an index for each grid cell to record the point cloud data it contains. The index structure can be stored in a hash table or direct array for fast access.

[0186] S4, Data storage and retrieval: The uniform grid structure is simple and suitable for fast point cloud data storage and retrieval. By calculating the spatial coordinates of the point, it can be directly located to the grid cell to which it belongs, and perform efficient neighborhood query and collision detection.

[0187] The advantages of the uniform grid partitioning method are that it is simple to implement and has low computational overhead, and is suitable for situations where point cloud data is relatively evenly distributed. However, for point clouds with highly uneven data distribution, uniform grids may cause some grid cells to be too dense or sparse, affecting processing efficiency and results.

[0188] Surface fitting block: The surface fitting block method divides points with similar features into blocks by analyzing the geometric features of point cloud data. This method is suitable for point cloud data with complex geometric shapes and can extract and maintain geometric features. The specific process is as follows:

[0189] S1, initial preprocessing: preprocess the point cloud data, including denoising, downsampling and other operations to ensure the quality and uniformity of the point cloud data.

[0190] S2, geometric feature extraction: Use geometric analysis algorithms (such as normal vector estimation and curvature calculation) to extract the geometric features of point cloud data. According to the feature values ​​(such as normal vector direction and curvature size), the points are classified.

[0191] S3, surface fitting: Perform surface fitting on each classified point cloud data, using the least square method or other fitting algorithms to obtain a fitting surface. The fitting surface can be a plane, a quadratic surface, or other high-order surfaces.

[0192] S4, block division: According to the fitting results, the point cloud data is divided into multiple blocks, each block corresponds to a fitting surface. The transition area between adjacent surfaces can be processed by interpolation or other smoothing methods.

[0193] S5, data storage and retrieval: Build a block index and record the point cloud data and fitted surface parameters of each block. The block-based point cloud data can be efficiently stored and retrieved, and is suitable for various geometric processing and analysis tasks.

[0194] The advantage of the surface fitting and blocking method is that it can maintain the geometric characteristics of point cloud data and is suitable for point cloud data with complex shapes. However, the fitting and blocking process requires high computational and storage overhead, especially when processing large-scale point cloud data, which requires parallel computing and optimization algorithms to accelerate.

[0195] Adaptive Partitioning: Adaptive partitioning is a point cloud partitioning method that combines multiple partitioning methods and heuristic strategies. It dynamically adjusts the partitioning strategy according to the distribution characteristics of the point cloud and application requirements. Figure 4 is a schematic diagram of another optional method for processing point cloud data according to an embodiment of the present application, such as Figure 4 As shown, the specific process is as follows:

[0196] S1, analyze the distribution characteristics of point cloud data, such as density change, curvature change, etc., and determine the initial method and parameters for segmentation.

[0197] S2, use the selected block method (such as octree, kd tree, uniform grid, etc.) to perform initial block division on the point cloud.

[0198] S3, evaluates the results of the initial block division and calculates indicators such as the distribution of points in each block and density change.

[0199] S4, dynamically adjusts the block division method and parameters according to the evaluation results, adopts a finer-grained block division method for blocks that need further division, and adopts a coarser-grained block division method for blocks with uniform point distribution.

[0200] S5, iteratively execute steps S3 and S4 until the global block conditions are met, such as the number of blocks, the difference in the number of points within a block, etc.

[0201] S6, stores the final blocks and builds a multi-level block structure to support the transmission and presentation of point clouds of different granularities.

[0202] The adaptive segmentation method combines the advantages of multiple segmentation methods, can dynamically adjust the segmentation strategy according to the point cloud characteristics and application requirements, and has good performance in segmentation efficiency and adaptability. However, the implementation complexity of this method is relatively high, and it is necessary to design reasonable evaluation indicators and adjustment strategies to balance the computational cost and segmentation quality.

[0203] Balance of block size: In the process of point cloud segmentation, the selection of block size needs to find a balance between the size and number of data blocks. Blocks that are too large will cause the point cloud within the block to change too much, increasing the complexity of data processing; blocks that are too small will lead to too many blocks, increasing the overhead of segmentation and management. The specific process is as follows:

[0204] Initial parameter setting: Set the initial block size according to the scale of point cloud data and application requirements. It can be preliminarily determined by empirical values ​​or preset parameters.

[0205] Change analysis within a block: After the initial block division, the point cloud data is analyzed for changes in each block. The uniformity of the point cloud within the block can be evaluated by calculating the range of changes in the geometric features (such as normal vectors and curvature) of the points within the block.

[0206] Balance adjustment: According to the changes within the block, adjust the block size. For blocks with large changes, you can further subdivide them; for blocks with small changes, you can merge adjacent blocks. Through iterative adjustment, you can achieve a balanced state with uniform changes within the block and a reasonable number of blocks.

[0207] Application of optimization algorithms: Optimization algorithms (such as dynamic programming and genetic algorithms) can be used to optimize the block size and find the best block strategy. The optimization algorithm can perform global search in multiple parameter dimensions to improve the block effect.

[0208] Balance evaluation and verification: Evaluate the balance effect of the chunk size through experiments and practical applications. Verify the rationality of the chunk strategy based on indicators such as processing efficiency, storage overhead, and data quality, and make adjustments and improvements based on feedback.

[0209] The balance of block size is a key link in the point cloud segmentation method, which directly affects the efficiency and effect of subsequent processing. Through a scientific and reasonable segmentation strategy, the processing performance and application effect of point cloud data can be effectively improved.

[0210] Detailed implementation process of adaptive blocking:

[0211] Analysis of distribution characteristics of point cloud data: In the first step of adaptive blocking, a comprehensive distribution characteristic analysis of the input point cloud data is required. Calculate the global statistical information of the point cloud, such as the number of points, density, bounding box, etc., to obtain the overall distribution profile of the point cloud. Use the moving window or local sampling method to calculate the local features of the point cloud in different areas, such as density change gradient, normal vector change gradient, curvature, etc. These local features reflect the spatial non-uniformity and geometric detail changes of the point cloud, and are an important basis for adaptive blocking. At the same time, according to application requirements, the semantic features of the point cloud, such as object categories, key points, etc., can also be calculated to provide higher-level information for subsequent blocking decisions. By comprehensively analyzing the global statistical information, local distribution characteristics and semantic features of the point cloud, the blocking method and parameters can be preliminarily determined, such as the maximum depth of the octree, the segmentation dimension of the kd tree, and the grid size of the uniform grid.

[0212] Initial block division and evaluation: According to the block division method and parameters determined in the first step, the point cloud data is initially divided into blocks. Common block division methods include octree, kd tree, uniform grid, etc. These methods have different strategies and optimization goals in space division. For example, octree uses recursive space division, dividing the space into eight sub-regions each time, which is suitable for the situation where the point cloud distribution is relatively uniform; kd tree uses recursive division based on coordinate axis, selecting a coordinate axis for binary division each time, which is suitable for the situation where the point cloud distribution has obvious directionality; uniform grid uses fixed-size grid division, which is simple and efficient, but has poor adaptability to non-uniform distribution. The result of the initial block division is a set of point cloud sub-blocks, each of which contains a part of the point cloud data. In order to evaluate the quality of the initial block division, it is necessary to calculate the local features of each sub-block, such as the number of points, density, variance, etc., as well as the similarity measures between sub-blocks, such as adjacency, distribution difference, etc. These evaluation indicators reflect the effect of the initial block division in capturing the local features of the point cloud and maintaining the consistency between blocks.

[0213] Adaptive block parameter adjustment: According to the evaluation results of the initial block, the block method and parameters are adaptively adjusted to further optimize the block quality. The specific adjustment strategy varies according to different evaluation indicators, and usually follows the following principles: For areas with large changes in point cloud density and complex local features, a finer-grained block method is used, such as increasing the division depth of the octree and reducing the grid size of the uniform grid, to better capture local geometric details; for areas with uniform point cloud distribution and simple local features, a coarser-grained block method is used, such as reducing the number of kd tree divisions and increasing the grid size of the uniform grid, to reduce unnecessary computational overhead. Adaptive block parameter adjustment usually adopts a heuristic strategy to set the adjustment threshold and step size based on experience. For example, when the point density of a sub-block exceeds the threshold, it is further divided; when the distribution difference of adjacent sub-blocks is less than the threshold, they are merged into a larger block. By iteratively evaluating and adjusting the block parameters, the block results can be continuously optimized until the preset global conditions, such as the number of blocks and the difference in block size, are met.

[0214] Multi-level block structure construction: After the adaptive block parameter adjustment, a set of optimized point cloud sub-blocks are obtained. In order to support the transmission and presentation of point clouds of different granularities, it is necessary to build a multi-level block structure based on these sub-blocks. A commonly used method is octree layering, that is, the sub-blocks obtained by adaptive block division are used as leaf nodes of the octree, and then these leaf nodes are merged from bottom to top to generate higher-level blocks. The merging strategy can be based on indicators such as block size, density, and similarity. For example, adjacent blocks of similar size and density are merged into a parent block. By recursively merging blocks, a hierarchical octree structure can be obtained, in which each node corresponds to a point cloud block, and the high-level blocks contain the information of the low-level blocks. Another multi-level block method is kd tree layering, that is, based on the kd tree obtained by adaptive block division, kd trees of different resolutions are generated by downsampling or merging subtrees, and each tree corresponds to a block level. The multi-level block structure provides the basis for hierarchical transmission and progressive rendering of point clouds. Users can dynamically select blocks of different levels for transmission and presentation according to the network bandwidth and the capabilities of the display device.

[0215] Dynamic update of adaptive blocking: In practical applications, point cloud data may change dynamically, such as the movement and deformation of objects, and the adaptive blocking results need to be dynamically updated. A commonly used update strategy is based on incremental update of blocks, that is, only the changed blocks and their neighborhoods are re-blocked, while other blocks remain unchanged. In specific implementation, the method of spatiotemporal coherence detection can be used to compare the differences between the two frames of point clouds to determine the blocks that need to be updated. For the blocks that need to be updated, the adaptive blocking process is re-executed, including local feature analysis, initial blocking, parameter adjustment and other steps. The updated blocks replace the original blocks to form a new blocking structure. Another update strategy is based on global reconstruction, that is, when the point cloud changes greatly or the cumulative change exceeds the threshold, the adaptive blocking is re-executed on the entire point cloud to generate a new multi-level blocking structure. Although global reconstruction has a large computational overhead, it can ensure the global optimality of the blocking results. In practical applications, the appropriate update strategy can be selected according to the frequency and amplitude of point cloud changes to balance computational efficiency and blocking quality.

[0216] Adaptive blocking is a dynamic point cloud blocking method that integrates multiple blocking methods and strategies. It analyzes the global statistical information, local distribution characteristics and semantic features of the point cloud, adaptively adjusts the blocking parameters, generates a multi-level blocking structure, and supports dynamic updates. This method can optimize the blocking effect and improve the efficiency of transmission and rendering according to the distribution characteristics and application requirements of the point cloud. However, the implementation of adaptive blocking is relatively complex, and it is necessary to design reasonable evaluation indicators, adjustment strategies and update mechanisms to balance the computational cost and blocking quality. In practical applications, appropriate adaptive blocking methods and parameters can be selected according to specific point cloud data and performance requirements to obtain the optimal blocking results.

[0217] In-block graphical representation: Its purpose is to convert the point cloud data in each block into a graph, and describe the geometric and topological relationships between points through the graph structure, so as to achieve point cloud compression and feature extraction. The following are the detailed implementation processes of several commonly used in-block graphical representation methods:

[0218] Vertex Graph construction: Vertex graph is an intuitive graphical representation method of point cloud, where each point is a vertex of the graph and the edges represent the connection relationship between the points.

[0219] The specific implementation process is as follows: For each point in the block, create a corresponding graph vertex, the attributes of the vertex include the coordinates, color, normal vector, etc. of the point. Common methods for calculating the distance or proximity between points include:

[0220] k-nearest neighbor (k-NN): For each point, find its k nearest neighbors.

[0221] Radius Search: For each point, find all points whose distance is less than a given radius r as neighbors.

[0222] Delaunay triangulation: The point cloud is divided into a set of triangles through the Delaunay triangulation algorithm, and edges are established between the vertices of the triangles.

[0223] Based on the calculated proximity relationships, graph edges are established between corresponding vertices, and the weight of the edge can represent the distance or similarity between points.

[0224] The established vertex graph is simplified and optimized, redundant edges are removed, and similar vertices are merged to reduce the complexity of the graph.

[0225] The advantages of vertex graphs are that they are simple to construct, intuitive and easy to understand, and suitable for representing the local structure of point clouds. However, for large-scale point clouds, the vertex graph is more complex and the number of edges may be large, which is not conducive to compression and transmission.

[0226] Edge Graph Construction: Edge graph is another graphical representation method of point cloud, where each edge is a node of the graph, representing the distance or attribute difference between two points. The specific implementation process is as follows:

[0227] For each pair of points in the block, the distance or attribute difference between them is calculated. Commonly used measurement methods include:

[0228] Euclidean distance: the straight-line distance between the coordinates of two points.

[0229] Surface distance: The geodesic distance between two points on a surface.

[0230] Normal vector difference: The angle or difference between the normal vectors of two points.

[0231] Color difference: The difference or similarity between the color values ​​of two points.

[0232] According to the calculated distance or difference value, select the point pairs that meet the conditions and generate the corresponding graph nodes. The attributes of the nodes include the index, distance, difference value, etc. of the two points.

[0233] For each graph node, calculate the similarity or compatibility between it and other nodes. Common methods include: distance threshold: if the distance between the point pairs corresponding to two nodes is less than the given threshold, the nodes are considered similar. Attribute consistency: if the point pairs corresponding to two nodes meet the consistency conditions in attributes such as normal vectors and colors, the nodes are considered similar. According to the calculated similarity, the edges of the graph are established between the corresponding nodes, and the weights of the edges represent the degree of similarity between the nodes. The established edge graph is simplified and optimized, similar nodes are merged, and redundant edges are removed to improve the expressive power of the graph. The advantage of the edge graph is that it can capture the overall structure and attribute changes of the point cloud, which is particularly suitable for representing point clouds with obvious geometric or attribute features. However, the construction of the edge graph is relatively complex, and it is necessary to calculate the distances and differences between all point pairs. For large-scale point clouds, the computational overhead is very high.

[0234] Hypergraph construction: Hypergraph is a more general point cloud graphical representation method, which allows one edge to connect multiple vertices to represent the high-order relationship between multiple points. The specific implementation process is as follows: For the point cloud in the block, extract its high-order features. Commonly used features include:

[0235] Coplanarity: Multiple points are coplanar, that is, they are in the same plane.

[0236] Collinearity: Multiple points are collinear, that is, they are on the same straight line.

[0237] Feature similarity: multiple points have similar local features, such as normal vectors, curvature, etc.

[0238] According to the extracted high-order features, multiple points that meet the conditions are combined into a hyperedge. The attributes of the hyperedge include the index and eigenvalue of the connected points.

[0239] For each hyperedge, calculate its similarity or compatibility with other hyperedges. Common methods include: Jaccard similarity: the ratio of the intersection size to the union size of the point sets connected by two hyperedges. Feature correlation: the correlation between the features represented by two hyperedges, such as the normal vector angle, curvature difference, etc.

[0240] According to the calculated similarity, hypergraph edges are established between corresponding hyperedges, and the edge weights represent the similarity between the hyperedges.

[0241] The established hypergraph is simplified and optimized, similar hyperedges are merged, and redundant edges are removed to improve the expressive power of the hypergraph.

[0242] The advantage of hypergraphs is that they can represent high-order relationships and complex structures in point clouds, and are particularly suitable for describing point clouds with obvious geometric or semantic features. However, the construction of hypergraphs is very complex, requiring the extraction and analysis of high-order features of point clouds, which results in high computational overhead.

[0243] Feature extraction and graph construction: In the graphical representation of point clouds, feature extraction is an important step, which converts the geometric and attribute information of point clouds into the attributes of the graph structure. The specific implementation process is as follows:

[0244] For each point in the block, extract its local features. Common features include:

[0245] Normal vector: represents the direction of the surface where the point is located.

[0246] Curvature: Indicates the degree of curvature of the surface where the point is located, including mean curvature, Gaussian curvature, etc.

[0247] Eigenvalue: Perform principal component analysis (PCA) on the point cloud to obtain eigenvalues ​​and eigenvectors, which represent the main distribution direction and shape of the point cloud.

[0248] For each pair of points or each point group, calculate the topological relationship between them. Commonly used relationships include:

[0249] Adjacency relationship: whether two points are adjacent, that is, whether there is an edge connecting them.

[0250] Triangular relationship: whether three points form a triangle, that is, whether they are adjacent to each other.

[0251] Shared edge relationship: whether two triangles share an edge, that is, whether they have two common vertices.

[0252] Based on the extracted features and topological relationships, a graphical representation of the point cloud is constructed, with the features as node attributes of the graph and the topological relationships as edges of the graph.

[0253] The constructed graph is post-processed and optimized, such as denoising, smoothing, simplification, etc., to improve the robustness and expressiveness of the graph.

[0254] The advantage of feature extraction and graph construction is that it can comprehensively utilize the geometric and topological information of point clouds to construct a more complete and effective graphical representation. However, the computational complexity of feature extraction and topological analysis is large, and it is necessary to balance the selection of features and computational efficiency.

[0255] Application of graphical representation: The graphical representation of point clouds can be applied in many aspects, such as point cloud compression, feature analysis, semantic understanding, etc. The following are some specific application examples:

[0256] Point cloud compression: Using graphical representation, point clouds can be converted into more compact data structures, such as adjacency matrices and edge lists. Point cloud compression can be achieved by encoding and quantizing the graph structure.

[0257] Feature analysis: By analyzing the node attributes and edge weights in the graphical representation, various features of the point cloud can be extracted, such as local shape descriptors, global structural features, etc., which can be used for tasks such as point cloud classification, segmentation, and retrieval.

[0258] Semantic understanding: By introducing semantic attributes such as object categories and component labels in graphical representations, semantic segmentation and understanding of point clouds can be achieved, and points in the point cloud can be given a higher level of meaning for applications such as scene understanding and object detection.

[0259] Point cloud completion: By utilizing the topological relationships in the graphical representation, incomplete or missing point clouds can be repaired and completed. By using methods such as graph node interpolation and edge prediction, the positions and attributes of missing points can be estimated to improve the integrity of the point cloud.

[0260] Point cloud generation: By learning the graphical representation of point clouds, we can build generative models, such as graph convolutional neural networks, graph variational autoencoders, etc., starting from the graph structure, to generate realistic point cloud data, which can be applied to tasks such as point cloud synthesis, enhancement, and editing.

[0261] The application of graphical representation demonstrates the advantages of converting point clouds into graph structures. Through the analysis and processing of graphs, the features and semantics of point clouds can be better mined, and efficient compression, robust representation, and intelligent analysis of point clouds can be achieved. However, the application of graphical representation is also limited by the complexity of graphs and computational efficiency, and it is necessary to balance the accuracy of representation and the computational overhead in specific tasks.

[0262] The above are several common methods of intra-block graphical representation and their implementation processes, including the construction of vertex graphs, edge graphs, and hypergraphs, feature extraction and graph construction, and the application of graphical representation. These methods characterize the geometric, topological, and semantic features of point clouds from different angles, converting point clouds into graphs, and providing a basis for subsequent transmission, analysis, and processing. Choosing a suitable graphical representation method requires considering factors such as the scale, complexity, and application requirements of the point cloud, and weighing the effectiveness of the representation and the efficiency of the calculation. At the same time, as an intermediate representation, graphical representation also needs to be collaboratively designed and optimized with other modules, such as encoding, transmission, and rendering, in order to maximize its potential in point cloud transmission and application.

[0263] Through this embodiment, large-scale point cloud data is divided into multiple small blocks, and each block contains a part of the point cloud data. The purpose of blocking is to transform the global point cloud processing problem into local sub-problems, reducing the complexity of processing and memory overhead. At the same time, blocking can also support layered transmission and progressive rendering of point clouds, improving the flexibility and efficiency of transmission. The data in each point cloud block is graphically represented to extract its structure and attribute characteristics. The purpose of the graphical representation is to convert irregular point cloud data into regular graph structure data, which is convenient for subsequent encoding, compression and transmission. At the same time, the graphical representation can also reflect the local characteristics and global topology of the point cloud, providing richer information for subsequent analysis and processing. Figure 5 is a schematic diagram of another optional method for processing point cloud data according to an embodiment of the present application, such as Figure 5 As shown in the figure, the process of combining point cloud segmentation and intra-block graphical representation in data flow and calculation method is as follows:

[0264] S1, data input and preprocessing: Input the original point cloud data and perform preprocessing on it, such as denoising, filtering, downsampling, etc., to improve data quality and processing efficiency. Calculate the global features of the point cloud, such as bounding box, octree, kd tree, etc., to provide reference for subsequent block segmentation.

[0265] S2, point cloud segmentation: According to the global features obtained by preprocessing, select a suitable segmentation method, such as octree segmentation, kd tree segmentation, uniform grid segmentation, etc. Traverse the point cloud data, assign each point to the corresponding block according to the segmentation method, and generate a set of point cloud blocks. For each point cloud block, calculate its bounding box, center point, average density and other attributes for subsequent graphical representation.

[0266] S3, graphical representation within the block: For each point cloud block, extract its local features, such as normal vectors, curvature, eigenvalues, etc., as the attributes of the graph nodes. According to the distribution and connection relationship of the point cloud within the block, construct graphical representations, such as vertex graphs, edge graphs, hypergraphs, etc. Simplify and optimize the constructed graph, such as removing redundant edges and merging similar nodes, to improve the compactness and robustness of the graph.

[0267] S4, graph encoding and compression: Encode the graphical representation of each block and convert it into a data format that is easy to transmit, such as an adjacency matrix, edge list, etc. Use graph signal processing and data compression techniques, such as graph Fourier transform, wavelet transform, entropy coding, etc. to compress the graph data. Generate a compressed graph data stream, including the graph structure information and node attribute information, ready for transmission.

[0268] S5, data transmission and decoding: The compressed graph data stream is transmitted in blocks, and the transmission order and rate are dynamically adjusted according to the network bandwidth and the capabilities of the receiving end. At the receiving end, the received graph data stream is decoded and decompressed to restore the original graphical representation. Based on the information in the graphical representation, the local geometry and attributes of each point cloud block are reconstructed.

[0269] S6, point cloud reconstruction and rendering: The received point cloud blocks are spliced ​​according to the original spatial positions to generate a complete point cloud model. The reconstructed point cloud is post-processed, such as smoothing, coloring, simplification, etc., to improve the visual quality of the point cloud. The processed point cloud is transmitted to the rendering engine for real-time display and interaction, supporting multi-level detail control.

[0270] In the above process, point cloud segmentation and intra-block graphical representation are closely combined, and large-scale point clouds are converted into multiple small-scale graph structure data through divide-and-conquer and graphical methods, thus achieving efficient compression and transmission of point clouds. Specifically:

[0271] Point cloud segmentation decomposes the global transmission problem into multiple local sub-problems, reducing the size of data processed each time, reducing computational complexity and memory overhead. The intra-block graphical representation converts irregular point cloud data into regular graph structure data, uses the characteristics of the graph to extract features and compress data, and improves data compactness and transmission efficiency. The segmentation and graphical representation cooperate with each other. The segmentation provides appropriate processing units and space divisions for the graphical representation, and the graphical representation provides efficient data representation and compression methods for the segmentation. In the actual transmission process, the segmentation and graphical representation can be adaptively adjusted according to specific application scenarios and performance requirements, such as the selection of segment size, the type of graphical representation, and parameter settings. By dynamically controlling the granularity of the segmentation and the accuracy of the graphical representation, a trade-off can be made between transmission efficiency and reconstruction quality to meet the needs of different users. In addition, the segmentation and graphical representation also provide a basis for the progressive transmission and multi-level display of point cloud data. By first transmitting low-level blocks and rough graphical representations, users can quickly be presented with a global overview of the point cloud; as the transmission progresses, high-level blocks and detailed graphical representations are gradually loaded to continuously improve the details and quality of the point cloud. This progressive transmission method can improve the user's interactive experience while reducing the waste of network bandwidth and computing resources.

[0272] Data flow: Point cloud data reading: Read the original point cloud data from the storage medium. Block processing: Spatially block the read point cloud data to generate multiple small blocks. In-block processing: Perform neighborhood search, graph construction and feature extraction in each block to generate a graphical representation. Encoding and compression: Encode and compress the graphical representation of each block to generate a data packet that is easy to transmit. Data transmission: Transmit the encoded and compressed data packet through the network. Decoding and decompression: The receiving end decodes and decompresses the data packet to restore the graphical representation and point cloud data. Point cloud reconstruction: Reconstruct the original point cloud data based on the restored graphical representation.

[0273] Calculation method:

[0274] Block algorithm: Use octree, kd-tree or uniform grid algorithm to block the point cloud.

[0275] Neighborhood search algorithm: Use k-nearest neighbor search or radius search algorithm to determine the neighbor points of each point.

[0276] Graph construction algorithm: construct vertex graph, edge graph or hypergraph, generate adjacency matrix and weight matrix.

[0277] Feature extraction algorithm: Use PCA, curvature calculation and other methods to extract the geometric and topological features of the point cloud.

[0278] Coding and compression algorithms: Encode and compress graphical representations and features, using sparse matrix representation and compression algorithms.

[0279] Decoding and decompression algorithms: Recover graphical representation and point cloud data using corresponding decoding and decompression algorithms.

[0280] Specific implementation case: Assume that there is a large-scale point cloud dataset that needs to be transmitted based on blocks and graphical representations:

[0281] Read data: Read point cloud data from a file and determine its bounding box.

[0282] Octree Blocking: Use the octree algorithm to recursively divide the point cloud data into multiple sub-blocks, each of which contains part of the point cloud data.

[0283] In-block processing: The following processing is performed in each sub-block:

[0284] Neighborhood search: Perform a k-nearest neighbor search on each point to determine its neighboring points.

[0285] Graph construction: Build a vertex graph based on the neighborhood search results and generate an adjacency matrix and an edge weight matrix.

[0286] Feature extraction: Extract geometric features (such as normal vectors and curvature) and topological features (such as adjacency relationships).

[0287] Encoding and compression: Encode and compress the graphical representation of each sub-block to generate a transmission data packet.

[0288] Data transmission: The transmission of data packets over the network to the receiving end.

[0289] Decoding and decompression: The receiving end decodes and decompresses the data packet to recover the graphical representation and point cloud data.

[0290] Point cloud reconstruction: Reconstruct the original point cloud data based on the recovered graphical representation to complete the transmission and presentation of the point cloud.

[0291] Detailed implementation process of graph encoding and transmission method:

[0292] Coding and transmission based on graph Fourier transform: Graph Fourier Transform (GFT) is a method that generalizes the classic Fourier transform to the field of graph signal processing. It uses the eigenvectors of the Laplacian matrix of the graph as basis functions to transform the graph signal from the vertex domain to the frequency domain, and obtains the graph spectrum coefficients as the frequency domain representation of the graph signal. In the graphical representation of the point cloud, the geometric and attribute information within the point cloud block can be regarded as a signal defined on the graph, which is transformed into the frequency domain for encoding through GFT. The specific implementation process is as follows:

[0293] For the graphical representation of each point cloud block, construct its Laplacian matrix L. The Laplacian matrix is ​​a basic representation of a graph, reflecting the topological structure of the graph and the similarity between nodes, and is defined as L = DA, where D is the degree matrix of the node and A is the adjacency matrix of the graph.

[0294] Perform eigendecomposition on the Laplace matrix L to obtain its eigenvalues ​​and eigenvectors. The eigenvalues ​​reflect the energy distribution of the graph signal at different frequencies, and the eigenvectors constitute the basis functions of the graph Fourier transform.

[0295] Map the graph signal f (such as node coordinates, normal vectors, colors, etc.) to the basis of the feature vector to obtain the graph coefficient The spectrum coefficient indicates the component size of the signal at different frequencies and reflects the frequency domain characteristics of the signal.

[0296] Spectral coefficient Encoding, such as quantization and entropy coding, is performed to obtain compressed data. Since the energy of image signals is usually concentrated in the low-frequency part and the energy in the high-frequency part is small, a method similar to JPEG can be used to finely quantize the low-frequency coefficients and roughly quantize the high-frequency coefficients to achieve efficient compression.

[0297] The encoded graph spectral coefficients are transmitted, and the graph topological structure information (such as the adjacency matrix) is transmitted at the same time. At the receiving end, the Laplacian matrix of the graph is reconstructed according to the topological structure, and the received graph spectral coefficients are decoded to obtain the reconstructed graph signal. The GFT-based graph coding method can effectively utilize the frequency domain characteristics of the graph signal to achieve efficient compression of graph data. However, its computational complexity is high, and it requires feature decomposition of large-scale graphs. In addition, both the encoding and decoding processes depend on the topological structure of the graph, and have poor adaptability to dynamically changing graphs.

[0298] Coding and transmission based on graph wavelet transform: Graph Wavelet Transform (GWT) is a method of applying traditional wavelet transform to graph signal processing. Similar to GFT, GWT is also a frequency domain transformation method, but it realizes multi-scale and multi-resolution representation of graph signals by defining wavelet basis functions on the graph. Compared with GFT, GWT can better capture the local characteristics of graph signals and has lower computational complexity. In the graphical representation of point clouds, GWT can be used to encode and transmit graph signals. The specific implementation process is as follows:

[0299] For the graphical representation of each point cloud block, select an appropriate graph wavelet basis function. Commonly used graph wavelet basis functions include Haar wavelet, Mexican hat wavelet, Meyer wavelet, etc. They are defined on the vertex domain of the graph and reflect the local characteristics at different scales.

[0300] The image signal is forward transformed using the selected image wavelet basis function to obtain the image wavelet coefficients. The image wavelet coefficients reflect the component sizes of the image signal at different scales and different spatial positions, and contain the multi-resolution information of the image signal.

[0301] The wavelet coefficients of the graph are encoded and compressed by thresholding, quantization, entropy coding and other methods. Since the wavelet transform of the graph has good energy concentration and the amplitude of most coefficients is small, a coding method similar to SPIHT can be used to achieve efficient compression.

[0302] The encoded graph wavelet coefficients are transmitted, and the scale and position information required for the graph wavelet transform are transmitted at the same time. At the receiving end, the graph wavelet basis function is reconstructed based on this information, and the received wavelet coefficients are decoded to obtain the reconstructed graph signal. The GWT-based graph coding method can achieve multi-scale representation and local feature capture of graph signals, and the computational complexity is relatively low. However, its coding efficiency depends on the selected wavelet basis function, and different basis functions may need to be used for different types of graph signals. In addition, GWT is also sensitive to changes in the topological structure of the graph, and the wavelet basis function needs to be recalculated.

[0303] Coding and transmission based on attribute quantization: In the graphical representation of point clouds, in addition to the topological structure information of the graph, nodes and edges also contain rich attribute information, such as the geometric coordinates, normal vectors, colors, etc. of nodes, and the weights and types of edges. In order to further improve the compression rate of graph data, these attribute information can be quantized and encoded. The basic idea of ​​attribute quantization is to reduce the amount of data required to represent the attributes by reducing the precision of the attribute values. The specific implementation process is as follows:

[0304] For each attribute dimension (such as X coordinate, Y coordinate, Z coordinate, etc.), determine its value range, and divide the value range into several intervals according to the required accuracy. Each interval corresponds to a quantization level, which is represented by a smaller number of bits.

[0305] For each node or edge attribute value, it is quantized to the corresponding level according to the interval it is in. The quantized attribute value is represented by fewer bits, but it will introduce some distortion.

[0306] Encode the quantized attribute values, such as lossless compression, entropy coding, etc., to further reduce the amount of data.

[0307] The encoded attribute data is transmitted together with the topological structure information of the graph. At the receiving end, the attribute value is dequantized according to the quantization interval and level to obtain the reconstructed attribute information. The encoding method based on attribute quantization can significantly reduce the transmission volume of graph data by reducing data accuracy in exchange for compression rate. However, the quantization process inevitably introduces information loss, and a trade-off needs to be made between compression rate and reconstruction quality. In addition, for different types of attributes, the selection of quantization methods and parameters also needs to be adaptively adjusted according to their characteristics.

[0308] Progressive transmission and update: It aims to dynamically adjust the transmission content and order of point cloud data according to network bandwidth and user needs, and realize hierarchical loading and local update of point cloud. The following are the detailed implementation processes of several common progressive transmission and update methods:

[0309] Progressive transmission based on graph pyramid: Graph pyramid is a commonly used hierarchical representation method for graph data. It generates graph approximations at different levels by simplifying and compressing the graph layer by layer. In the graphical representation of point clouds, graph pyramids can be used to achieve progressive transmission of point clouds. The specific implementation process is as follows:

[0310] For each graphical representation of a point cloud block, a graph pyramid is constructed. Starting from the original graph, the graph structure is gradually simplified through edge collapse, node clustering and other operations to generate graph approximations at different levels. Each layer of the graph is a compressed version of the next layer of the graph, and the number of nodes and edges decreases layer by layer.

[0311] During the transmission process, the top-level graph of the pyramid, i.e. the coarsest-grained graph approximation, is transmitted first. This layer of graph contains the basic structure and main features of the point cloud, and can quickly present the overall outline of the point cloud to the user.

[0312] The lower layers of the pyramid are transmitted gradually according to the network bandwidth and user interaction. Each transmission of a layer of the pyramid is equivalent to providing a more refined point cloud representation to the user, gradually loading more geometric and attribute details.

[0313] At the receiving end, different levels of point cloud approximation are reconstructed based on the received image data. Users can select the appropriate level of point cloud for display and interaction based on their needs and device capabilities.

[0314] When network conditions allow or the user requests higher accuracy, the lower layers of the pyramid continue to be transmitted until the entire pyramid is transmitted to achieve the accuracy of the original point cloud. The progressive transmission method based on the graph pyramid can dynamically adjust the transmission content and accuracy of the point cloud according to network and user needs, and realize multi-level presentation of the point cloud. However, this method requires the pre-construction of the graph pyramid. For dynamically changing point clouds, the pyramid structure needs to be updated, which has a large computational overhead.

[0315] Priority-based progressive transmission: In addition to the graph pyramid, the priority of transmission can be determined based on the importance of nodes and edges in the graph to achieve progressive loading of point clouds. The importance of nodes and edges can be measured based on their geometric, topological, and visual features, such as the curvature of nodes, the weight of edges, and the visibility of nodes. The specific implementation process is as follows:

[0316] For each point cloud block graphical representation, calculate the importance metrics of its nodes and edges. Commonly used metrics include:

[0317] Geometric importance: such as the curvature and normal vector change of the node, reflecting the significance of the node in the local or global geometry.

[0318] Topological importance: such as the degree of a node and the betweenness centrality of an edge, reflecting the key role of a node or edge in the graph structure.

[0319] Visual importance: such as node visibility, color contrast, etc., reflecting the visual significance of the node in the rendered image.

[0320] Nodes and edges are sorted according to the importance metric to determine their transmission priority. Nodes and edges with high importance are transmitted first to ensure that the main features and structures of the point cloud are presented to users as early as possible.

[0321] During the transmission process, nodes and edges are gradually transmitted in order of priority. An importance threshold can be set to only transmit nodes and edges with importance higher than the threshold to reduce the amount of data.

[0322] At the receiving end, the point cloud is gradually reconstructed based on the received node and edge data. Since the nodes and edges with high importance are transmitted first, users can quickly see the main features of the point cloud, and then the details of the point cloud are continuously enriched as the transmission proceeds.

[0323] When all nodes and edges are transferred, the point cloud reconstruction reaches full accuracy. Users can interact and analyze with high-fidelity point clouds. The priority-based progressive transmission method can optimize the transmission order according to the importance characteristics of the point cloud and improve the efficiency of point cloud presentation. However, this method requires reasonable setting of importance metrics and thresholds to balance the amount of transmitted data and reconstruction quality.

[0324] Viewpoint-dependent progressive transmission: In many application scenarios, the viewpoint position and direction of the user's point cloud are constantly changing. In order to further optimize the transmission efficiency, the transmission priority and detail level of the point cloud block can be adjusted according to the user's real-time viewpoint to achieve viewpoint-dependent progressive transmission. The specific implementation process is as follows:

[0325] During the transmission process, the user's viewpoint position and viewing direction are continuously tracked. The user's viewpoint information can be obtained through interactive devices (such as mouse, keyboard, gyroscope, etc.).

[0326] According to the user's viewpoint, the point cloud blocks within the current viewing cone are determined. The viewing cone is a three-dimensional space area within a certain angle range with the user's viewpoint as the vertex and the viewing direction as the central axis.

[0327] For point cloud blocks within the view frustum, increase their transmission priority and level of detail. These blocks correspond to the point cloud areas that the user is currently focusing on and need to be loaded and rendered as quickly as possible.

[0328] For point cloud blocks outside the view frustum, reduce their transmission priority and detail level. These blocks are not currently in the user's field of view and can be temporarily simplified or delayed to save transmission resources.

[0329] As the user's viewpoint moves and changes, the transmitted point cloud blocks and detail levels are dynamically adjusted. When a new point cloud block enters the view frustum, its transmission priority is increased; when the original block leaves the view frustum, its priority is reduced or the transmission is stopped.

[0330] At the receiving end, the rendering and display of the point cloud are dynamically updated according to the received point cloud block data. The point cloud blocks within the user's field of view are presented with high priority and high fidelity, while the blocks outside the field of view are simplified or hidden to improve rendering efficiency. The viewpoint-related progressive transmission method can make full use of the user's interactive behavior, dynamically adjust the transmission and presentation strategy of the point cloud, and provide a more real-time and immersive user experience. However, this method requires real-time tracking of the user's viewpoint and updating of the transmission priority, which places higher requirements on the system's response speed and computing power.

[0331] Progressive transmission based on region selection: In addition to automatically adjusting the transmission priority, users can also actively select the point cloud area of ​​interest, focus on transmitting and loading it, and realize progressive transmission based on region selection. The specific implementation process is as follows:

[0332] During the initial transmission, a low-level graphical representation of the point cloud is transmitted first, such as the top-level graph of the pyramid or the first few nodes and edges sorted by importance.

[0333] Based on the received data, a rough preview of the point cloud is reconstructed on the client, and an interactive interface for area selection is provided. Users can mark the point cloud area of ​​interest by selecting with the mouse or by circling on the touch screen.

[0334] The area information selected by the user is sent to the server, and the server determines the point cloud blocks that need to be transmitted based on the information.

[0335] For point cloud blocks within the area selected by the user, their transmission priority and detail level are increased. These blocks contain point cloud details that the user is interested in and need to be transmitted and rendered as quickly as possible.

[0336] For unselected areas, their original transmission priority and detail level are maintained or appropriately reduced to balance transmission resources.

[0337] On the receiving end, the rendering and display of the selected area is dynamically updated based on the received point cloud block data. The area of ​​interest to the user is presented with high fidelity, while other areas maintain a coarse or simplified representation.

[0338] Users can change the selected area at any time, and the server adjusts the transmission strategy based on the new selection information to achieve interactive area refinement and exploration.

[0339] The progressive transmission method based on region selection can give users greater control, dynamically adjust the transmission and presentation of point clouds according to the user's points of interest, and provide a more personalized and proactive point cloud interaction experience. However, this method requires the design of an intuitive and efficient region selection interface and the handling of transmission scheduling issues when users frequently change their selections.

[0340] Performance optimization of progressive transmission: In order to further improve the efficiency of progressive transmission and user experience, various performance optimization strategies can be introduced during the transmission process, such as caching mechanism, prediction mechanism, adaptive adjustment, etc. The specific implementation process is as follows:

[0341] Cache mechanism: The server and client maintain caches of transmitted point cloud blocks. When a user repeatedly requests a block, it is read from the cache first to avoid repeated transmission. In addition, the cache update and replacement strategy can be dynamically adjusted based on factors such as block importance and temporal locality.

[0342] Prediction mechanism: By analyzing the user's interactive behavior and browsing patterns, predict the point cloud areas that the user may be interested in, and transfer and load the data of these areas in advance. Common prediction methods include:

[0343] Motion prediction: Based on the movement trajectory and speed of the user's viewpoint, predict the point cloud blocks that may enter the field of view in the future.

[0344] Hotspot prediction: Based on the interactive behavior statistics of multiple users, point cloud areas with high general attention are identified, and data of these areas are actively pushed.

[0345] Adaptive adjustment: Adaptively adjust transmission parameters and strategies based on dynamic changes in factors such as network bandwidth, terminal performance, and user interaction. For example, reduce the transmission bit rate and resolution when the network is congested, and increase the response speed and update frequency when user interaction is frequent.

[0346] Multi-level buffering: Maintain multiple levels of point cloud rendering buffers on the client, such as geometry buffer, attribute buffer, texture buffer, etc. According to the arrival of data at different levels, gradually update and synthesize the final rendering results to avoid freezes or delays due to waiting for data at a certain level.

[0347] Progressive encoding and decoding: A progressive strategy is also used in the encoding and decoding process of point cloud data. For example, during encoding, multiple levels of bit streams are generated according to importance or priority; during decoding, the point cloud model is gradually restored and refined according to the received bit stream to achieve progressive compression and reconstruction.

[0348] By introducing these performance optimization strategies, we can dynamically adapt the content and rhythm of transmission in different network environments and interaction scenarios, improve the efficiency and robustness of point cloud progressive transmission, and bring users a smoother and more immersive point cloud interaction experience. At the same time, these optimization strategies also lay a technical foundation for the large-scale application and popularization of point clouds.

[0349] In summary, progressive transmission and update is an important strategy in point cloud transmission methods based on blocks and graphical representation. Through mechanisms such as graph pyramid, priority sorting, viewpoint correlation, and region selection, the transmission order and detail level of point cloud blocks are dynamically adjusted to achieve hierarchical loading and local update of point clouds. At the same time, performance optimization strategies such as caching, prediction, and adaptation are introduced to further improve the efficiency of transmission and user experience. This progressive transmission method makes full use of the multi-level representation and signal processing technology of the graph structure, and can efficiently compress and transmit large-scale point cloud data under limited bandwidth and computing resources. Compared with directly transmitting the original point cloud, this method greatly reduces the amount of data while retaining the main features and details of the point cloud, providing efficient, flexible, and scalable data transmission and interaction support for various point cloud applications, such as virtual reality, reverse engineering, and autonomous driving. With the explosive growth of point cloud data and the continuous improvement of application requirements, progressive transmission and update will surely become an indispensable key technology in point cloud processing and application, with broad application prospects and research value.

[0350] Therefore, traditional point cloud processing methods usually operate directly on the original point coordinates and attributes, which makes it difficult to fully utilize the structure and topological information of the point cloud. This method maps the point cloud into a graph, uses nodes to represent points, and uses edges to represent the relationship between points, so as to better characterize the intrinsic structure and characteristics of the point cloud. The graph-based representation can not only simplify the processing and analysis of point clouds, but also lay the foundation for the theory and methods of applying graph theory and graphic signal processing. Dividing large-scale point cloud data into multiple small blocks according to spatial positions for processing is another important innovation of this method. Traditional point cloud processing methods usually process the entire point cloud as a whole, which is difficult to cope with the computing and storage challenges brought by large-scale point cloud data. This method adopts a block strategy to transform the global processing problem into a local sub-problem, and improves the processing efficiency through parallel and distributed methods. At the same time, block processing also provides support for the hierarchical transmission and local update of point clouds, making the transmission and presentation of point clouds more flexible and efficient. On the basis of graphical representation, further constructing a multi-level and multi-resolution representation of point clouds is another innovation of this method. Through graph pyramid, graph wavelet and other technologies, the graphical point cloud data is converted into different levels of approximate representation, from coarse to fine, from global to local, gradually depicting the structure and details of the point cloud. The multi-level representation can not only adapt to different transmission bandwidths and display devices, but also support progressive loading and interactive exploration of point clouds, providing a more flexible and adaptive point cloud presentation method. Dynamically adjusting the transmission content and order of point cloud data according to network bandwidth and user needs to achieve progressive loading and updating is an important innovation of this method. Traditional point cloud transmission methods usually adopt a one-time transmission strategy, which is difficult to adapt to changes in network conditions and user interaction needs. This method introduces a progressive transmission strategy based on mechanisms such as importance, viewpoint correlation, and region selection. It can dynamically adjust the transmission priority and detail level according to the real-time network status and user behavior, providing a more real-time and interactive point cloud browsing experience.

[0351] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0352] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0353] According to another aspect of the embodiment of the present application, a point cloud data processing device for implementing the above-mentioned point cloud data processing method is also provided. Figure 6 As shown, the device comprises:

[0354] An acquisition module 602 is used to acquire initial point cloud data to be sent;

[0355] A division module 604 is used to divide the initial point cloud data into a group of point cloud blocks;

[0356] A generating module 606 is used to generate a set of graph structure data based on the graphical representation of the point cloud data in each point cloud block in the set of point cloud blocks, wherein one graph structure data in the set of graph structure data corresponds to one point cloud block in the set of point cloud blocks, and the graph structure data is used to represent the features of the corresponding point cloud block;

[0357] The processing module 608 is used to encode the set of graph structure data to obtain a set of data packets, and send the set of data packets to a target terminal to be decoded and reconstructed at the target terminal to obtain target point cloud data.

[0358] As an optional scheme, the device is used to generate a set of graph structure data by graphically representing the point cloud data in each point cloud block in the set of point cloud blocks in at least one of the following ways: performing a feature extraction operation on the target point cloud block to determine the target local features corresponding to the target point cloud block; calculating the topological relationship for at least two points in the target point cloud block to determine the target topological relationship corresponding to the target point cloud block; generating target graph structure data based on the target local features and the target topological relationship, wherein the node attributes of each node in the target graph structure data are determined by the target local features, and the connection relationship of each node in the target graph structure data is determined by the target topological relationship.

[0359] As an optional scheme, the device is used to perform a feature extraction operation on the target point cloud block to determine the target local features corresponding to the target point cloud block in at least one of the following ways: performing a normal vector feature extraction operation on each point in the target point cloud block to determine the normal vector feature corresponding to the target point cloud block, wherein the normal vector feature is used to represent the direction of the surface where the corresponding point is located; performing a curvature feature extraction operation on each point in the target point cloud block to determine the curvature feature corresponding to the target point cloud block, wherein the curvature feature is used to represent the degree of curvature of the surface where the corresponding point is located; performing principal component analysis on the target point cloud block to obtain eigenvalues ​​and eigenvectors, wherein the eigenvalues ​​and the eigenvectors are used to represent the distribution direction and shape of the target point cloud block.

[0360] As an optional scheme, the device is used to calculate the topological relationship of at least two points in the target point cloud block and determine the target topological relationship corresponding to the target point cloud block in at least one of the following ways: calculating the topological relationship of two points in the target point cloud block to determine whether the two points are adjacent; calculating the topological relationship of three points in the target point cloud block to determine whether the three points constitute a target triangle, wherein the three vertices of the target triangle are adjacent to each other; calculating the topological relationship of two target triangles in the target point cloud block to determine whether there is a shared edge, wherein the shared edge indicates that there are two common vertices between the two target triangles, and the three vertices of the target triangle are adjacent to each other.

[0361] As an optional scheme, the device is used to generate target graph structure data based on the target local features and the target topological relationship in at least one of the following ways: generating a vertex graph based on the target local features and the target topological relationship, wherein the target graph structure data includes the vertex graph, a vertex of the vertex graph represents a point in the target point cloud block, and an edge of the vertex graph represents a connection relationship between two points in the target point cloud block to indicate the similarity between the two points; generating an edge graph based on the target local features and the target topological relationship, wherein the target graph structure data includes the edge graph, a node of the edge graph represents a pair of points in the target point cloud block whose similarity satisfies a first condition, and an edge of the edge graph represents that the similarity between two pairs of points in the target point cloud block satisfies a second condition; generating a hypergraph based on the target local features and the target topological relationship, wherein the target graph structure data includes the hypergraph, the hypergraph includes hyperedges and edges of the hypergraph, the hyperedges represent at least two points whose similarity of high-order features satisfies a third condition, and the edges of the hypergraph represent that the similarity between two of the hyperedges satisfies a fourth condition.

[0362] As an optional scheme, the device is used to divide the initial point cloud data into a group of point cloud blocks in the following manner: determine an initial blocking method based on the distribution characteristics of the initial point cloud data; block the initial point cloud data according to the initial blocking method to obtain a group of initial point cloud blocks; determine an initial distribution quality parameter corresponding to the group of initial point cloud blocks, determine a target blocking method based on the initial distribution quality parameter, block the group of initial point cloud blocks according to the target blocking method to obtain a group of intermediate point cloud blocks; repeat the following steps until the intermediate distribution quality parameter meets a preset condition, and determine the group of intermediate point cloud blocks as the group of point cloud blocks, wherein the intermediate distribution quality parameter corresponds to the group of intermediate point cloud blocks: update the target blocking method based on the intermediate distribution quality parameter, re-block according to the updated target blocking method, and determine the updated intermediate distribution quality parameter.

[0363] As an optional scheme, the device is also used for: after dividing the initial point cloud data into a group of point cloud blocks, merging the point cloud blocks based on the degree of similarity between the respective point cloud blocks in the group of point cloud blocks to obtain multi-level point cloud blocks including different levels, wherein the group of point cloud blocks includes the multi-level point cloud blocks; graphically representing the point cloud data in each point cloud block in the multi-level point cloud blocks to generate multi-level graph structure data, wherein the group of graph structure data includes the multi-level graph structure data; encoding the multi-level graph structure data layer by layer to obtain a multi-level data packet; and sending the multi-level data packet to the target terminal layer by layer according to the network bandwidth and display capability of the target terminal, so as to be decoded and reconstructed at the target terminal to obtain the target point cloud data.

[0364] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0365] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0366] According to one aspect of the present application, a computer program product is provided. The computer program product includes a computer program.

[0367] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0368] Figure 7 The structure block diagram of a computer system for implementing an electronic device according to an embodiment of the present application is schematically shown.

[0369] It should be noted that Figure 7 The computer system 700 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0370] like Figure 7 As shown, the computer system 700 includes a central processing unit 701 (CPU), which can perform various appropriate actions and processes according to the program stored in the read-only memory 702 (ROM) or the program loaded from the storage part 708 to the random access memory 703 (RAM). Various programs and data required for system operation are also stored in the random access memory 703. The central processing unit 701, the read-only memory 702 and the random access memory 703 are connected to each other through a bus 704. An input / output interface 705 (Input / Output interface, i.e., I / O interface) is also connected to the bus 704.

[0371] The following components are connected to the input / output interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed so that a computer program read therefrom is installed into the storage section 708 as needed.

[0372] In particular, according to an embodiment of the present application, the process described in each method flow chart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program contains a program code for executing the method shown in the flow chart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processor 701, various functions defined in the system of the present application are executed.

[0373] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the central processor 701, various functions provided by the embodiment of the present application are performed.

[0374] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned point cloud data processing method is also provided. The electronic device may be Figure 1 The terminal device or server shown in the figure. This embodiment is described by taking the electronic device as a terminal device as an example. Figure 8 As shown, the electronic device includes a memory 802 and a processor 804. The memory 802 stores a computer program, and the processor 804 is configured to execute the steps in any of the above method embodiments through the computer program.

[0375] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0376] Optionally, in this embodiment, the above-mentioned processor can be configured to execute the methods in each embodiment of the present application through a computer program.

[0377] Alternatively, a person skilled in the art may understand that: Figure 8 The structure shown is for illustration only. Figure 8 The structure of the electronic device is not limited. Figure 8 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 8 Different configurations are shown.

[0378] Among them, the memory 802 can be used to store software programs and modules, such as the program instructions / modules corresponding to the point cloud data processing method and device in the embodiment of the present application. The processor 804 executes various functional applications and data processing by running the software programs and modules stored in the memory 802, that is, realizing the above-mentioned point cloud data processing method. The memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 802 may further include a memory remotely located relative to the processor 804, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 802 can be specifically, but not limited to, used to store information such as point cloud data. As an example, such as Figure 8 As shown, the memory 802 may include, but is not limited to, the acquisition module 602, the division module 604, the generation module 606, and the processing module 608 in the point cloud data processing device. In addition, it may also include, but is not limited to, other module units in the point cloud data processing device, which will not be repeated in this example.

[0379] Optionally, the transmission device 806 is used to receive or send data via a network. Specific examples of the network may include wired networks and wireless networks. In one example, the transmission device 806 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices and routers via a network cable so as to communicate with the Internet or a local area network. In one example, the transmission device 806 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0380] In addition, the electronic device further includes: a display 808 for displaying the point cloud data; and a connection bus 810 for connecting various module components in the electronic device.

[0381] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting the multiple nodes through network communication. The nodes may form a peer-to-peer network, and any form of computing device, such as a server, terminal or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.

[0382] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the point cloud data processing method provided in various optional implementations of the above-mentioned point cloud data processing.

[0383] Optionally, in this embodiment, the above-mentioned computer-readable storage medium can be configured to store data for executing the methods in various embodiments of the present application.

[0384] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0385] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0386] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for one or more electronic devices to execute all or part of the steps of the methods described in each embodiment of the present application.

[0387] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0388] In the several embodiments provided in the present application, it should be understood that the disclosed application can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0389] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0390] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0391] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for processing point cloud data, characterized in that: include: Get the initial point cloud data to be sent; Dividing the initial point cloud data into a group of point cloud blocks; Based on the graphical representation of the point cloud data in each point cloud block in the set of point cloud blocks, a set of graph structure data is generated, wherein one graph structure data in the set of graph structure data corresponds to one point cloud block in the set of point cloud blocks, and the graph structure data is used to represent the features of the corresponding point cloud block; The set of graph structure data is encoded to obtain a set of data packets, and the set of data packets is sent to a target terminal to be decoded and reconstructed at the target terminal to obtain target point cloud data.

2. The method according to claim 1, characterized in that The step of graphically representing the point cloud data in each point cloud block in the set of point cloud blocks to generate a set of graph structure data includes: Performing a feature extraction operation on the target point cloud block to determine the target local features corresponding to the target point cloud block; Calculating a topological relationship between at least two points in the target point cloud block to determine a target topological relationship corresponding to the target point cloud block; Target graph structure data is generated based on the target local features and the target topological relationship, wherein the node attributes of each node in the target graph structure data are determined by the target local features, and the connection relationship of each node in the target graph structure data is determined by the target topological relationship.

3. The method according to claim 2, characterized in that The performing of a feature extraction operation on the target point cloud block to determine the target local feature corresponding to the target point cloud block includes at least one of the following: Performing a normal vector feature extraction operation on each point in the target point cloud block to determine a normal vector feature corresponding to the target point cloud block, wherein the normal vector feature is used to represent the direction of the surface where the corresponding point is located; Performing a curvature feature extraction operation on each point in the target point cloud block to determine a curvature feature corresponding to the target point cloud block, wherein the curvature feature is used to represent the curvature degree of the surface where the corresponding point is located; A principal component analysis is performed on the target point cloud block to obtain eigenvalues ​​and eigenvectors, wherein the eigenvalues ​​and the eigenvectors are used to represent the distribution direction and shape of the target point cloud block.

4. The method according to claim 2, characterized in that: The step of calculating a topological relationship between at least two points in the target point cloud block and determining a target topological relationship corresponding to the target point cloud block includes at least one of the following: Calculating a topological relationship between two points in the target point cloud block to determine whether the two points are adjacent; Calculating a topological relationship between three points in the target point cloud block to determine whether the three points form a target triangle, wherein three vertices of the target triangle are adjacent to each other; The topological relationship between the two target triangles in the target point cloud block is calculated to determine whether there is a shared edge, wherein the shared edge indicates that there are two common vertices between the two target triangles, and the three vertices of the target triangles are adjacent to each other.

5. The method according to claim 2, characterized in that: The generating target graph structure data based on the target local features and the target topological relationship includes at least one of the following: Generate a vertex graph based on the target local features and the target topological relationship, wherein the target graph structure data includes the vertex graph, a vertex of the vertex graph represents a point in the target point cloud block, and an edge of the vertex graph represents a connection relationship between two points in the target point cloud block to indicate the similarity between the two points; Generate an edge graph based on the target local features and the target topological relationship, wherein the target graph structure data includes the edge graph, a node of the edge graph represents a pair of points in the target point cloud block whose similarity satisfies a first condition, and an edge of the edge graph represents that the similarity between two pairs of points in the target point cloud block satisfies a second condition; A hypergraph is generated based on the target local features and the target topological relationship, wherein the target graph structure data includes the hypergraph, the hypergraph includes hyperedges and edges of the hypergraph, the hyperedges represent at least two points whose high-order feature similarity satisfies a third condition, and the edges of the hypergraph represent that the similarity between two of the hyperedges satisfies a fourth condition.

6. The method according to claim 1, characterized in that The step of dividing the initial point cloud data into a group of point cloud blocks comprises: Determining an initial block division method based on the distribution characteristics of the initial point cloud data; Divide the initial point cloud data into blocks according to the initial block division method to obtain a group of initial point cloud blocks; Determining initial distribution quality parameters corresponding to the group of initial point cloud blocks, determining a target block division method based on the initial distribution quality parameters, and dividing the group of initial point cloud blocks according to the target block division method to obtain a group of intermediate point cloud blocks; Repeat the following steps until the intermediate distribution quality parameter meets a preset condition, and determine the group of intermediate point cloud blocks as the group of point cloud blocks, wherein the intermediate distribution quality parameter corresponds to the group of intermediate point cloud blocks: The target block partitioning method is updated based on the intermediate distribution quality parameter, and blocks are re-performed according to the updated target block partitioning method to determine the updated intermediate distribution quality parameter.

7. The method according to claim 6, characterized in that After dividing the initial point cloud data into a group of point cloud blocks, the method further includes: Merging the point cloud blocks in the group of point cloud blocks based on the similarity between the point cloud blocks to obtain multi-level point cloud blocks including different levels, wherein the group of point cloud blocks includes the multi-level point cloud blocks; Generating multi-level graph structure data based on graphical representation of point cloud data in each point cloud block in the multi-level point cloud blocks, wherein the set of graph structure data includes the multi-level graph structure data; Encoding the multi-level graph structure data layer by layer to obtain a multi-level data packet; The multi-level data packets are sent to the target terminal layer by layer according to the network bandwidth and display capability of the target terminal, so as to be decoded and reconstructed at the target terminal to obtain the target point cloud data.

8. A point cloud data processing device, characterized in that: include: An acquisition module, used to acquire initial point cloud data to be sent; A division module, used for dividing the initial point cloud data into a group of point cloud blocks; A generating module, configured to generate a set of graph structure data based on a graphical representation of the point cloud data in each point cloud block in the set of point cloud blocks, wherein one graph structure data in the set of graph structure data corresponds to one point cloud block in the set of point cloud blocks, and the graph structure data is used to represent features of the corresponding point cloud block; The processing module is used to encode the set of graph structure data to obtain a set of data packets, and send the set of data packets to a target terminal so as to decode and reconstruct the data packets at the target terminal to obtain target point cloud data.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein the computer program can be executed by an electronic device to perform the method described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

11. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

Images