Point cloud data processing method, system, and storage medium

Abstract

The application discloses a point cloud data processing method and system and a storage medium, and relates to the technical field of computers. In the method, when the point cloud data is sampled for the i-th time, a first matrix composed of unsampled points in the point cloud data and a second matrix composed of i-1 sampled points obtained when the point cloud data is sampled for the i-1-th time are acquired, the distance between the unsampled points and the i-1 sampled points is acquired by constructing the first matrix and the second matrix of the same size, the distance between the current unsampled points and the sampled point set is determined according to the distance between the unsampled points and the sampled point set when the point cloud data is sampled for the i-1-th time, and then the point with the maximum distance to the sampled point set is determined as the i-th sampled point. In the above manner, when sampling is performed each time, only the distance between the sampled points obtained by the last sampling and each of the unsampled points needs to be calculated, so that the calculation cost is effectively saved, and the calculation efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a point cloud data processing method, system, and storage medium. Background Technology

[0002] In deep learning, 3D mesh data is typically converted into point cloud data, and the point cloud data is normalized using the farthest point sampling (FPS) algorithm. The core principle of the FPS algorithm is to iteratively select the point farthest from the selected point set in the point cloud data, so as to select a representative and evenly distributed point set from a large number of points.

[0003] In related technologies, the FPS algorithm includes the following steps: randomly select a point from the given point cloud data as the first sampling point and add it to the selected point set; calculate the distance from the unselected point to all points in the selected point set, take the shortest distance from each unselected point to the selected point set, select the maximum value among these shortest distances, and the unselected point corresponding to the maximum value is the next sampling point and add it to the selected point set; repeat the above iterative steps until the number of points in the selected point set reaches the preset number, and the sampling ends.

[0004] However, the aforementioned FPS algorithm requires calculating the distance from each unselected point to all points in the selected point set during each sampling, which incurs a large computational overhead and results in poor computational efficiency. Summary of the Invention

[0005] This application provides a point cloud data processing method, system, and storage medium that can save the computational overhead of the FPS algorithm and improve computational efficiency.

[0006] Firstly, this application provides a point cloud data processing method applicable to scenarios where point cloud data is processed using the FPS algorithm. For example, in 3D reconstruction scenarios, the original point cloud data volume is enormous. Sampling using the FPS algorithm can obtain a representative subset of the point cloud, reducing the data volume while retaining the main features of the original point cloud, facilitating subsequent processing and model building, and improving reconstruction efficiency. As another example, in target recognition and classification scenarios, the point cloud sampled by the FPS algorithm can serve as a feature representation of the target. These sampling points can capture key information such as the target's shape and structure. Machine learning or deep learning models can be trained based on these features to learn the feature patterns of different targets, thereby classifying new point cloud data and determining what type of target object it belongs to, and so on. The point cloud data processing method provided in this application includes:

[0007] Obtain a first matrix composed of multiple unsampled first points in the point cloud data and a second matrix composed of the (i-1)th sampled point in the first sampled point set. The first matrix and the second matrix have the same size, and i is a positive integer.

[0008] Based on the first matrix and the second matrix, determine the distance between each of the plurality of first points and the (i-1)th sampling point;

[0009] Based on the distance between each of the plurality of first points and the (i-1)th sampling point, and the distance between each of the plurality of unsampled second points in the point cloud data obtained when sampling the (i-1)th sampling point and the second sampling point set, the distance between each of the plurality of first points and the first sampling point set is determined. The second sampling point set includes i-2 sampling points. The distance between each of the plurality of first points and the first sampling point set is the minimum distance among the distances from each of the plurality of first points to the sampling points in the first sampling point set. Similarly, the distance between each of the plurality of second points and the second sampling point set is the minimum distance among the distances from each of the plurality of second points to the sampling points in the second sampling point set.

[0010] The point with the largest distance from the first set of sampling points among the plurality of first points is determined as the i-th sampling point.

[0011] Using the above method, when sampling the point cloud data for the i-th time, a first matrix composed of unsampled points and a second matrix composed of the (i-1)-th sampled points obtained from the (i-1)-th sampling are acquired. By constructing the first and second matrices of the same size, the distance between the unsampled points and the (i-1)-th sampled point is obtained. Based on the distance between the unsampled points and the sampled point set from the (i-1)-th sampling, the distance between the current unsampled point and the sampled point set is determined. The point with the largest distance to the sampled point set is then identified as the i-th sampled point. This approach means that in each sampling, only the distances between the sampled points obtained from the previous sampling and the unsampled points need to be calculated, effectively saving computational overhead and improving computational efficiency.

[0012] In some embodiments, the rows of the first matrix are used to indicate the number of the plurality of first points, and the columns of the first matrix are used to indicate the number of attributes of each of the plurality of first points. The values ​​in each row of the second matrix are identical, and each row's value is used to indicate the attribute of the (i-1)th sampling point. The information represented by the rows and columns of the first matrix can be interchanged; that is, the value of each column in the first matrix is ​​used to indicate the attribute of each point, and the number of attributes for each point can also be greater. This application does not limit this.

[0013] In some embodiments, determining the distance between each of the plurality of first points and the (i-1)th sampling point based on the first matrix and the second matrix includes:

[0014] Obtain the third matrix obtained by subtracting the first matrix from the second matrix;

[0015] The product of the third matrix and its transpose is used to determine the fourth matrix;

[0016] Based on the values ​​on the diagonal of the fourth matrix, determine the distance between each of the plurality of first points and the (i-1)th sampling point.

[0017] In some embodiments, determining the distance between each of the plurality of first points and the (i-1)th sampling point based on the first matrix and the second matrix includes:

[0018] Divide the first matrix and the second matrix into blocks to obtain K first matrix blocks corresponding to the first matrix and K second matrix blocks corresponding to the second matrix, where K is a positive integer;

[0019] Based on the K first matrix blocks and the K second matrix blocks, determine the distance between each point corresponding to each of the K first matrix blocks and the (i-1)th sampling point.

[0020] In some embodiments, determining the distance between each point corresponding to each of the K first matrix blocks and the (i-1)th sampling point, based on the K first matrix blocks and the K second matrix blocks, includes:

[0021] Obtain the fifth matrix by subtracting the j-th first matrix block from the K first matrix blocks and the j-th matrix block from the K second matrix blocks, where j is a positive integer;

[0022] The product of the fifth matrix and its transpose is used to determine the sixth matrix;

[0023] Based on the values ​​on the diagonal of the sixth matrix, determine the distance between each point corresponding to the j-th first matrix block and the (i-1)-th sampling point.

[0024] In some embodiments, the first matrix and the second matrix are divided into blocks to obtain K first matrix blocks corresponding to the first matrix and K second matrix blocks corresponding to the second matrix, including:

[0025] Based on at least one of the following factors in the point cloud data processing system: the number of computing units, the batch size of the computing units, and the communication bandwidth between the computing units, the first matrix and the second matrix are divided into blocks to obtain the K first matrix blocks and the K second matrix blocks.

[0026] By utilizing the parallel computing capabilities of the point cloud data processing system, the first and second matrices are divided into blocks, thereby performing sampling at the matrix block level and further improving computational efficiency.

[0027] In some embodiments, determining the distance between each of the plurality of first points and the first sampling point set based on the distance between each of the plurality of first points and the (i-1)th sampling point, and the distance between each of the plurality of unsampled second points in the point cloud data obtained when sampling the (i-1)th sampling point and the second sampling point set, includes:

[0028] If the distance between the target point among the plurality of first points and the (i-1)th sampling point is less than the distance between the target point among the plurality of second points and the set of second sampling points, then the distance between the target point among the plurality of first points and the (i-1)th sampling point is determined as the distance between the target point and the set of first sampling points; or,

[0029] If the distance between the target point and the (i-1)th sampling point is greater than or equal to the distance between the target point and the second set of sampling points, the distance between the target point and the second set of sampling points is determined as the distance between the target point and the first set of sampling points.

[0030] In some embodiments, a third set of sampling points containing the i-th sampling point is input into a trained point cloud super-resolution network, and the point cloud super-resolution network outputs target point cloud data that meets the required number of points. The target point cloud data is used to train an artificial intelligence (AI) model. The loss function of the point cloud super-resolution network includes at least one of root mean square error, chamfer distance, and repulsion loss.

[0031] It should be understood that since the FPS algorithm iterates through the number of points during the calculation process, the more points there are, the greater the computational load; however, the more points there are, the more accurately the shape can be represented, and the mesh generated based on the point cloud will also be more accurate. Therefore, by processing the sampled point set through a point cloud super-resolution network to output the target point cloud data, the main shape and structural features of the original point cloud can be preserved in the key areas of the point cloud.

[0032] In a second aspect, a point cloud data processing system is provided, the system comprising at least one functional module for implementing the point cloud data processing method provided by the first aspect or any possible implementation thereof.

[0033] Thirdly, this application provides a computer program product for implementing the point cloud data processing method provided by the first aspect or any possible implementation thereof.

[0034] Fourthly, this application provides a computer-readable storage medium for storing at least one piece of program code for implementing the point cloud data processing method provided by the first aspect or any possible implementation thereof. This storage medium includes, but is not limited to, volatile memory, such as random access memory, and non-volatile memory, such as flash memory, hard disk drive (HDD), and solid-state drive (SSD). Attached Figure Description

[0035] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0036] Figure 2 This is a schematic diagram of the hardware structure of a computing device provided in an embodiment of this application;

[0037] Figure 3 This is a schematic diagram illustrating the principle of a point cloud data processing method provided in an embodiment of this application;

[0038] Figure 4 This is a schematic diagram of a matrix operation provided in an embodiment of this application;

[0039] Figure 5 This is a schematic diagram illustrating the determination of sampling points provided in an embodiment of this application;

[0040] Figure 6 This is a schematic diagram of a cache space in a point cloud data processing system provided in an embodiment of this application;

[0041] Figure 7 This is a flowchart of a point cloud data processing method provided in an embodiment of this application;

[0042] Figure 8 This is a schematic diagram illustrating the use of a point cloud super-resolution network to output target point cloud data, as provided in an embodiment of this application.

[0043] Figure 9 This is a flowchart of another point cloud data processing method provided in the embodiments of this application;

[0044] Figure 10 This is a schematic diagram of a point cloud data processing method provided in an embodiment of this application;

[0045] Figure 11 This is a schematic diagram of the structure of a point cloud data processing system provided in an embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings. It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the point cloud data and artificial intelligence models involved in this application were obtained under fully authorized conditions.

[0047] To facilitate understanding, the key terms and concepts involved in this application will be explained below.

[0048] Point cloud data, also known as 3D point cloud or simply point cloud, is a discrete dataset composed of a series of 3D coordinate points used to describe the geometric features of an object, such as its surface shape, spatial location, and size. Each point's data typically includes 3D coordinate information, and in some scenarios, it may also include attribute information such as color and normals. Point cloud data is a common input format in deep learning tasks such as 3D object recognition and scene reconstruction.

[0049] The farthest point sampling (FPS) algorithm is a sampling algorithm for point cloud data. Its core principle is to iteratively select the point farthest from the selected point set in the point cloud data to select a representative and evenly distributed point set from a large number of points. For example, the FPS algorithm includes the following steps: randomly select a point from the given point cloud data as the first sampling point and add it to the selected point set; calculate the distance from the unselected point to all points in the selected point set, take the shortest distance from each unselected point to all points in the selected point set (take this shortest distance as the distance from each unselected point to the selected point set), select the maximum value among these shortest distances, and the unselected point corresponding to this maximum value is the next sampling point and added to the selected point set; repeat the above iterative steps until the number of points in the selected point set reaches a preset number, and the sampling ends.

[0050] Artificial intelligence (AI) models are a class of mathematical algorithm models that use machine learning concepts to solve practical problems. Typically, AI models include a large number of parameters and calculation formulas (or calculation rules).

[0051] The application scenarios and implementation environment of this application are described below.

[0052] This application applies to scenarios where the FPS algorithm is used to process point cloud data. For example, in 3D reconstruction scenarios, the original point cloud data is massive. Sampling using the FPS algorithm can obtain a representative subset of the point cloud, reducing the data volume while preserving the main features of the original point cloud, facilitating subsequent processing and model building, and improving reconstruction efficiency. As another example, in target recognition and classification scenarios, the point cloud sampled by the FPS algorithm can serve as a feature representation of the target. These sampling points can capture key information such as the target's shape and structure. Machine learning or deep learning models can be trained based on these features to learn the feature patterns of different targets, thereby classifying new point cloud data and determining what kind of target object it belongs to, and so on.

[0053] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application. For example... Figure 1 As shown, the implementation environment includes a point cloud data processing system, which is implemented through a terminal 101 and / or a server 102. The terminal 101 is directly or indirectly connected to the server 102 through a wireless network or a wired network.

[0054] Terminal 101 is used to provide point cloud data acquisition functions, such as collecting point cloud data using devices like LiDAR or depth cameras. Terminal 101 is also used to provide preprocessing functions for the point cloud data, such as removing noise points from the acquired raw point cloud data and performing data format conversion. In some embodiments, terminal 101 is also used to provide user interaction functions, such as providing a user interface to receive user input commands, such as selecting the range of point cloud data to be processed by the FPS algorithm and setting parameters such as the number of sampling points. Simultaneously, the processing results are visualized and presented to the user, allowing the user to intuitively understand the effect of the point cloud data after FPS algorithm processing. Indicatively, terminal 101 includes, but is not limited to, at least one of smartphones, smart cars, game consoles, desktop computers, augmented reality terminals, tablet computers, e-book readers, and laptop computers.

[0055] Server 102 can provide computing power to terminal 101. For example, server 102 receives point cloud data sent by terminal 101 and samples the point cloud data using an FPS algorithm. In addition, server 102 also provides data storage and management functions, such as storing point cloud data and related metadata, including raw point cloud data and point cloud data processed by the FPS algorithm. Simultaneously, it effectively manages this data, such as data backup and retrieval. Server 102 can be, for example, a standalone physical server, a server cluster consisting of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud data centers, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0056] Taking server 102 as an example of a cloud server, a cloud server, also known as a cloud computing platform, refers to a service based on hardware and software resources that provides computing, networking, and storage capabilities. Through the network "cloud," massive amounts of data are processed and analyzed remotely before being returned to the user. It features large scale, distributed architecture, virtualization, high availability, scalability, on-demand service, and security. Cloud servers can achieve rapid deployment and release of configurable computing resources with relatively low management costs or low interaction complexity between users and service providers. In some embodiments, a cloud server can also be a virtual machine instance, a container instance, etc., but this application is not limited to these.

[0057] In this embodiment, the point cloud data processing system is used to implement a sampling function for point cloud data, that is, to execute the point cloud data processing method involved in the following method embodiments. Both terminal 101 and server 102 can independently implement the functions of the point cloud data processing system. In some embodiments, terminal 101 can also cooperate with server 102 to implement the functions of the point cloud data processing system. For example, terminal 102 converts the acquired point cloud data into a matrix and sends it to server 102, which then performs matrix operations on the received matrix to achieve sampling. It should be understood that terminal 101 can refer to one of multiple terminals or a collection of multiple terminals; server 102 can refer to one of multiple servers or a collection of multiple servers. This embodiment does not limit the number or type of each device in the implementation environment.

[0058] In some embodiments, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. Networks include, but are not limited to, data center networks, storage area networks (SANs), local area networks (LANs), metropolitan area networks (MANs), wide area networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some implementations, technologies and / or formats, including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Sockets Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or part of the link. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0059] The hardware structure of the terminal and server in the above implementation environment is described below.

[0060] This application provides a computing device that can be configured as the aforementioned terminal, an independent physical server, or any computing node in a cloud data center, etc. Accordingly, the point cloud data processing method provided in this application can be executed by a single computing device, or by a cluster of computing devices (such as a cloud data center). Illustratively, refer to... Figure 2 , Figure 2 This is a schematic diagram of the hardware structure of a computing device provided in an embodiment of this application. Figure 2 As shown, the computing device 200 includes a memory 201, a processor 202, a communication interface 203, and a bus 204. The memory 201, processor 202, and communication interface 203 are interconnected via the bus 204.

[0061] The memory 201 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or it may be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Illustratively, the memory 201 is used to store at least one piece of program code. When the program code stored in the memory 201 is executed by the processor 202, the processor 202 is used to perform the steps performed by the point cloud data processing system in the following method embodiments.

[0062] The processor 202 can be a network processor (NP), a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or an integrated circuit used to control the execution of the program in this application. The processor 202 can be a single-core processor or a multi-core processor. The number of processors 202 can be one or more. For example, depending on the actual needs of point cloud data processing, the computing device 200 may include multiple processors 202, each of which may include one or more processor cores. These multiple processors 202 may include CPUs, GPUs, neural network processing units (NPUs), intelligent processing units (IPUs), tensor processing units (TPUs), domain-specific architecture (DSA) chips, and so on, and are not limited to these. Among them, a GPU, also known as a display core, visual processor, or display chip, is a microprocessor specifically designed for image processing. The NPU simulates human neurons and synapses at the circuit layer and directly processes large-scale neurons and synapses using a deep learning instruction set, with one instruction completing the processing of a group of neurons.

[0063] In this application, the processor 202 possesses matrix computation and vector computation capabilities. For example, the processor 202 includes scalar units, vector units, and matrix units. The scalar units are used to process single data elements, performing basic arithmetic operations (such as addition, subtraction, multiplication, and division) and logical operations (such as AND, OR, and NOT). The vector units are used to perform the same operation on multiple data elements (vectors), achieving parallel data processing. The matrix units are used to perform matrix operations, such as matrix multiplication, matrix addition, matrix subtraction, and matrix decomposition.

[0064] The communication interface 203 uses a transceiver module, such as a transceiver, to enable communication between the computing device 200 and other devices or communication networks. For example, data can be acquired through the communication interface 203.

[0065] The memory 201 and the processor 202 can be set separately or integrated together.

[0066] Bus 204 may include a pathway for transmitting information between various components of computing device 200 (e.g., memory 201, processor 202, communication interface 203).

[0067] For ease of understanding, please refer to the following: Figures 3 to 6 The principle of the point cloud data processing method provided in this application is introduced.

[0068] As described above, the core principle of the FPS algorithm is to iteratively select the point farthest from the selected point set in the point cloud data, thereby selecting a representative and evenly distributed point set from a large number of points. Related techniques require calculating the distance from each unselected point to all points in the selected point set during each sampling, resulting in significant computational overhead and poor computational efficiency.

[0069] Based on this, this application provides a point cloud data processing method based on matrix / vector operations. When sampling point cloud data, a point cloud matrix and a sampling point matrix of the same size are constructed, and the distance between each unsampled point and the set of sampling points is recorded in each sampling. This means that in each sampling, only the distance between the newly added sampling points obtained in the previous sampling and each unsampled point needs to be calculated, without having to calculate the distance between all sampling points and each unsampled point, thereby effectively saving computational overhead and improving computational efficiency.

[0070] The following example uses point cloud data consisting of N points and M sampling points (N and M are both positive integers, with N being greater than M) to illustrate the point cloud data processing method provided in this application.

[0071] Figure 3 This is a schematic diagram illustrating the principle of a point cloud data processing method provided in an embodiment of this application. Figure 3As shown, for point cloud data containing N points, the point cloud data is matrixed to obtain a point cloud matrix. Based on the parallel computing capability of the point cloud data processing system, the point cloud matrix is ​​divided into K blocks, where K is a positive integer. The point cloud data processing system is implemented, for example, through a server in the aforementioned implementation environment. The parallel computing capability of the point cloud data processing system can be determined based on at least one of the following: the number of computing units in the system, the batch size of the computing units, and the communication bandwidth between computing units. This application does not impose any limitations on these factors. For example, if the point cloud data includes 15 points, and each point's attributes include its x-axis coordinate, y-axis coordinate, and z-axis coordinate (i.e., each point has 3 attributes), then the point cloud matrix is ​​a 15x3 matrix. The value of each row in the point cloud matrix indicates the attribute of each point. Of course, this is only an example; the information represented by the rows and columns of the point cloud matrix can be interchanged. That is, the value of each column in the point cloud matrix indicates the attribute of each point, and the number of attributes for each point can also be greater. This application does not impose any limitations on this. Furthermore, taking a point cloud data processing system with a batch size of 5 and a number of 3 computing units as an example, the point cloud matrix is ​​divided into three 5x3 matrix blocks, and each matrix block is stored in the cache space of the corresponding computing unit. For example, matrix block 1 is stored in the cache space of computing unit 1, matrix block 2 in the cache space of computing unit 2, and matrix block 3 in the cache space of computing unit 3. It should be noted that the block division method of the point cloud matrix here is only an example and does not constitute a limitation of this application. In practical applications, it can be flexibly adjusted according to the parallel computing capabilities of the point cloud data processing system. In some scenarios, the point cloud matrix may not be divided into blocks, for example, when the size of the point cloud matrix is ​​smaller than a preset size. This application does not limit this.

[0072] Taking the first sampling as an example, a point is randomly selected from the point cloud data as the sampling point. A sampling point matrix is ​​constructed for this sampling point. This sampling point matrix has the same size as the point cloud matrix. Following the same processing method as the point cloud matrix, the sampling point matrix is ​​divided into blocks, resulting in K sampling point matrix blocks. For example, if the point cloud matrix is ​​a 15x3 matrix where each row's value indicates the attribute of each point, the sampling point matrix is ​​also a 15x3 matrix where each row's value indicates the attribute of the sampling point; that is, each row in the sampling point matrix has the same value. In some scenarios, when the point cloud matrix is ​​not divided into blocks, the sampling point matrix is ​​similarly not divided into blocks.

[0073] In some embodiments, after determining a sampling point, the value corresponding to that sampling point in the point cloud matrix is ​​deleted. For example, if the point cloud data includes N points, after matrixing, a point cloud matrix with N rows is obtained. After obtaining the first sampling point in the first sampling, the value corresponding to that first sampling point in the point cloud matrix is ​​deleted, resulting in a point cloud matrix with N-1 rows. Correspondingly, the sampling point matrix includes N-1 rows. Similarly, after obtaining the second sampling point in the second sampling, the value corresponding to that second sampling point in the point cloud matrix is ​​deleted, resulting in a point cloud matrix with N-2 rows. Correspondingly, the sampling point matrix includes N-2 rows, and so on, without further elaboration.

[0074] Next, using matrix blocks as the granularity, matrix / vector operations are performed on the point cloud matrix and the sampling point matrix to obtain the distance between each point indicated by the point cloud matrix and the sampling point. That is, for the j-th point cloud matrix block out of K point cloud matrix blocks, processing is performed on both the j-th point cloud matrix block and the j-th sampling point matrix block out of the K sampling point matrix blocks to obtain the distance between each point indicated by the j-th point cloud matrix block and the sampling point. For example, refer to... Figure 4 , Figure 4 This is a schematic diagram of a matrix operation provided in an embodiment of this application. For example... Figure 4 As shown, taking any point cloud matrix block and sampling point matrix block as an example, the size of the matrix block is 3 × batchsize. First, obtain the difference matrix C obtained by subtracting the point cloud matrix block A and the sampling point matrix block B, that is, AB = C. Next, the difference matrix C and its transpose C are... T The product between them is determined as the result matrix D, which is C·C T =D. Using the following formulas (1) to (3), the values ​​on the diagonal of the resulting matrix D are obtained, which is the distance between each point corresponding to the point cloud matrix block A and the sampling point:

[0075] u = (1, 1, ..., 1) T (1)

[0076]

[0077] In formulas (1) to (3) above, formula (1) defines a column vector u, where each element of column vector u is 1; formula (2) defines a sparse matrix P. (v) Each row has only one element that is 1 and the rest are 0, and v is the index variable; in formula (3), E represents the calculation result and D represents the result matrix D. By performing matrix and vector operations as shown in formulas (1) to (3), the values ​​on the diagonal of the result matrix D with a size of batchsize×batchsize can be obtained, which is to say, a column vector E with a size of batchsize×1 is obtained.

[0078] It should be noted that the above Figure 4 The matrix operation method shown for obtaining the distance between each point and the sampling point is only an example and does not constitute a limitation of this application. In practical applications, other operation methods can also be used to obtain the distance between each point and the sampling point.

[0079] Continue to refer to Figure 3 In the point cloud data processing method provided in this application, during each sampling, the distances between each unsampled point and the sampled point set are stored. This ensures that during each sampling, only the distances between the newly added sampled points obtained in the previous sampling and the unsampled points need to be calculated. The distances between the unsampled points and the sampled point set from the previous sampling are then updated based on the distances calculated in the current sampling round. This yields the distances between the unsampled points and the sampled point set in the current sampling round. The point corresponding to the maximum value of these distances is then determined as the sampled point for the current sampling round, and its index is stored. This process is repeated until the number of sampled points in the sampled point set reaches M, resulting in M ​​sampled points.

[0080] For example, after obtaining the first sampling point, the second sampling is performed. In the current sampling round, after obtaining the distance between each unsampled point and the first sampling point (at this time, since there is only one sampling point in the sampling point set, the distance between each unsampled point and the first sampling point is also the distance between each unsampled point and the sampling point set), the distance between each unsampled point and the sampling point is stored (that is, the distance between each unsampled point and the sampling point set is stored), and the point with the largest distance from the first sampling point is determined as the second sampling point. Next, a third sampling is performed. In the current sampling round, after obtaining the distance between each unsampled point and the second sampling point, the distance between each unsampled point in the second sampling and the set of sampling points (including the first sampling point) is obtained. The distance between each unsampled point in the second sampling and the set of sampling points is updated based on the distances calculated in the current sampling round; that is, the shortest distance is selected. The distance between the unsampled points in the current sampling round and the set of sampling points is obtained and stored (the set of sampling points includes the first and second sampling points). The point with the largest distance from the set of sampling points is determined as the third sampling point, resulting in a set of sampling points containing three sampling points. It should be understood that the above process is repeated iteratively until the number of sampling points in the set reaches M.

[0081] The above process can be used as an illustration. Figure 5 , Figure 5 This is a schematic diagram illustrating the determination of sampling points provided in an embodiment of this application. For example... Figure 5As shown, taking any matrix block as an example, in the current sampling round, after obtaining the distance between each unsampled point and the sampling points obtained in the previous sampling, the distance between each unsampled point and the set of sampling points in the previous sampling is obtained. By comparing the magnitudes of the distances in the two sampling rounds, the shortest distance is selected as the distance between each unsampled point and the set of sampling points in the current sampling round. After obtaining the distances between each unsampled point and the set of sampling points for K matrix blocks, the point corresponding to the maximum value of these distances is determined as the sampling point in the current sampling round, and the index of this sampling point is stored.

[0082] Furthermore, for the point cloud data processing system that performs the point cloud data processing method, the point cloud data processing method provided in this application is supported by storing the matrix blocks corresponding to the point cloud data, the shortest distance from each unsampled point to all points in the selected point set (i.e., the distance between each unsampled point and the sampled point set), and the index of each sampled point in the sampled point set (i.e., the index of the point farthest from the sampled point set) in the cache space of the point cloud data processing system. Indicatively, the information stored in the cache space of the point cloud data processing system is as follows: Figure 6 As shown, Figure 6 This is a schematic diagram of a cache space in a point cloud data processing system provided in an embodiment of this application.

[0083] In combination with the above Figures 3 to 6 As shown, during the i-th sampling of the point cloud data, the point cloud matrix composed of the unsampled points and the sampling point matrix composed of the (i-1)-th sampled point obtained in the (i-1)-th sampling are acquired. By constructing point cloud matrices and sampling point matrices of the same size, the distance between the unsampled points and the (i-1)-th sampled point is obtained. Based on the distance between the unsampled points and the sampled point set in the (i-1)-th sampling, the distance between the current unsampled point and the sampled point set is determined. The point with the largest distance to the sampled point set is then identified as the i-th sampled point. Through this method, in each sampling, only the distances between the sampled points obtained in the previous sampling and the unsampled points need to be calculated, effectively saving computational overhead and improving computational efficiency.

[0084] The following describes the process of the point cloud data processing method provided in this application through several method implementation examples.

[0085] Based on the foregoing Figures 3 to 6 As can be seen from the principle of the scheme shown, the point cloud data processing method provided in this application can utilize the parallel computing capabilities of the point cloud data processing system to divide the point cloud matrix and the sampling point matrix into blocks, thereby performing the operation at the matrix block level and improving computational efficiency. Alternatively, the point cloud matrix and the sampling point matrix can be processed without dividing them into blocks. The following references... Figure 7 The method embodiments shown and Figure 9 The method embodiments shown herein describe the point cloud data processing method provided in this application.

[0086] Figure 7 This is a flowchart of a point cloud data processing method provided in an embodiment of this application. Figure 7 As shown, the method is executed by a point cloud data processing system, which can be implemented through a terminal and / or server in the aforementioned implementation environment. Accordingly, the following method can be executed by a single computing device or by a cluster of computing devices (such as a cloud data center) consisting of multiple computing devices. Schematic, taking the point cloud data processing system performing the i-th sampling of point cloud data as an example, the method includes the following steps 701 to 704.

[0087] 701. The point cloud data processing system obtains a first matrix composed of multiple unsampled first points in the point cloud data and a second matrix composed of the (i-1)th sampling point in the first sampling point set. The first matrix and the second matrix have the same size, and i is a positive integer.

[0088] In this embodiment of the application, taking i as a positive integer greater than 1 as an example, the multiple points that were not sampled when the point cloud data processing system performed the i-th sampling are called multiple first points, the point cloud matrix composed of multiple first points is called the first matrix, the sampling point matrix composed of the i-1 sampling points obtained when the point cloud data processing system performed the (i-1)-th sampling is called the second matrix, and the set of sampling points obtained after the point cloud data processing system performed the (i-1)-th sampling is called the first sampling point set, which includes the i-1 sampling points obtained after the point cloud data processing system performed the (i-1)-th sampling.

[0089] Referring to the above Figures 3 to 6 As illustrated, taking the first matrix as an example where rows indicate the number of multiple first points and columns indicate the number of attributes for each of the multiple first points, the second matrix has the same value in each row, and each row's value indicates the attribute of the (i-1)th sampling point. For instance, if the point cloud data processing system performs the i-th sampling and there are 10 unsampled first points, each with 3 attributes, then the first matrix is ​​a 10-row, 3-column matrix, with each row's value indicating the attribute of each of the 10 points. Correspondingly, the second matrix is ​​also a 10-row, 3-column matrix, with each row's value being the same and indicating the attribute of the (i-1)th sampling point.

[0090] 702. The point cloud data processing system determines the distance between each of the multiple first points and the (i-1)th sampling point based on the first matrix and the second matrix.

[0091] In this embodiment, the point cloud data processing system can process the first matrix and the second matrix through matrix operations to obtain the distance between each of the plurality of first points indicated by the first matrix and the (i-1)th sampling point. Illustratively, in conjunction with the foregoing... Figure 4 As shown in the matrix operation process, the point cloud data processing system obtains the third matrix obtained by subtracting the first and second matrices, and determines the fourth matrix by multiplying the third matrix and its transpose. Based on the values ​​on the diagonal of the fourth matrix, the distance between each of the multiple first points and the (i-1)th sampling point is determined. Specifically, the point cloud data processing system can calculate the values ​​on the diagonal of the fourth matrix according to the aforementioned formulas (1) to (3), and thus obtain the distance between each of the multiple first points and the (i-1)th sampling point.

[0092] 703. The point cloud data processing system determines the distance between each of the multiple first points and the first sampling point set based on the distance between each of the multiple first points and the (i-1)th sampling point, and the distance between each of the multiple unsampled second points in the point cloud data obtained when sampling the (i-1)th sampling point and the second sampling point set.

[0093] In this embodiment, the multiple points that were not sampled during the (i-1)th sampling by the point cloud data processing system are referred to as multiple second points, and the set of sampled points obtained after the (i-2)th sampling by the point cloud data processing system is referred to as the second sampled point set. The second sampled point set includes the (i-2) sampled points obtained after the (i-2)th sampling by the point cloud data processing system. Schematic, the distance between each of the multiple first points and the first sampled point set is the minimum distance among the distances from each of the multiple first points to the sampled points in the first sampled point set, and the distance between each of the multiple second points and the second sampled point set is the minimum distance among the distances from each of the multiple second points to the sampled points in the second sampled point set.

[0094] In some embodiments, taking the example where the target point is included in both the plurality of first points and the plurality of second points (i.e., the point that was not sampled in the i-th and i-1-th samplings), the point cloud data processing system performs step 703 as follows: if the distance between the target point and the i-1-th sampling point is less than the distance between the target point and the set of second sampling points among the plurality of second points, the distance between the target point and the i-1-th sampling point among the plurality of first points is determined as the distance between the target point and the set of first sampling points; or, if the distance between the target point and the i-1-th sampling point is greater than or equal to the distance between the target point and the set of second sampling points, the distance between the target point and the set of second sampling points is determined as the distance between the target point and the set of first sampling points. The principle of this process is described above. Figures 3 to 6 The principles behind the proposed scheme will not be elaborated upon here.

[0095] 704. The point cloud data processing system determines the i-th sampling point as the point with the largest distance from the set of first sampling points among multiple first points.

[0096] In this embodiment of the application, the point cloud data processing system determines the maximum value of the distance between multiple first points and the first set of sampling points, and determines the first point corresponding to the maximum value as the i-th sampling point.

[0097] The above steps 701 to 704 describe the process by which the point cloud data processing system performs the i-th sampling of the point cloud data. It should be understood that the point cloud data processing system iterates through steps 701 to 704 until the number of sampling points in the resulting set reaches a preset number. This preset number can be set according to business requirements, and this application does not impose any restrictions on it. For example, after performing step 704, the point cloud data processing system obtains a third set of sampling points containing i sampling points, where i is less than the preset number. Then, following the same process as steps 701 to 704, the point cloud data processing system performs the (i+1)-th sampling, obtaining the (i+1)-th sampling point. At this point, i+1 equals the preset number, and the point cloud data processing system stops iterating, obtaining a set of sampling points containing i+1 sampling points.

[0098] In some embodiments, the point cloud data processing system can also encrypt the sampling point set of the point cloud data through a point cloud super-resolution network to output target point cloud data that meets the required number of points. For example, the target point cloud data can be used to train an AI model, enabling the AI ​​model to have 3D reconstruction capabilities, etc. This application does not limit the type of AI model or its application scenario. Illustratively, taking the example of a third sampling point set containing the i-th sampling point reaching a preset number, the point cloud data processing system inputs the third sampling point set containing the i-th sampling point into the trained point cloud super-resolution network, and outputs target point cloud data that meets the required number of points through the point cloud super-resolution network. The target point cloud data is used to train the artificial intelligence AI model. The loss function of the point cloud super-resolution network includes at least one of root mean square error, chamfer distance, and repulsion loss. Illustratively, when training the point cloud super-resolution network, an appropriate input point cloud and network layer number can be selected based on hardware performance and the number of sampling points in the required sampling point set. A training sample set can be constructed for model training, and the corresponding weights can be saved.

[0099] For example, the loss function of a point cloud super-resolution network is shown in equations (4) to (6):

[0100] Loss = Root Mean Square Error (RMSE)(x, y) + Chamfer Distance + Repulsion Loss (4)

[0101] chamfer distance

[0102] Repulsive force loss

[0103] In formulas (4) to (6) above, the root mean square error, chamfer distance, and repulsion loss measure the difference between the point cloud obtained after super-resolution and the sample point cloud from different perspectives. By minimizing the total loss, the network can be guided to optimize parameters, making the generated point cloud closer to the sample point cloud. Among them, the root mean square error is used to measure the error between the point cloud obtained after super-resolution and the sample point cloud at corresponding point positions, where x represents the point in the super-resolution point cloud and y represents the point in the sample point cloud. The chamfer distance is used to ensure that the super-resolution point cloud is close to the surface of the sample point cloud; the repulsion loss is used to ensure that the distribution of the super-resolution point cloud is as uniform as possible.

[0104] In the chamfer distance shown in formula (5), S1 and S2 represent the point cloud obtained after super-resolution and the sample point cloud, respectively. This represents the sum of the squared distances from all points in S1 to the nearest point in S2. This represents the sum of the squared distances from all points in S2 to the nearest point in S1.

[0105] In the repulsive loss shown in formula (6), Q represents the number of points in the super-resolution point cloud, and a represents the index of a point in the super-resolution point cloud. This is achieved by assigning a value to each point x. a and its neighboring points x a′ The distance relationships between points are calculated and weighted to adjust the distribution of the point cloud and avoid excessive aggregation of points in certain areas; η and w are weighting coefficients; H(a) represents the set of points related to point a.

[0106] refer to Figure 8 , Figure 8 This is a schematic diagram illustrating the use of a point cloud super-resolution network to output target point cloud data, as provided in an embodiment of this application. Figure 8 As shown, the point cloud data processing system randomly divides the sampled point set (which can also be understood as sparse point cloud data) into blocks. This allows the point cloud super-resolution network to focus on the point cloud distribution and features within each block, thus better learning the local geometric features and structural information of the point cloud. For the block-based point cloud, the point cloud super-resolution network extracts features, converting the feature representation of the block-based point cloud into token form. This transforms the point cloud data into a format suitable for deep learning network processing, facilitating subsequent computation and information transmission within the network. Next, the feature-extracted point cloud tokens are input into a fully connected network, outputting encrypted point cloud tokens. These encrypted tokens are then connected to a prediction head module (which is typically a specially designed network layer or sub-network), outputting the super-resolution point cloud coordinates, thus obtaining the target point cloud data. This achieves the super-resolution conversion from sparse point cloud to high-resolution point cloud.

[0107] It should be understood that because the FPS algorithm iterates through the number of points during computation, the more points there are, the greater the computational load. However, a larger number of points allows for a more accurate representation of the shape, and the mesh generated from the point cloud will also be more accurate. Therefore, processing the sampled point set using a point cloud super-resolution network to output target point cloud data can preserve the main shape and structural features of the original point cloud in key areas. Taking a 3D object point cloud as an example, sampling points are often concentrated on important parts such as the object's outline and corners. Based on this, super-resolution can refine and enhance key features, improving the ability of the super-resolution point cloud to represent the object's details.

[0108] In summary, in the point cloud data processing method provided in this application embodiment, when sampling the point cloud data for the i-th time, a first matrix composed of unsampled points in the point cloud data and a second matrix composed of the (i-1)-th sampled points obtained in the (i-1)-th sampling are obtained. The distance between the unsampled points and the (i-1)-th sampled point is obtained by constructing a first matrix and a second matrix of the same size. Based on the distance between the unsampled points and the sampled point set in the (i-1)-th sampling, the distance between the currently unsampled points and the sampled point set is determined. Then, the point with the largest distance to the sampled point set is determined as the i-th sampled point. Through this method, in each sampling, only the distance between the sampled points obtained in the previous sampling and each unsampled point needs to be calculated, thereby effectively saving computational overhead and improving computational efficiency.

[0109] Figure 9 This is a flowchart of another point cloud data processing method provided in the embodiments of this application, such as... Figure 9 As shown, the method is executed by a point cloud data processing system, which can be implemented through a terminal and / or server in the aforementioned implementation environment. Accordingly, the following method can be executed by a single computing device or by a cluster of computing devices (such as a cloud data center) consisting of multiple computing devices. Schematic, taking the point cloud data processing system performing the i-th sampling of point cloud data as an example, the method includes the following steps 901 to 905.

[0110] 901. The point cloud data processing system obtains a first matrix composed of multiple unsampled first points in the point cloud data and a second matrix composed of the (i-1)th sampling point in the first sampling point set. The first matrix and the second matrix have the same size, and i is a positive integer.

[0111] This step is the same as step 701 mentioned above, so it will not be repeated here.

[0112] 902. The point cloud data processing system divides the first matrix and the second matrix into blocks to obtain K first matrix blocks corresponding to the first matrix and K second matrix blocks corresponding to the second matrix, where K is a positive integer.

[0113] In this embodiment, the point cloud data processing system divides the first matrix and the second matrix into K first matrix blocks and K second matrix blocks based on at least one of the following: the number of computing units in the point cloud data processing system, the batch size of the computing units, and the communication bandwidth between the computing units. This process can be referred to the foregoing. Figures 3 to 6 The principles behind the proposed scheme will not be elaborated upon here.

[0114] 903. The point cloud data processing system determines the distance between each point in each of the K first matrix blocks and the (i-1)th sampling point based on the K first matrix blocks and the K second matrix blocks.

[0115] In this embodiment, the point cloud data processing system can utilize parallel computing capabilities to calculate the distance between each point corresponding to each first matrix block and the (i-1)th sampling point through matrix operations, using matrix blocks as the granularity. (Illustratively, in conjunction with the foregoing...) Figure 4 As shown in the matrix operation process, for the j-th first matrix block among the K first matrix blocks, where j is a positive integer, the point cloud data processing system obtains the fifth matrix obtained by subtracting the j-th first matrix block from the j-th matrix block among the K second matrix blocks; the product between the fifth matrix and the transpose of the fifth matrix is ​​determined as the sixth matrix; based on the values ​​on the diagonal of the sixth matrix, the distance between each point corresponding to the j-th first matrix block and the (i-1)-th sampling point is determined. Specifically, the point cloud data processing system can calculate the values ​​on the diagonal of the sixth matrix according to the aforementioned formulas (1) to (3), thereby obtaining the distance between each point corresponding to the j-th first matrix block and the (i-1)-th sampling point.

[0116] It should be understood that steps 902 and 903 above are the process by which the point cloud data processing system determines the distance between each of the multiple first points and the (i-1)th sampling point based on the first matrix and the second matrix. Based on this, the point cloud data processing system executes steps 904 and 905 in the same manner as steps 703 and 704 above.

[0117] 904. The point cloud data processing system determines the distance between each of the multiple first points and the first sampling point set based on the distance between each of the multiple first points and the (i-1)th sampling point, and the distance between each of the multiple unsampled second points in the point cloud data obtained when sampling the (i-1)th sampling point and the second sampling point set.

[0118] This step is the same as step 703 mentioned above, so it will not be repeated here.

[0119] 905. The point cloud data processing system determines the i-th sampling point as the point with the largest distance from the set of first sampling points among multiple first points.

[0120] This step is the same as step 704 above, so it will not be repeated here. In addition, in this embodiment, the point cloud data processing system can also encrypt the sampling point set of the point cloud data through the point cloud super-resolution network to output the target point cloud data that meets the required number of points, which will not be repeated here.

[0121] The above steps 901 to 905 describe the process by which the point cloud data processing system performs the i-th sampling of the point cloud data. It should be understood that the point cloud data processing system iterates through steps 901 to 905 until the number of sampling points in the resulting set reaches a preset number. This preset number can be set according to business needs, and this application does not impose any restrictions on it. For example, after performing step 905, the point cloud data processing system obtains a third set of sampling points containing i sampling points, where i is less than the preset number. Then, following a similar process to steps 901 to 905, the point cloud data processing system performs the (i+1)-th sampling, obtaining the (i+1)-th sampling point. At this point, i+1 equals the preset number, and the point cloud data processing system stops iterating, obtaining a set of sampling points containing i+1 sampling points.

[0122] Indicatively, for reference Figure 10 , Figure 10 This is a schematic diagram of a point cloud data processing method provided in an embodiment of this application. Figure 10 As shown, after acquiring point cloud data, the point cloud data processing system matrixes the point cloud data to obtain a point cloud matrix. Based on the parallel computing capabilities of the point cloud data processing system, the point cloud matrix is ​​divided into blocks, and each matrix block is stored in the cache space of the corresponding computing unit. After using the parallel computing capabilities of the point cloud data processing system to perform matrix / vector operations on the point cloud matrix and the sampling point matrix, a set of sampling points is obtained. The set of sampling points is input into the point cloud super-resolution network, and the target point cloud data that meets the required number of points is output.

[0123] In summary, in the point cloud data processing method provided in this application embodiment, when sampling the point cloud data for the i-th time, a first matrix composed of unsampled points in the point cloud data and a second matrix composed of the (i-1)-th sampled points obtained in the (i-1)-th sampling are acquired. The distance between the unsampled points and the (i-1)-th sampled point is obtained by constructing a first matrix and a second matrix of the same size. Based on the distance between the unsampled points and the sampled point set in the (i-1)-th sampling, the distance between the currently unsampled points and the sampled point set is determined, and the point with the largest distance from the sampled point set is determined as the i-th sampled point. Through this method, only the distance between the sampled points obtained in the previous sampling and each unsampled point needs to be calculated during each sampling, thereby effectively saving computational overhead and improving computational efficiency. Furthermore, in the above process, the parallel computing capability of the point cloud data processing system can be utilized to divide the first and second matrices into blocks, thereby performing sampling at the matrix block granularity, further improving computational efficiency.

[0124] Figure 11 This is a schematic diagram of the structure of a point cloud data processing system provided in an embodiment of this application. Figure 11 As shown, this system can achieve some or all of the functions of the aforementioned point cloud data processing system through software, hardware, or a combination of both. For example... Figure 11 As shown, the system includes an acquisition module 1101, a first determination module 1102, a second determination module 1103, and a third determination module 1104. For example, the acquisition module 1101, the first determination module 1102, the second determination module 1103, and the third determination module 1104 serve as software or hardware functional units. These modules constitute a point cloud data processing device, used to implement some or all of the functions of the aforementioned point cloud data processing system.

[0125] The acquisition module 1101 is used to acquire a first matrix composed of multiple unsampled first points in the point cloud data and a second matrix composed of the (i-1)th sampling point in the first sampling point set. The first matrix and the second matrix have the same size, and i is a positive integer.

[0126] The first determining module 1102 is used to determine the distance between each of the plurality of first points and the (i-1)th sampling point based on the first matrix and the second matrix;

[0127] The second determining module 1103 is used to determine the distance between each of the plurality of first points and the first sampling point set based on the distance between each of the plurality of first points and the (i-1)th sampling point, and the distance between each of the plurality of unsampled second points in the point cloud data obtained when sampling the (i-1)th sampling point and the second sampling point set, wherein the second sampling point set includes i-2 sampling points, the distance between each of the plurality of first points and the first sampling point set is the minimum distance among the distances from each of the plurality of first points to the sampling points in the first sampling point set, and the distance between each of the plurality of second points and the second sampling point set is the minimum distance among the distances from each of the plurality of second points to the sampling points in the second sampling point set;

[0128] The third determining module 1104 is used to determine the i-th sampling point as the point with the largest distance from the first sampling point set among the plurality of first points.

[0129] In some embodiments, the rows of the first matrix are used to indicate the number of the plurality of first points, the columns of the first matrix are used to indicate the number of attributes of each of the plurality of first points, and the values ​​of each row in the second matrix are the same and each row value is used to indicate the attribute of the (i-1)th sampling point.

[0130] In some embodiments, the first determining module 1102 is configured to:

[0131] Obtain the third matrix obtained by subtracting the first matrix from the second matrix;

[0132] The product of the third matrix and its transpose is used to determine the fourth matrix;

[0133] Based on the values ​​on the diagonal of the fourth matrix, determine the distance between each of the plurality of first points and the (i-1)th sampling point.

[0134] In some embodiments, the first determining module 1102 includes:

[0135] The block division unit is used to divide the first matrix and the second matrix into blocks to obtain K first matrix blocks corresponding to the first matrix and K second matrix blocks corresponding to the second matrix, where K is a positive integer;

[0136] The determining unit is configured to determine, based on the K first matrix blocks and the K second matrix blocks, the distance between each point corresponding to each of the K first matrix blocks and the (i-1)th sampling point.

[0137] In some embodiments, the determining unit is configured to:

[0138] Obtain the fifth matrix by subtracting the j-th first matrix block from the K first matrix blocks and the j-th matrix block from the K second matrix blocks, where j is a positive integer;

[0139] The product of the fifth matrix and its transpose is used to determine the sixth matrix;

[0140] Based on the values ​​on the diagonal of the sixth matrix, determine the distance between each point corresponding to the j-th first matrix block and the (i-1)-th sampling point.

[0141] In some embodiments, the segmentation unit is used for:

[0142] Based on at least one of the following factors in the point cloud data processing system: the number of computing units, the batch size of the computing units, and the communication bandwidth between the computing units, the first matrix and the second matrix are divided into blocks to obtain the K first matrix blocks and the K second matrix blocks.

[0143] In some embodiments, the second determining module 1103 is configured to:

[0144] If the distance between the target point among the plurality of first points and the (i-1)th sampling point is less than the distance between the target point among the plurality of second points and the set of second sampling points, then the distance between the target point among the plurality of first points and the (i-1)th sampling point is determined as the distance between the target point and the set of first sampling points; or,

[0145] If the distance between the target point and the (i-1)th sampling point is greater than or equal to the distance between the target point and the second set of sampling points, the distance between the target point and the second set of sampling points is determined as the distance between the target point and the first set of sampling points.

[0146] The system described above, when sampling point cloud data for the i-th time, obtains a first matrix composed of unsampled points and a second matrix composed of the (i-1)-th sampled points obtained in the (i-1)-th sampling. By constructing the first and second matrices of the same size, the distance between the unsampled points and the (i-1)-th sampled point is obtained. Based on the distance between the unsampled points and the sampled point set in the (i-1)-th sampling, the distance between the current unsampled point and the sampled point set is determined. The point with the largest distance to the sampled point set is then identified as the i-th sampled point. This method effectively saves computational overhead and improves computational efficiency by calculating only the distances between the previously sampled points and the unsampled points during each sampling.

[0147] It should be noted that the point cloud data processing system provided in the above embodiments is only illustrated by the division of the above functional modules when processing point cloud data. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above. In addition, the point cloud data processing system and the point cloud data processing method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0148] In this application, the terms "first," "second," etc., are used to distinguish identical or similar items that have substantially the same function and purpose. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor does it limit the quantity or execution order. It should also be understood that although the following description uses the terms "first," "second," etc., to describe various elements, these elements should not be limited by the terms. These terms are merely used to distinguish one element from another. For example, without departing from the various examples described, a first sampling point can be referred to as a second sampling point, and similarly, a second sampling point can be referred to as a first sampling point. Both the first and second sampling points can be sampling points, and in some cases, they can be separate and distinct sampling points.

[0149] In this application, the term "at least one" means one or more, and the term "multiple" means two or more. For example, multiple sampling points means two or more sampling points.

[0150] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0151] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, in the form of program structure information. This program structure information includes one or more program instructions. When these program instructions are loaded and executed on a computing device, the processes or functions according to the embodiments of this application are generated, in whole or in part.

[0152] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0153] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for processing point cloud data, characterized in that, The method includes: Obtain a first matrix composed of multiple unsampled first points in the point cloud data and a second matrix composed of the (i-1)th sampled point in the first sampled point set. The first matrix and the second matrix have the same size, and i is a positive integer. Based on the first matrix and the second matrix, determine the distance between each of the plurality of first points and the (i-1)th sampling point; Based on the distance between each of the plurality of first points and the (i-1)th sampling point, and the distance between each of the plurality of unsampled second points in the point cloud data obtained when sampling the (i-1)th sampling point and the second sampling point set, the distance between each of the plurality of first points and the first sampling point set is determined. The second sampling point set includes i-2 sampling points. The distance between each of the plurality of first points and the first sampling point set is the minimum distance among the distances from each of the plurality of first points to the sampling points in the first sampling point set. Similarly, the distance between each of the plurality of second points and the second sampling point set is the minimum distance among the distances from each of the plurality of second points to the sampling points in the second sampling point set. The point with the largest distance from the first set of sampling points among the plurality of first points is determined as the i-th sampling point.

2. The method of claim 1, wherein, The rows of the first matrix are used to indicate the number of the plurality of first points, and the columns of the first matrix are used to indicate the number of attributes of each of the plurality of first points. The values ​​of each row in the second matrix are the same, and the values ​​of each row are used to indicate the attributes of the (i-1)th sampling point.

3. The method according to claim 1 or 2, characterized in that, Determining the distance between each of the plurality of first points and the (i-1)th sampling point based on the first matrix and the second matrix includes: Obtain the third matrix obtained by subtracting the first matrix from the second matrix; The product of the third matrix and its transpose is used to determine the fourth matrix; Based on the values ​​on the diagonal of the fourth matrix, determine the distance between each of the plurality of first points and the (i-1)th sampling point.

4. The method according to claim 1 or 2, characterized in that, Determining the distance between each of the plurality of first points and the (i-1)th sampling point based on the first matrix and the second matrix includes: Divide the first matrix and the second matrix into blocks to obtain K first matrix blocks corresponding to the first matrix and K second matrix blocks corresponding to the second matrix, where K is a positive integer; Based on the K first matrix blocks and the K second matrix blocks, determine the distance between each point corresponding to each of the K first matrix blocks and the (i-1)th sampling point.

5. The method of claim 4, wherein, The step of determining the distance between each point corresponding to each of the K first matrix blocks and the (i-1)th sampling point based on the K first matrix blocks and the K second matrix blocks includes: Obtain the fifth matrix by subtracting the j-th first matrix block from the K first matrix blocks and the j-th matrix block from the K second matrix blocks, where j is a positive integer; The product of the fifth matrix and its transpose is used to determine the sixth matrix; Based on the values ​​on the diagonal of the sixth matrix, determine the distance between each point corresponding to the j-th first matrix block and the (i-1)-th sampling point.

6. The method according to claim 4 or 5, characterized in that, The step of dividing the first matrix and the second matrix into blocks to obtain K first matrix blocks corresponding to the first matrix and K second matrix blocks corresponding to the second matrix includes: Based on at least one of the following factors in the point cloud data processing system: the number of computing units, the batch size of the computing units, and the communication bandwidth between the computing units, the first matrix and the second matrix are divided into blocks to obtain the K first matrix blocks and the K second matrix blocks.

7. The method according to any one of claims 1 to 6, characterized in that, The step of determining the distance between each of the plurality of first points and the first sampling point set based on the distance between each of the plurality of first points and the (i-1)th sampling point, and the distance between each of the plurality of unsampled second points in the point cloud data obtained when sampling the (i-1)th sampling point and the second sampling point set, includes: If the distance between the target point among the plurality of first points and the (i-1)th sampling point is less than the distance between the target point among the plurality of second points and the set of second sampling points, then the distance between the target point among the plurality of first points and the (i-1)th sampling point is determined as the distance between the target point and the set of first sampling points; or, If the distance between the target point and the (i-1)th sampling point is greater than or equal to the distance between the target point and the second set of sampling points, the distance between the target point and the second set of sampling points is determined as the distance between the target point and the first set of sampling points.

8. A point cloud data processing system, characterized by, The system includes: The acquisition module is used to acquire a first matrix composed of multiple unsampled first points in the point cloud data and a second matrix composed of the (i-1)th sampling point in the first sampling point set. The first matrix and the second matrix have the same size, and i is a positive integer. The first determining module is used to determine the distance between each of the plurality of first points and the (i-1)th sampling point based on the first matrix and the second matrix; The second determining module is used to determine the distance between each of the plurality of first points and the first sampling point set based on the distance between each of the plurality of first points and the (i-1)th sampling point, and the distance between each of the plurality of unsampled second points in the point cloud data obtained when sampling the (i-1)th sampling point and the second sampling point set, wherein the second sampling point set includes i-2 sampling points, the distance between each of the plurality of first points and the first sampling point set is the minimum distance among the distances from each of the plurality of first points to the sampling points in the first sampling point set, and the distance between each of the plurality of second points and the second sampling point set is the minimum distance among the distances from each of the plurality of second points to the sampling points in the second sampling point set; The third determining module is used to determine the i-th sampling point as the point with the largest distance from the first set of sampling points among the plurality of first points.

9. The system according to claim 8, characterized in that, The rows of the first matrix are used to indicate the number of the plurality of first points, and the columns of the first matrix are used to indicate the number of attributes of each of the plurality of first points. The values ​​of each row in the second matrix are the same, and the values ​​of each row are used to indicate the attributes of the (i-1)th sampling point.

10. The system according to claim 8 or 9, characterized in that, The first determining module is used for: Obtain the third matrix obtained by subtracting the first matrix from the second matrix; The product of the third matrix and its transpose is used to determine the fourth matrix; Based on the values ​​on the diagonal of the fourth matrix, determine the distance between each of the plurality of first points and the (i-1)th sampling point.

11. The system of claim 8 or 9, wherein, The first determining module includes: The block division unit is used to divide the first matrix and the second matrix into blocks to obtain K first matrix blocks corresponding to the first matrix and K second matrix blocks corresponding to the second matrix, where K is a positive integer; The determining unit is configured to determine, based on the K first matrix blocks and the K second matrix blocks, the distance between each point corresponding to each of the K first matrix blocks and the (i-1)th sampling point.

12. The system of claim 11, wherein, The determining unit is used for: Obtain the fifth matrix by subtracting the j-th first matrix block from the K first matrix blocks and the j-th matrix block from the K second matrix blocks, where j is a positive integer; The product of the fifth matrix and its transpose is used to determine the sixth matrix; Based on the values ​​on the diagonal of the sixth matrix, determine the distance between each point corresponding to the j-th first matrix block and the (i-1)-th sampling point.

13. The system of claim 11 or 12, wherein, The block unit is used for: Based on at least one of the following factors in the point cloud data processing system: the number of computing units, the batch size of the computing units, and the communication bandwidth between the computing units, the first matrix and the second matrix are divided into blocks to obtain the K first matrix blocks and the K second matrix blocks.

14. The system of any one of claims 8 to 13, wherein, The second determining module is used for: If the distance between the target point among the plurality of first points and the (i-1)th sampling point is less than the distance between the target point among the plurality of second points and the set of second sampling points, then the distance between the target point among the plurality of first points and the (i-1)th sampling point is determined as the distance between the target point and the set of first sampling points; or, If the distance between the target point and the (i-1)th sampling point is greater than or equal to the distance between the target point and the second set of sampling points, the distance between the target point and the second set of sampling points is determined as the distance between the target point and the first set of sampling points.

15. A computer program product, characterised in that, The computer program product is used to implement the point cloud data processing method as described in any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store at least one piece of program code, which is used to implement the point cloud data processing method as described in any one of claims 1 to 7.

Images