Point cloud normal vector calculation method and device, computer device and storage medium

Abstract

A point cloud normal vector calculation method comprises: acquiring point cloud data; calling a pre-trained semantic segmentation model, inputting the point cloud data into the semantic segmentation model, performing semantic segmentation on the point cloud data by the semantic segmentation model to obtain a semantic segmentation result; clustering the point cloud data according to the semantic segmentation result to obtain a clustering result; selecting neighborhood points corresponding to each point in the point cloud data according to the clustering result; and calculating a normal vector corresponding to each point according to the neighborhood points corresponding to each point.

Description

TECHNICAL FIELD

[0001] The present application relates to a point cloud normal vector calculation method and device, computer equipment and a storage medium. BACKGROUND

[0002] With the advancement of sensor technology, it is easier and easier to obtain point clouds, and point cloud normal vectors, as the most basic point cloud features, play a crucial role in many point cloud processing algorithms. Point cloud normal vectors can provide information related to the surface of the point cloud and are widely used in three-dimensional reconstruction, point cloud rendering, augmented reality, virtual reality, point cloud plane estimation and other fields.

[0003] The traditional method is to calculate the point cloud normal vector by calculating the eigenvalue and eigenvector of the covariance matrix of a point and its neighborhood points. However, the traditional method is easily disturbed by adjacent objects in the process of selecting neighborhood points, resulting in low accuracy of the calculated point cloud normal vector. SUMMARY

[0004] According to various embodiments disclosed in the present application, a point cloud normal vector calculation method, device, computer equipment and storage medium capable of improving the calculation accuracy of point cloud normal vectors are provided.

[0005] A point cloud normal vector calculation method comprises:

[0006] obtaining point cloud data;

[0007] calling a pre-trained semantic segmentation model, inputting the point cloud data into the semantic segmentation model, performing semantic segmentation on the point cloud data by the semantic segmentation model, and obtaining a semantic segmentation result;

[0008] clustering the point cloud data according to the semantic segmentation result to obtain a clustering result;

[0009] selecting neighborhood points corresponding to each point in the point cloud data according to the clustering result; and

[0010] calculating the normal vector corresponding to each point according to the neighborhood points corresponding to each point.

[0011] A point cloud normal vector calculation device comprises:

[0012] an acquisition module configured to acquire point cloud data;

[0013] a segmentation module configured to call a pre-trained semantic segmentation model, input the point cloud data into the semantic segmentation model, perform semantic segmentation on the point cloud data by the semantic segmentation model, and obtain a semantic segmentation result;

[0014] a clustering module configured to cluster the point cloud data according to the semantic segmentation result, to obtain a clustering result;

[0015] a selecting module configured to select, according to the clustering result, neighborhood points corresponding to each point in the point cloud data; and

[0016] a calculating module configured to calculate a normal vector corresponding to each point according to the neighborhood points corresponding to the point.

[0017] A computer device includes a memory and one or more processors, the memory stores computer readable instructions, and the computer readable instructions are executed by the processors to cause the one or more processors to perform the following steps:

[0018] obtain point cloud data;

[0019] invoke a pre-trained semantic segmentation model, input the point cloud data into the semantic segmentation model, perform semantic segmentation on the point cloud data by the semantic segmentation model, and obtain a semantic segmentation result;

[0020] cluster the point cloud data according to the semantic segmentation result, to obtain a clustering result;

[0021] select, according to the clustering result, neighborhood points corresponding to each point in the point cloud data; and

[0022] calculate a normal vector corresponding to each point according to the neighborhood points corresponding to the point.

[0023] One or more non-volatile computer readable storage media having computer readable instructions stored thereon, the computer readable instructions, when executed by one or more processors, cause the one or more processors to perform the following steps:

[0024] obtain point cloud data;

[0025] invoke a pre-trained semantic segmentation model, input the point cloud data into the semantic segmentation model, perform semantic segmentation on the point cloud data by the semantic segmentation model, and obtain a semantic segmentation result;

[0026] cluster the point cloud data according to the semantic segmentation result, to obtain a clustering result;

[0027] select, according to the clustering result, neighborhood points corresponding to each point in the point cloud data; and

[0028] calculate a normal vector corresponding to each point according to the neighborhood points corresponding to the point.

[0029] Details of one or more embodiments of the present application are presented in the following drawings and description. Other features and advantages of the present application will become apparent from the description, drawings, and claims. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is an application environment diagram of the point cloud normal vector calculation method in one or more embodiments.

[0032] Figure 2 This is a flowchart illustrating the point cloud normal vector calculation method in one or more embodiments.

[0033] Figure 3 This is a flowchart illustrating the steps of clustering point cloud data based on semantic segmentation results in one or more embodiments to obtain clustering results.

[0034] Figure 4 This is a flowchart illustrating the steps for selecting neighborhood points corresponding to each point in point cloud data based on clustering results in one or more embodiments.

[0035] Figure 5 This is a flowchart illustrating the point cloud normal vector calculation method in another embodiment.

[0036] Figure 6 This is a block diagram of a point cloud normal vector calculation device in one or more embodiments.

[0037] Figure 7 This is a block diagram of a computer device in one or more embodiments.

[0038] Figure 8 A block diagram of a computer device in another embodiment. Detailed Implementation

[0039] To make the technical solutions and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0040] The point cloud normal vector calculation method provided in this application can be applied to computer devices, which can be terminals or servers. It is understood that the point cloud normal vector calculation method provided in this application can be applied to terminals, servers, or systems including both terminals and servers, and can be implemented through interaction between the terminal and the server.

[0041] The point cloud normal vector calculation method provided in the application can be applied in an application environment as shown in Figure 1 The point cloud sensor 102 sends the collected point cloud data to the server 104. The point cloud sensor can be any one of a laser radar, a laser scanner, a camera, and the like, which are sensors for collecting point cloud data. After obtaining the point cloud data, the server 104 calls a pre-trained semantic segmentation model, inputs the point cloud data into the semantic segmentation model, performs semantic segmentation on the point cloud data through the semantic segmentation model, and obtains a semantic segmentation result. Thus, the server 104 clusters the point cloud data according to the semantic segmentation result and obtains a clustering result. The computer device 104 selects the neighborhood points corresponding to each point in the point cloud data according to the clustering result, and then calculates the normal vector corresponding to each point according to the neighborhood points corresponding to each point.

[0042] In one of the embodiments, as shown in Figure 2 A point cloud normal vector calculation method is provided, which is taken as an example to illustrate the application of the method to a computer device, which can be a terminal or a server, and includes the following steps:

[0043] Step 202: Obtain point cloud data.

[0044] The point cloud data can be data recorded in the form of point cloud by a point cloud sensor on the scanned surrounding environment information. The point cloud data is point cloud data collected by the point cloud sensor within the visual range. The visual range of different point cloud sensors can be different. The point cloud data can specifically include three-dimensional coordinates (x, y, z), laser reflection intensity (Intensity), color information (RGB), and the like of each point. The three-dimensional coordinates are used to represent the position information of the surface of an object in the surrounding environment. For example, the three-dimensional coordinates can be the coordinates of a point in a Cartesian coordinate system, specifically including the horizontal axis coordinate, the vertical axis coordinate, and the vertical axis coordinate of the point in the Cartesian coordinate system. The Cartesian coordinate system is a three-dimensional space coordinate system established with the point cloud sensor as the origin, and the three-dimensional space coordinate system includes a horizontal axis (x-axis), a vertical axis (y-axis), and a vertical axis (z-axis). The three-dimensional space coordinate system established with the point cloud sensor as the origin satisfies the right-hand rule.

[0045] Specifically, the point cloud sensor obtains the corresponding point cloud data by scanning the current environment, and the vehicle-mounted sensor transmits the collected point cloud data to the computer device. For example, the vehicle-mounted sensor can be a laser radar.

[0046] Step 204: Call a pre-trained semantic segmentation model, input the point cloud data into the semantic segmentation model, perform semantic segmentation on the point cloud data through the semantic segmentation model, and obtain a semantic segmentation result.

[0047] The server stores a pre-trained semantic segmentation model. The semantic segmentation model is trained by a large amount of sample data. In an embodiment, the server can train the deep learning model according to a large amount of manually annotated data carrying semantic class labels to obtain the semantic segmentation model. Training the deep learning model by the data carrying semantic class labels can improve the accuracy of semantic segmentation. For example, the semantic segmentation model can be any one of a FCN (Fully Convolutional Networks) model, a conditional random field (CRF) model, a pointnet, a pointnet++, and the like. Semantic segmentation refers to dividing each point of the point cloud into a corresponding class, i.e., giving a semantic class corresponding to each point. For example, the semantic classes can include people, vehicles, roads, buildings, vegetation, guardrails, and the like. After obtaining the point cloud data, the server can perform semantic segmentation on the point cloud data by using the pre-trained semantic segmentation model, and output a semantic segmentation result. The semantic segmentation result includes the semantic classes corresponding to the points in the point cloud data.

[0048] In an embodiment, the computer device can also use any one of an edge-based, region growing-based, attribute-based, and graph-based segmentation algorithm to perform semantic segmentation on the point cloud data.

[0049] In one of the embodiments, the computer device can further pre-process the point cloud data before calling the pre-trained semantic segmentation model. The pre-processing can include clutter point removal and ground point filtering. Since there can be a large number of clutter points in the point cloud data. The computer device can remove a large number of clutter points by performing a straight-through filtering process on the point cloud data. Further, the computer device can further perform ground point filtering on the point cloud data after removing the clutter points. The ground point filtering refers to filtering out the ground points in the point cloud data, and the remaining points are non-ground points. The computer device can identify the ground points in the point cloud data after removing the clutter points by performing ground segmentation on the point cloud data after removing the clutter points, and filter out the ground points to obtain non-ground point cloud data. Specifically, the computer device can first divide the point cloud region where the point cloud data after removing the clutter points is located into a plurality of sub-regions. The point cloud region refers to the three-dimensional data space where the point cloud data after removing the clutter points is located. The division manner can be grid division on the point cloud data after removing the clutter points, that is, dividing the horizontal plane formed by the x-axis direction and the y-axis direction of the point cloud region. The grid division manner can be equal division or random division. For example, for a point cloud sensor with a visual range of 100 m, the horizontal plane size of the scanning region of the point cloud sensor is 100 m*100 m, and the point cloud region can be equally divided into 10*10 horizontal cells. For each sub-region obtained by division, the computer device can estimate the corresponding ground surface according to the preset plane equation by using the least squares method, thereby obtaining the ground surface corresponding to each sub-region. For example, the preset plane equation can be a cubic equation. The ground surface corresponding to each sub-region is embodied in the form of a cubic equation. The computer device traverses the point coordinates in the corresponding sub-region in the equation corresponding to the ground surface, calculates the distance between each point and the corresponding ground surface, and determines the point as a ground point when the distance is less than a threshold value. When the distance is greater than or equal to the threshold value, the point is determined as a non-ground point. The threshold value refers to the distance threshold value for determining whether the point is a ground point. The computer device further filters the ground points to obtain non-ground point cloud data. The computer device performs ground filtering on the point cloud data after removing the clutter points, and can obtain effective point cloud data, that is, non-ground point cloud data. Further, the computer device can perform semantic segmentation on the non-ground point cloud data.

[0050] Step 206, clustering the point cloud data according to the semantic segmentation result to obtain a clustering result.

[0051] Since the semantic segmentation result includes the semantic category corresponding to each point in the point cloud data, the computer device clusters the point cloud data according to the semantic category corresponding to each point in the semantic segmentation result to obtain a clustering result, which includes a plurality of clustering categories and the clustering category to which each point in the point cloud data belongs. By clustering the point cloud data according to the semantic segmentation result, the same points in the point cloud data can be classified into the same category, such as classifying a certain person or a certain vehicle in the point cloud data into a category, thereby effectively avoiding the problem of clustering points of different semantic categories together.

[0052] In one of the embodiments, the computer device can use any one of a method based on connected component analysis, a K-means clustering method, Euclidean clustering, a DBSCAN (Density-Based Spatial Clustering of Applications with Noise) density clustering algorithm, and the like to cluster the point cloud data.

[0053] In step 208, the neighborhood points corresponding to each point in the point cloud data are selected according to the clustering result.

[0054] The computer device can select the neighborhood points of each point in the point cloud data and calculate the normal vector of the corresponding point by using the selected neighborhood points. The computer device can select the neighborhood points by using the clustering result. Specifically, the clustering result includes a plurality of clustering categories and the clustering category to which each point in the point cloud data belongs. For a certain point in the point cloud data, the computer device can determine the clustering category corresponding to the point according to the clustering result, and select points belonging to the same clustering category around the point to obtain the neighborhood points corresponding to the point. The computer device can select the neighborhood points of all points in the point cloud data by traversing all points in the point cloud data and selecting the neighborhood points in the above manner. The neighborhood points selected by the computer device can be a fixed number or points within a fixed spherical radius. The points in the same clustering category belong to the same semantic category. Therefore, the computer device can select the neighborhood points in the same clustering category, thereby avoiding the interference of adjacent objects, i.e., avoiding the problem that the neighborhood points of object A are on object B when objects A and B are close to each other, thereby improving the accuracy of neighborhood point selection and the quality of neighborhood points.

[0055] In step 210, the normal vector corresponding to each point is calculated according to the neighborhood points corresponding to the point.

[0056] The normal vector corresponding to the point pair can become a point cloud normal vector, which refers to a vector perpendicular to the point cloud surface. After selecting the neighborhood points corresponding to each point, the computer device can perform principal component analysis on the neighborhood points to calculate the normal vector corresponding to each point. Principal component analysis refers to calculating the eigenvalues and eigenvectors of the covariance matrix of a point and the corresponding neighborhood points in the point cloud data. The computer device can calculate the normal vector corresponding to each point in the point cloud data according to the eigenvalues and eigenvectors of the covariance matrix.

[0057] In one of the embodiments, calculating the normal vector corresponding to each point according to the neighborhood points corresponding to each point includes: determining the covariance matrix corresponding to each point according to the neighborhood points corresponding to each point; calculating the eigenvalues of the covariance matrix corresponding to each point, and the eigenvectors corresponding to the eigenvalues; and calculating the normal vector corresponding to each point according to the neighborhood points according to the eigenvalues and the eigenvectors.

[0058] Specifically, for each point in the point cloud data, the computer device can determine the covariance matrix corresponding to the point according to the point and the neighborhood points corresponding to the point. Eigenvalue decomposition processing can be performed on the covariance matrix to obtain the eigenvalues of the covariance matrix, so as to calculate the eigenvectors corresponding to each eigenvalue, and arrange the eigenvalues in a predetermined order. The computer device determines the smallest eigenvalue of the covariance matrix, selects the eigenvector corresponding to the smallest eigenvalue, normalizes the selected eigenvector, and obtains the normal vector corresponding to each point in the point cloud data. Normalization refers to normalizing the selected eigenvector to a unit vector. For example, the computer device obtains a 3*3 symmetric positive semi-definite matrix as the covariance matrix corresponding to a point. The computer device calculates three eigenvalues of the covariance matrix, obtains the eigenvectors corresponding to each eigenvalue, and arranges the three eigenvalues in descending order. In the three eigenvalues, the eigenvector corresponding to the smallest eigenvalue is selected, the selected eigenvector is normalized, and the normal vector corresponding to the point is obtained.

[0059] In this embodiment, after obtaining the point cloud data, the point cloud data is subjected to semantic segmentation by a semantic segmentation model to obtain a semantic segmentation result, so as to cluster the point cloud data according to the semantic segmentation result to obtain a clustering result. According to the clustering result, the neighborhood points corresponding to each point in the point cloud data are selected, and then the normal vector corresponding to each point is calculated according to the neighborhood points corresponding to each point. Since the semantic segmentation result includes the semantic category corresponding to each point, clustering the point cloud data according to the semantic segmentation result avoids the problem of clustering points of different semantic categories together, and improves the accuracy of clustering. Further, selecting the neighborhood points corresponding to each point in the point cloud data according to the clustering result can ensure that the neighborhood points are points in the same clustering category as each point, effectively avoiding the interference of adjacent objects, and more accurate neighborhood points can be obtained, so that the normal vector of each point is calculated according to the neighborhood points, thereby improving the accuracy of point cloud normal vector calculation. In addition, compared with the method based on deep learning which needs to generate training data by simulating a CAD model, the semantic segmentation model is obtained by training a large amount of data carrying semantic category labels annotated by artificial labeling, and the model is trained by real data, thereby improving the accuracy of semantic segmentation.

[0060] In one of the embodiments, as shown in Figure 3 the step of clustering the point cloud data according to the semantic segmentation result to obtain a clustering result includes:

[0061] In step 302, the point cloud data is subjected to connected component detection according to the semantic segmentation result to obtain a plurality of connected components.

[0062] In step 304, the plurality of clustering categories corresponding to the point cloud data and the clustering category corresponding to each point in the point cloud data are determined according to the connected components, and the clustering category and the clustering category corresponding to each point in the point cloud data are taken as the clustering result.

[0063] The semantic segmentation result includes the semantic category corresponding to each point. The computer device can perform connected component detection on the point cloud data according to the semantic category corresponding to each point, combine the points belonging to the same connected component together, and obtain a plurality of connected components. The connected component detection can be performed by spatially dividing the point cloud data, selecting a point as a starting point, extending from the starting point to obtain the relevant region, until there is no continuous point set in the neighborhood of the point, and finally combining the points belonging to the same connected component together to obtain a plurality of connected components. One connected component can correspond to one clustering category, so that the computer device can determine the plurality of clustering categories corresponding to the point cloud data and the clustering category corresponding to each point in the point cloud data according to the connected components, and generate the clustering result according to the clustering category and the clustering category corresponding to each point in the point cloud data.

[0064] In the embodiment, the point cloud data is subjected to connected domain detection according to the semantic segmentation result, and a plurality of clustering categories corresponding to the point cloud data and a clustering category corresponding to each point in the point cloud data are determined according to the plurality of connected domains obtained through the detection, so that points belonging to the same clustering category can be combined together, and the block processing is realized. Meanwhile, it is beneficial to subsequently select the neighborhood points in the same clustering category, and the accuracy of the neighborhood point selection is improved.

[0065] In one of the embodiments, as shown in Figure 4 the step of selecting the neighborhood points corresponding to each point in the point cloud data according to the clustering result comprises:

[0066] In step 402, a current point in the point cloud data is obtained.

[0067] In step 404, the same-category points belonging to the same clustering category as the current point are determined in the clustering result.

[0068] In step 406, the neighborhood points corresponding to the current point are selected from the same-category points.

[0069] In step 408, all the points in the point cloud data are traversed to obtain the neighborhood points corresponding to each point in the point cloud data.

[0070] For each point in the point cloud data, the computer device can directly select the neighborhood points corresponding to the point according to the clustering result. The computer device can select any one point in the point cloud data as the current point. The clustering result includes the clustering categories corresponding to the points in the point cloud data, the computer device can determine the clustering category corresponding to the current point in the clustering result, and all the points in the clustering category corresponding to the current point are selected as the same-category points, so that the neighborhood points are selected from the same-category points. The computer device traverses all the points in the point cloud data, and selects the neighborhood points corresponding to each point in the above manner.

[0071] The computer device can select the neighborhood points in the same-category points in multiple ways. In one of the embodiments, the step of selecting the neighborhood points corresponding to the current point in the same-category points comprises: selecting a preset number of points in the same-category points according to a preset distance parameter to obtain the neighborhood points corresponding to the current point. The preset distance parameter and the preset number are pre-stored in the computer device. For example, the preset distance parameter can be the nearest point, and the preset number can be 30. The computer device thus selects the 30 points nearest to the current point from the same-category points corresponding to the current point.

[0072] In one of the embodiments, the step of selecting the neighborhood points corresponding to the current point in the same-category points comprises: selecting the points within a preset spherical radius from the current point in the same-category points to obtain the neighborhood points corresponding to the current point. The preset spherical radius is stored in the computer device. The computer device takes the current point as the center of the sphere, calculates the spherical space corresponding to the current point according to the preset spherical radius, and takes the same-category points in the spherical space as the neighborhood points corresponding to the current point.

[0073] In the embodiment, the same points belonging to the same cluster category as the current point are determined in the clustering result, the neighborhood points corresponding to the current point are selected from the same points, and the neighborhood points corresponding to all points in the point cloud data are obtained by traversing all points in the point cloud data. Only the neighborhood points in the same cluster category are selected, and the points of one object will not be taken as the neighborhood points of another object, thereby effectively avoiding the interference between adjacent objects and further improving the accuracy of neighborhood point selection.

[0074] In another embodiment, as shown in Figure 5 , a point cloud normal vector calculation method is provided, which specifically includes the following steps:

[0075] Step 502, acquiring point cloud data.

[0076] Step 504, calling a pre-trained semantic segmentation model, inputting the point cloud data into the semantic segmentation model, performing semantic segmentation on the point cloud data by the semantic segmentation model, and obtaining a semantic segmentation result.

[0077] Step 506, clustering the point cloud data according to the semantic segmentation result, and obtaining a clustering result.

[0078] Step 508, extracting a cluster category in the clustering result.

[0079] Step 510, counting the number of points corresponding to the cluster category according to the clustering result.

[0080] Step 512, removing all points corresponding to the cluster category with a number of points less than a threshold.

[0081] Step 514, selecting neighborhood points corresponding to each point in the clustering result after the removal processing.

[0082] Step 516, calculating the normal vector corresponding to each point according to the neighborhood points corresponding to each point.

[0083] After obtaining the point cloud data, the computer device performs semantic segmentation on the point cloud data by using a semantic segmentation model to obtain a semantic segmentation result, and then performs clustering on the point cloud data according to the semantic segmentation result to obtain a clustering result. The computer device can extract clustering categories in the clustering result and count the number of points corresponding to each clustering category. The computer device has a threshold value stored in advance, which is used to determine whether a clustering category needs to be removed. The number of points corresponding to each clustering category is compared with the threshold value. When the number of points is less than the threshold value, the computer device removes the points in the corresponding clustering category as noise points to obtain a clustering result after removal processing, and the clustering result after removal processing includes the point cloud data after removing the noise points. When the number of points is greater than or equal to the threshold value, the computer device retains the corresponding clustering category. The computer device further selects neighborhood points for each point in the clustering result after removal processing. Specifically, for any one point in the clustering result after removal processing, the computer device can determine the clustering category corresponding to the point in the clustering result after removal processing, and select all points in the clustering category corresponding to the point as homogeneous points, so as to select neighborhood points in the homogeneous points. The computer device can select neighborhood points in the homogeneous points in multiple ways. For example, a preset number of points can be selected in the homogeneous points according to a preset distance parameter to obtain neighborhood points corresponding to the point. For example, the preset distance parameter can be the nearest point, and the preset number can be 30. The computer device thus selects 30 points closest to the point in the homogeneous points corresponding to the point. Alternatively, points within a preset spherical radius from the point can be selected in the homogeneous points to obtain neighborhood points corresponding to the point. The computer device calculates a spherical space corresponding to the point with the point as the center of the sphere according to the preset spherical radius, and selects homogeneous points in the spherical space as neighborhood points corresponding to the point. Only neighborhood points in the same clustering category are selected, so that points of one object are not selected as neighborhood points of another object, effectively avoiding interference between adjacent objects and further improving the accuracy of neighborhood point selection. The computer device further calculates a normal vector corresponding to each point according to the neighborhood points corresponding to the point to obtain normal vectors corresponding to the points.

[0084] In this embodiment, the computer device removes points in a clustering category with a number of points less than a threshold value as noise points by counting the number of points in each clustering category in the clustering result. Retaining a clustering category with a number of points greater than or equal to the threshold value can remove noise points in the point cloud data, avoid the problem of selecting noise points as neighborhood points, improve the accuracy of neighborhood point selection, and improve the accuracy of point normal vector calculation according to neighborhood points.

[0085] In one of the embodiments, as shown in Figure 6 A point cloud normal vector calculation device is provided, which includes:

[0086] The acquisition module 602 is configured to acquire point cloud data.

[0087] The segmentation module 604 is configured to call a pre-trained semantic segmentation model, input the point cloud data into the semantic segmentation model, perform semantic segmentation on the point cloud data by using the semantic segmentation model, and obtain a semantic segmentation result.

[0088] The clustering module 606 is configured to cluster the point cloud data according to the semantic segmentation result, and obtain a clustering result.

[0089] The selection module 608 is configured to select, according to the clustering result, a neighborhood point corresponding to each point in the point cloud data.

[0090] The calculation module 610 is configured to calculate a normal vector corresponding to each point according to the neighborhood point corresponding to the point.

[0091] In one of the embodiments, the clustering module 606 is further configured to perform connected component detection on the point cloud data according to the semantic segmentation result, to obtain a plurality of connected components; determine, according to the connected components, a plurality of clustering categories corresponding to the point cloud data and a clustering category corresponding to each point in the point cloud data, and take the clustering categories and the clustering category corresponding to each point in the point cloud data as the clustering result.

[0092] In one of the embodiments, the selection module 608 is further configured to acquire a current point in the point cloud data; determine, according to the clustering result, a same-category point belonging to a same clustering category as the current point; select, from the same-category point, a neighborhood point corresponding to the current point; and traverse all points in the point cloud data to obtain the neighborhood point corresponding to each point in the point cloud data.

[0093] In one of the embodiments, the selection module 608 is further configured to select, according to a preset distance parameter, a preset number of points from the same-category point to obtain the neighborhood point corresponding to the current point.

[0094] In one of the embodiments, the selection module 608 is further configured to select, from the same-category point, a point within a preset spherical radius of the current point to obtain the neighborhood point corresponding to the current point.

[0095] In one of the embodiments, the calculation module 610 is further configured to determine a covariance matrix corresponding to each point according to the neighborhood point corresponding to the point; calculate an eigenvalue of the covariance matrix corresponding to each point and an eigenvector corresponding to the eigenvalue; and calculate the normal vector corresponding to each point according to the neighborhood point corresponding to the point according to the eigenvalue and the eigenvector.

[0096] In one of the embodiments, the apparatus further includes:

[0097] The removal module is configured to extract a clustering category in the clustering result; count a point number corresponding to the clustering category according to the clustering result; and perform removal processing on all points corresponding to a clustering category whose point number is less than a threshold value.

[0098] The selection module 608 is also used to select the neighborhood points corresponding to each point in the clustering results after the removal process.

[0099] The calculation module 610 is also used to calculate the normal vector corresponding to each point based on the neighboring points of each point.

[0100] Specific limitations regarding the point cloud normal vector calculation device can be found in the limitations of the point cloud normal vector calculation method described above, and will not be repeated here. Each module in the aforementioned trajectory prediction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in a computer device, or stored in software in the memory of a computer device, so that the processor can call and execute the corresponding operations of each module.

[0101] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows. Figure 7 As shown, the computer device includes a processor, memory, communication interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer-readable instructions, and a database. The internal memory provides an environment for the operation of the operating system and computer-readable instructions stored in the non-volatile storage medium. The database stores data for a point cloud normal vector calculation method. The communication interface connects to an external terminal for communication. When the computer-readable instructions are executed by the processor, they implement a point cloud normal vector calculation method.

[0102] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer-readable instructions. The internal memory provides an environment for the operation of the operating system and computer-readable instructions stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a point cloud normal vector calculation method. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0103] Those skilled in the art can understand that, Figure 7 The structure shown in FIG. 8 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0104] A computer device includes a memory and one or more processors, the memory storing computer readable instructions, the computer readable instructions being executed by the one or more processors to cause the one or more processors to perform the following steps: obtaining point cloud data; calling a pre-trained semantic segmentation model, inputting the point cloud data into the semantic segmentation model, performing semantic segmentation on the point cloud data by the semantic segmentation model to obtain a semantic segmentation result; clustering the point cloud data according to the semantic segmentation result to obtain a clustering result; selecting, according to the clustering result, a neighborhood point corresponding to each point in the point cloud data; and calculating a normal vector corresponding to each point according to the neighborhood point corresponding to each point.

[0105] In one of the embodiments, the processor, when executing the computer readable instructions, further implements the following steps: performing connected component detection on the point cloud data according to the semantic segmentation result to obtain a plurality of connected components; determining, according to the connected components, a plurality of clustering categories corresponding to the point cloud data and a clustering category corresponding to each point in the point cloud data, and taking the clustering category and the clustering category corresponding to each point in the point cloud data as the clustering result.

[0106] In one of the embodiments, the processor, when executing the computer readable instructions, further implements the following steps: obtaining a current point in the point cloud data; determining, according to the clustering result, a same-category point belonging to a same clustering category as the current point; selecting, among the same-category points, a neighborhood point corresponding to the current point; and traversing all points in the point cloud data to obtain a neighborhood point corresponding to each point in the point cloud data.

[0107] In one of the embodiments, the processor, when executing the computer readable instructions, further implements the following steps: selecting, according to a preset distance parameter, a preset number of points among the same-category points to obtain the neighborhood point corresponding to the current point.

[0108] In one of the embodiments, the processor, when executing the computer readable instructions, further implements the following steps: selecting, among the same-category points, a point within a preset spherical radius of the current point to obtain the neighborhood point corresponding to the current point.

[0109] In one of the embodiments, the processor, when executing the computer readable instructions, further implements the following steps: determining, according to the neighborhood point corresponding to each point, a covariance matrix corresponding to each point; calculating an eigenvalue of the covariance matrix corresponding to each point and an eigenvector corresponding to the eigenvalue; and calculating, according to the eigenvalue and the eigenvector, a normal vector corresponding to each point.

[0110] In one of the embodiments, the processor further implements the following steps when executing the computer readable instructions: extracting a clustering category in the clustering result; counting a point number corresponding to the clustering category according to the clustering result; removing all points corresponding to the clustering category with the point number less than a threshold; selecting a neighborhood point corresponding to each point in the clustering result after the removal; and calculating a normal vector corresponding to each point according to the neighborhood point corresponding to each point.

[0111] One or more non-transitory computer-readable storage media storing computer readable instructions that, when executed by one or more processors, cause the one or more processors to perform the following steps: obtaining point cloud data; calling a pre-trained semantic segmentation model, inputting the point cloud data into the semantic segmentation model, performing semantic segmentation on the point cloud data by the semantic segmentation model, and obtaining a semantic segmentation result; clustering the point cloud data according to the semantic segmentation result to obtain a clustering result; selecting a neighborhood point corresponding to each point in the point cloud data according to the clustering result; and calculating a normal vector corresponding to each point according to the neighborhood point corresponding to each point.

[0112] In one of the embodiments, the computer readable instructions further implement the following steps when executed by the processor: performing connected component detection on the point cloud data according to the semantic segmentation result to obtain a plurality of connected components; determining a plurality of clustering categories corresponding to the point cloud data and a clustering category corresponding to each point in the point cloud data according to the connected components, and taking the clustering category and the clustering category corresponding to each point in the point cloud data as the clustering result.

[0113] In one of the embodiments, the computer readable instructions further implement the following steps when executed by the processor: obtaining a current point in the point cloud data; determining a same-type point belonging to a same clustering category as the current point according to the clustering result; selecting a neighborhood point corresponding to the current point in the same-type point; and traversing all points in the point cloud data to obtain a neighborhood point corresponding to each point in the point cloud data.

[0114] In one of the embodiments, the computer readable instructions further implement the following steps when executed by the processor: selecting a preset number of points in the same-type point according to a preset distance parameter to obtain the neighborhood point corresponding to the current point.

[0115] In one of the embodiments, the computer readable instructions further implement the following steps when executed by the processor: selecting a point within a preset spherical radius from the current point in the same-type point to obtain the neighborhood point corresponding to the current point.

[0116] In one of the embodiments, the computer readable instructions, when executed by the processor, further implement the following steps: determining the covariance matrix corresponding to each point according to the neighborhood points corresponding to the point; calculating the eigenvalue of the covariance matrix corresponding to each point and the eigenvector corresponding to the eigenvalue; and calculating the normal vector corresponding to each point according to the eigenvalue and the eigenvector.

[0117] In one of the embodiments, the computer readable instructions, when executed by the processor, further implement the following steps: extracting the clustering categories in the clustering result; counting the number of points corresponding to the clustering categories according to the clustering result; removing all points corresponding to the clustering categories with the number of points less than the threshold; selecting the neighborhood points corresponding to each point in the clustering result after the removal; and calculating the normal vector corresponding to each point according to the neighborhood points corresponding to the point.

[0118] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by computer readable instructions instructing related hardware, and the computer readable instructions can be stored in a non-volatile computer readable storage medium. When the computer readable instructions are executed, the processes of the above-mentioned embodiments can be included. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM) and the like.

[0119] Each technical feature of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of each technical feature in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present disclosure.

[0120] The above-described embodiments are merely illustrative of several embodiments of the present application, which are described in more detail and in a specific manner, but should not be construed as limiting the scope of the patent. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. A method for calculating point cloud normal vectors, comprising: Acquire point cloud data; A pre-trained semantic segmentation model is invoked, the point cloud data is input into the semantic segmentation model, and the semantic segmentation model performs semantic segmentation on the point cloud data to obtain a semantic segmentation result; the semantic segmentation result includes the semantic category corresponding to each point in the point cloud data; The point cloud data is clustered based on the semantic segmentation results to obtain clustering results; Selecting neighborhood points corresponding to each point in the point cloud data based on the clustering results includes: extracting cluster categories from the clustering results; counting the number of points corresponding to each cluster category based on the clustering results; removing all points corresponding to cluster categories with a number of points less than a threshold; obtaining the current point in the point cloud data; determining points of the same type as the current point in the clustering results after removal; selecting points within a preset sphere radius from the points of the same type to obtain the neighborhood points corresponding to the current point; traversing all points in the point cloud data to obtain the neighborhood points corresponding to each point in the point cloud data; and Calculate the normal vector corresponding to each point based on its neighboring points.

2. The method according to claim 1, characterized in that, The step of clustering the point cloud data based on the semantic segmentation result to obtain the clustering result includes: Based on the semantic segmentation results, connected component detection is performed on the point cloud data to obtain multiple connected components; and Based on the connected components, determine multiple cluster categories corresponding to the point cloud data and the cluster category corresponding to each point in the point cloud data, and use the cluster categories and the cluster categories corresponding to each point in the point cloud data as the clustering results.

3. The method according to claim 1, characterized in that, Selecting neighboring points corresponding to the current point from the same type of points includes: Based on a preset distance parameter, a preset number of points are selected from the points of the same type to obtain the neighboring points corresponding to the current point.

4. The method according to claim 1, characterized in that, The step of calculating the normal vector corresponding to each point based on its neighboring points includes: Determine the covariance matrix of each point based on its neighboring points. Calculate the eigenvalues ​​of the covariance matrix corresponding to each point, and the eigenvectors corresponding to the eigenvalues; and Calculate the normal vector corresponding to each point based on the eigenvalue and eigenvector.

5. A point cloud normal vector calculation device, comprising: The acquisition module is used to acquire point cloud data; The segmentation module is used to call a pre-trained semantic segmentation model, input the point cloud data into the semantic segmentation model, and perform semantic segmentation on the point cloud data through the semantic segmentation model to obtain a semantic segmentation result; the semantic segmentation result includes the semantic category corresponding to each point in the point cloud data; The clustering module is used to cluster the point cloud data according to the semantic segmentation result to obtain the clustering result; The selection module is used to select neighborhood points corresponding to each point in the point cloud data based on the clustering results, including: extracting cluster categories from the clustering results; counting the number of points corresponding to each cluster category based on the clustering results; removing all points corresponding to cluster categories with a number of points less than a threshold; obtaining the current point in the point cloud data; determining points of the same type as the current point in the clustering results after removal; selecting points within a preset sphere radius from the points of the same type to obtain the neighborhood points corresponding to the current point; traversing all points in the point cloud data to obtain the neighborhood points corresponding to each point in the point cloud data; and The calculation module is used to calculate the normal vector corresponding to each point based on its neighboring points.

6. The apparatus according to claim 5, characterized in that, The clustering module is further configured to perform connected component detection on the point cloud data based on the semantic segmentation result to obtain multiple connected components; and to determine multiple clustering categories corresponding to the point cloud data and the clustering category corresponding to each point in the point cloud data based on the connected components, and to use the clustering categories and the clustering categories corresponding to each point in the point cloud data as the clustering result.

7. A computer device comprising a memory and one or more processors, the memory storing computer-readable instructions that, when executed by the one or more processors, cause the one or more processors to perform the following steps: Acquire point cloud data; A pre-trained semantic segmentation model is invoked, the point cloud data is input into the semantic segmentation model, and the semantic segmentation model performs semantic segmentation on the point cloud data to obtain a semantic segmentation result; the semantic segmentation result includes the semantic category corresponding to each point in the point cloud data; The point cloud data is clustered based on the semantic segmentation results to obtain clustering results; Selecting neighborhood points corresponding to each point in the point cloud data based on the clustering results includes: extracting cluster categories from the clustering results; counting the number of points corresponding to each cluster category based on the clustering results; removing all points corresponding to cluster categories with a number of points less than a threshold; obtaining the current point in the point cloud data; determining points of the same type as the current point in the clustering results after removal; selecting points within a preset sphere radius from the points of the same type to obtain the neighborhood points corresponding to the current point; traversing all points in the point cloud data to obtain the neighborhood points corresponding to each point in the point cloud data; and Calculate the normal vector corresponding to each point based on its neighboring points.

8. The computer device according to claim 7, characterized in that, When the processor executes the computer-readable instructions, it further performs the following steps: clustering the point cloud data according to the semantic segmentation result to obtain the clustering result includes: performing connected component detection on the point cloud data according to the semantic segmentation result to obtain multiple connected components; and determining multiple clustering categories corresponding to the point cloud data and the clustering category corresponding to each point in the point cloud data according to the connected components, and taking the clustering category and the clustering category corresponding to each point in the point cloud data as the clustering result.

9. The computer device according to claim 7, characterized in that, When the processor executes the computer-readable instructions, it also performs the following steps: selecting a preset number of points among the same type of points according to a preset distance parameter to obtain the neighboring points corresponding to the current point.

10. One or more non-volatile computer-readable storage media storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the following steps: Acquire point cloud data; A pre-trained semantic segmentation model is invoked, the point cloud data is input into the semantic segmentation model, and the semantic segmentation model performs semantic segmentation on the point cloud data to obtain a semantic segmentation result; the semantic segmentation result includes the semantic category corresponding to each point in the point cloud data; The point cloud data is clustered based on the semantic segmentation results to obtain clustering results; Selecting neighborhood points corresponding to each point in the point cloud data based on the clustering results includes: extracting cluster categories from the clustering results; counting the number of points corresponding to each cluster category based on the clustering results; removing all points corresponding to cluster categories with a number of points less than a threshold; obtaining the current point in the point cloud data; determining points of the same type as the current point in the clustering results after removal; selecting points within a preset sphere radius from the points of the same type to obtain the neighborhood points corresponding to the current point; traversing all points in the point cloud data to obtain the neighborhood points corresponding to each point in the point cloud data; and Calculate the normal vector corresponding to each point based on its neighboring points.

11. The storage medium according to claim 10, characterized in that, When the computer-readable instructions are executed by the processor, the following steps are also performed: performing connected component detection on the point cloud data based on the semantic segmentation result to obtain multiple connected components; and determining multiple clustering categories corresponding to the point cloud data and the clustering category corresponding to each point in the point cloud data based on the connected components, and taking the clustering category and the clustering category corresponding to each point in the point cloud data as the clustering result.

12. The storage medium according to claim 10, characterized in that, When the computer-readable instructions are executed by the processor, the following steps are also performed: selecting a preset number of points among the same type of points according to a preset distance parameter to obtain the neighboring points corresponding to the current point.

Images