Three-dimensional point cloud semantic segmentation method based on directional grouping and parameter adaptation

Abstract

The invention provides a three-dimensional point cloud semantic segmentation method based on directional grouping and parameter self-adaption, and the method comprises the steps: carrying out the initial grouping of point clouds, and adaptively adjusting the parameters of a directional grouping method under the condition that at least a part of groups in the initial grouping do not meet the grouping requirements of a high-curvature curved surface or region, and re-grouping is carried out according to the adjusted parameters, and semantic segmentation processing is carried out on the obtained target grouping result after grouping is finished. According to the method, adaptive adjustment of the grouping parameters of the high-curvature curved surface or region can be realized, so that a more reasonable grouping result is obtained, subsequent feature extraction is facilitated, and a positive influence is generated on a final segmentation result.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular, to a three-dimensional point cloud semantic segmentation method based on directional grouping and parameter adaptation. Background Art

[0002] Three-dimensional semantic segmentation is the basis for realizing positioning navigation, path planning, and motion control. High-quality three-dimensional semantic segmentation results are the key prerequisites for the successful application of autonomous driving and robot navigation. In a three-dimensional point cloud semantic segmentation model based on deep learning, the grouping layer is mainly used for feature extraction and aggregation, and is an important part of the three-dimensional point cloud semantic segmentation model.

[0003] However, in the related art, the grouping method in three-dimensional point cloud understanding cannot well handle the problems of high-curvature surfaces or regions. Summary of the Invention

[0004] Embodiments of the present application provide a three-dimensional point cloud semantic segmentation method and device based on directional grouping and parameter adaptation.

[0005] In a first aspect, an embodiment of the present application provides a three-dimensional point cloud semantic segmentation method based on directional grouping and parameter adaptation, including: Obtain a three-dimensional point cloud to be processed, and perform initial grouping on the three-dimensional point cloud by using a first directional grouping method to obtain m groups, where each group in the m groups includes k points, and m and k are positive integers; In the case that at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions, adaptively adjust the parameters of the first directional grouping method, and use the first directional grouping method to perform re-grouping on the at least some groups respectively based on the adjusted parameters until the obtained groups meet the grouping requirements for high-curvature surfaces or regions; Obtain a target grouping result based on the at least some groups, the groups obtained after re-grouping, and other groups; where the other groups are the groups in the m groups except the at least some groups; Perform semantic segmentation processing based on the target grouping result.

[0006] In a second aspect, an embodiment of the present application provides a three-dimensional point cloud semantic segmentation device based on directional grouping and parameter adaptation, including: An initial grouping module, configured to obtain a three-dimensional point cloud to be processed, and perform initial grouping on the three-dimensional point cloud by using a first directional grouping method to obtain m groups, where each group in the m groups includes k points, and m and k are positive integers; A regrouping module, configured to adaptively adjust parameters of the first directional grouping method when at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions, and use the first directional grouping method to respectively regroup the at least some groups based on the adjusted parameters until the obtained groups meet the grouping requirements for the high-curvature surfaces or regions; An obtaining module, configured to obtain a target grouping result based on the at least some groups, the groups obtained after regrouping, and other groups; wherein, the other groups are the groups other than the at least some groups among the m groups; A semantic segmentation processing module, configured to perform semantic segmentation processing based on the target grouping result.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, including: At least one processor; A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the foregoing first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a storage medium, where the storage medium stores instructions, and when the instructions run on an electronic device, the electronic device is enabled to execute the method described in the foregoing first aspect.

[0009] In a fifth aspect, an embodiment of the present application provides a program product, including at least one of a program and instructions, wherein when at least one of the program and instructions is executed by a processor, steps of the method described in the foregoing first aspect are implemented.

[0010] According to the technical solution of the present application, it is possible to adaptively adjust the grouping parameters for high-curvature surfaces or regions, thereby obtaining a more reasonable grouping result, which is helpful for subsequent feature extraction and has a positive impact on the final segmentation result.

[0011] Additional aspects and advantages of the present application will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of the present application. Description of the Drawings

[0012] The above and / or additional aspects and advantages of the present application will become obvious and easy to understand from the following description of the embodiments in conjunction with the drawings, where: Figure 1 is a schematic flowchart of a three-dimensional point cloud semantic segmentation method based on directional grouping and parameter adaptation provided by an embodiment of the present application; Figure 2 Schematic diagram of the directional k-nearest neighbor grouping result provided by the embodiment of the present application; Figure 3 Schematic diagram of the directional sphere query grouping result provided by the embodiment of the present application; Fig. 4(a) is an example of the directional serialized nearest neighbor grouping result provided by the embodiment of the present application Figure 1 ; Fig. 4(b) is an example of the directional serialized nearest neighbor grouping result provided by the embodiment of the present application Figure 2 ; Fig. 4(c) is an example of the directional serialized nearest neighbor grouping result provided by the embodiment of the present application Figure 3 ; Fig. 4(d) is an example diagram of the directional serialized nearest neighbor grouping result provided by the embodiment of the present application; Figure 5 Block diagram of the three-dimensional point cloud semantic segmentation device based on directional grouping and parameter adaptation provided by the embodiment of the present application; Figure 6 is a block diagram of an electronic device according to an embodiment of the present application. Detailed implementation manners

[0013] The embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, and should not be construed as limiting the present application.

[0014] The embodiments of the present application are not exhaustive. They are only schematic diagrams of some embodiments and do not specifically limit the protection scope of the present application. Without contradiction, each step in a certain embodiment can be implemented as an independent embodiment, and the steps can be combined arbitrarily. For example, the solution after removing some steps in a certain embodiment can also be implemented as an independent embodiment, and the order of the steps in a certain embodiment can be arbitrarily exchanged. In addition, the optional implementation manners in a certain embodiment can be combined arbitrarily; furthermore, the embodiments can be combined arbitrarily. For example, some or all of the steps of different embodiments can be combined arbitrarily, and a certain embodiment can be combined arbitrarily with the optional implementation manners of other embodiments.

[0015] In each embodiment of the present application, if there is no special description and logical conflict, the terms and / or descriptions between the embodiments are consistent and can be cited from each other. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0016] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0017] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information and other processing all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0018] It is worth noting that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0019] It should be noted that in the embodiments of this application, some industry-existing solutions such as certain software, components, models, etc. may be mentioned. They should be considered exemplary. The purpose is only to illustrate the feasibility in the implementation of the technical solution of this application, but it does not mean that the applicant has already or necessarily used this solution.

[0020] Next, a three-dimensional point cloud semantic segmentation method and device based on directional grouping and parameter adaptation according to the embodiments of this application will be described with reference to the accompanying drawings.

[0021] Among them, it should be noted that the execution subject of the three-dimensional point cloud semantic segmentation method based on directional grouping and parameter adaptation in the embodiments of this application can be a three-dimensional point cloud semantic segmentation device based on directional grouping and parameter adaptation. This device can be implemented in software and / or hardware, and this device can be configured in an electronic device. Exemplarily, the electronic device can include but not be limited to a terminal, a server, etc.

[0022] Figure 1 It is a schematic flowchart of the three-dimensional point cloud semantic segmentation method based on directional grouping and parameter adaptation provided for the embodiments of this application. As Figure 1 shown, the three-dimensional point cloud semantic segmentation method based on directional grouping and parameter adaptation may include but not be limited to the following steps.

[0023] In step 101, the three-dimensional point cloud to be processed is obtained, and the first directional grouping method is used to perform an initial grouping on the three-dimensional point cloud to obtain m groups, and each group among the m groups includes k points, where m and k are positive integers.

[0024] In some embodiments, when obtaining the three-dimensional point cloud to be processed, the first directional grouping method can be used to perform an initial grouping on the three-dimensional point cloud. For example, taking the complete three-dimensional point cloud to be processed as an example, it contains the coordinates of the points and features , ( ); The first directional grouping method can be used to perform initial grouping on the 3D point cloud, that is, sampling the point cloud, with a total of m points (such as designated points or center points can be denoted), and there are k points in each group where the sampling point is located. After grouping, the shape of the point cloud is , where d is the dimensionality of the point cloud features.

[0025] In some embodiments, the first directional grouping method can include any one of the directional k-nearest neighbor grouping method, the directional ball query grouping method, and the directional sequential nearest neighbor grouping method. Among them, the directional k-nearest neighbor grouping method can refer to dividing the k points with the closest Euclidean distance to the designated point into a group. That is to say, the directional k-nearest neighbor grouping is to select the k points with the closest weighted distance to the designated point and divide them into a group. As Figure 2 shown, a schematic diagram of the directional k-nearest neighbor grouping result. This grouping strategy can obtain an accurate nearest neighbor grouping, which is beneficial to ensuring the performance of the model. Exemplarily, in order to implement the directional k-nearest neighbor grouping, weights a, b, and c can be assigned to the x, y, and z components of the coordinates respectively, and the weighted distance is used for measurement. Optionally, the measurement calculation formula of the weighted distance is expressed as follows:

[0026] Among them, , , respectively represent the coordinate values of the sampling center point , , , respectively represent the coordinate values of the neighborhood point , , , respectively represent , , the weights of the components, is the weighted distance between point and point .

[0027] In the embodiments of the present application, the directional ball query grouping method can refer to dividing the k points within the Euclidean distance of R from the designated point into a group. For example, if it is necessary to perform directional ball query sampling centered on in , first, it is necessary to calculate the weighted distance between and the remaining points respectively, and then randomly select k points to achieve grouping. As Figure 3 shown, a schematic diagram of the directional ball query grouping result. Considering the geometric meaning, the directional ball query grouping is equivalent to screening out those centered on centered at the sphere center, with semi-major axis lengths of , , respectively. Points within the ellipsoid are thus grouped by directional sphere queries, which can also be referred to as "ellipsoid query grouping".

[0028] In an embodiment of the present application, the directional serialized nearest neighbor grouping method may refer to encoding a three-dimensional point cloud using a space-filling curve and then dividing k nearest points on the space-filling curve path into a group. Optionally, the space-filling curve used in the directional serialized nearest neighbor grouping method may be a Z-curve or a Hilbert curve. It should be noted that a space-filling curve is a path through each point in a high-dimensional discrete space and can, to a certain extent, preserve spatial neighborhood relationships. Such a space-filling curve can be mathematically represented as a bijective function . Serializing and encoding the point cloud means converting the three-dimensional point cloud coordinates into serialized integer encodings . First, project the three-dimensional point cloud onto a discrete space with a grid size of and encode it: . Since points that are neighbors on the space-filling curve path also have a certain neighborhood relationship in space, this neighborhood relationship can be used to approximately replace the spatial neighborhood relationship, providing the possibility for efficient neighborhood search of the point cloud. After encoding the three-dimensional point cloud using a space-filling curve, k nearest points on the space-filling curve path can be divided into a group.

[0029] As an example, the space-filling curve used in the directional serialized nearest neighbor grouping method may be a Z-curve. The core idea of the Z-curve is to encode multi-dimensional coordinates into a one-dimensional value by alternately cutting the space, and the lower three bits of its encoded value are composed of the low bits of the x, y, and z coordinates in an interleaved manner. For the grouping method based on the Z-curve, by changing the number of binary digits in the xyz alternation, more points can be sampled in a certain direction. For example, changing "x(1bit)-y(1bit)-z(1bit)" to "x(1bit)-y(1bit)-z(2bit)" allows for sampling more points in the z direction than in the x and y directions. As shown in Figure 4(a), it is an example diagram of the directional serialized nearest neighbor grouping result based on Z-curve encoding (xyz order), Figure 4(b) shows an example diagram of the directional serialized nearest neighbor grouping result based on directional Z-curve encoding (xyyz order), Figure 4(c) shows an example diagram of the directional serialized nearest neighbor grouping result based on directional Z-curve encoding (xxyz order), and Figure 4(d) shows an example diagram of the directional serialized nearest neighbor grouping result based on directional Z-curve encoding (xyzz order).

[0030] In step 102, when at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions, the parameters of the first directional grouping method are adaptively adjusted, and the first directional grouping method is used to re-group at least some of the groups respectively based on the adjusted parameters until the obtained groups meet the grouping requirements for high-curvature surfaces or regions.

[0031] In some embodiments, it may be determined whether there are groups among the m groups that do not meet the grouping requirements for high-curvature surfaces or regions. Optionally, when at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions, the parameters of the first directional grouping method can be adaptively adjusted and re-grouped according to the adjusted parameters. Optionally, for the group to be re-grouped currently, the points used in this group during the initial grouping can continue to be used as the center point, and the first directional grouping method is used to sample the points near the center point based on the adjusted parameters to re-group the current group. It should be noted that the 3D point cloud may be divided into one or more different groups.

[0032] In some embodiments, the optional implementation manner of determining that at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions includes: determining the point cloud covariance matrix within each group; determining that at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions according to the eigenvalues obtained from the decomposition of the covariance matrix. Exemplarily, that a group does not meet the grouping requirements for high-curvature surfaces or regions may include: the maximum value among the eigenvalues obtained from the decomposition of the point cloud covariance matrix within the group is greater than or equal to a threshold, that is: for each group, if the maximum value among the eigenvalues obtained from the decomposition of the point cloud covariance matrix within the current group is greater than or equal to the threshold, then the current group does not meet the grouping requirements for high-curvature surfaces or regions, and the parameters of the first directional grouping method need to be adjusted to re-group the points within the current group; if the maximum value among the eigenvalues obtained from the decomposition of the point cloud covariance matrix within the current group is less than the threshold, it is considered that the current group meets the grouping requirements for high-curvature surfaces or regions, and there is no need to re-group the current group. Optionally, as an example, the calculation formula of the point cloud covariance matrix within the group is expressed as follows:

[0033] where, is the point cloud covariance matrix within the group; is the coordinate value of the i th point within the group; is the average value of the point cloud coordinates within the group.

[0034] Exemplarily, the covariance matrix can be decomposed into eigenvalues, and the formula is expressed as follows:

[0035] Among them, is an orthogonal matrix, and its column vectors are the eigenvectors of the covariance matrix; is a diagonal matrix, and the elements on the diagonal are eigenvalues.

[0036] In the embodiments of the present application, for each group, when the maximum value among the eigenvalues obtained by decomposing the covariance matrix within the group is greater than or equal to a threshold, the point cloud within the group can be regrouped using the first directional grouping method, and a larger receptive field is set for the axis (or coordinate component) corresponding to the maximum eigenvalue, while the receptive fields of the other two axes (or the other two coordinate components) remain unchanged. Optionally, the method for obtaining a larger receptive field is as follows: 1) For the directional k-nearest neighbor grouping method or the directional spherical query grouping method For a group that does not meet the grouping requirements of a high-curvature surface or region, based on the maximum value among the eigenvalues obtained by decomposing the covariance matrix within the group, the weight of the coordinate component corresponding to the maximum value can be adjusted to serve as an adjustment of the parameters of the first directional grouping method. As an example, the weight of the coordinate component corresponding to the maximum value can be increased by a first value.

[0037] For example, assume that the maximum value among the eigenvalues obtained by decomposing the covariance matrix within a certain group A among m groups is greater than the threshold, and the coordinate component corresponding to the maximum eigenvalue is the y component. Then, the weight b of the y component can be adjusted. For example, the initial value of the weight b is 1, and the first value can be added to the value of the weight b. Exemplarily, the first value can be a fixed value (such as 1). The present application does not specifically limit the adjustment method of the weight of the coordinate component, and other methods can also be used to implement the adjustment of the weight. For example, based on the mapping relationship between the preset weight value and the adjustment value, the size of the weight b of the y component can be adjusted. For example, the larger the maximum eigenvalue, the greater the adjustment amplitude of the weight b. After completing the adjustment of the weight b of the y component, based on the sizes of the weights a, c, and the adjusted weight b, the point cloud within the group A can be regrouped. For example, sampling can continue in the point cloud near the center point of the group A to obtain a new group A1. After regrouping, it can continue to be determined whether the group obtained after regrouping meets the grouping requirements of a high-curvature surface or region. If not, the parameters of the first directional grouping method can continue to be adaptively adjusted until the obtained group meets the grouping requirements of a high-curvature surface or region.

[0038] 2) For the directional sequential nearest neighbor grouping method, the space-filling curve used is the Z curve For a grouping that does not meet the grouping requirements of a high-curvature surface or region, the encoding bits and / or order parameters of the first directional grouping method can be adjusted based on the maximum value among the eigenvalues obtained from the covariance matrix decomposition within the grouping. As an example, the encoding bits in the coordinate component direction corresponding to the maximum value can be increased by a first number of bits to adjust the encoding bits and order parameters of the first directional grouping method. Exemplarily, the first number can be 1, or the first number can be other values. For example, the value of the first number can be determined based on the specific magnitude of the maximum value. For instance, the larger the maximum value, the larger the value of the first number, but it is not limited thereto.

[0039] For example, assume that the maximum value among the eigenvalues obtained from the covariance matrix decomposition within a certain grouping A among m groupings is greater than the threshold, and the coordinate component corresponding to the maximum eigenvalue is the y-component. Then, the encoding bits and / or encoding order in the y-component direction can be adjusted. Taking the encoding order as the xyz order and the encoding bits in each component direction as 1 bit during the initial grouping, when it is determined that the point cloud within grouping A needs to be regrouped, the encoding bits in the coordinate component y-component direction corresponding to the maximum value can be changed from 1 bit to 2 bits, and the encoding order can be changed to the xyyz order. The point cloud within grouping A can be regrouped using the adjusted grouping parameters. For example, sampling can continue in the vicinity of the center point of grouping A to obtain a new grouping A1. After regrouping, it can continue to be determined whether the groupings obtained after regrouping meet the grouping requirements of a high-curvature surface or region. If not, the parameters of the first directional grouping method can continue to be adaptively adjusted until the obtained groupings meet the grouping requirements of a high-curvature surface or region.

[0040] In step 103, based on at least some of the groupings, the groupings obtained after regrouping, and other groupings, a target grouping result is obtained; where the other groupings are the groupings among the m groupings other than at least some of the groupings.

[0041] In some embodiments, each grouping in at least some of the groupings can be merged with the grouping obtained after its regrouping to obtain the merged groupings associated with at least some of the groupings; k-point sampling processing is performed on each merged grouping to obtain the corresponding sampled groupings, and the sampled groupings and other groupings are used as the target grouping result.

[0042] For example, taking m groupings including grouping A, grouping B, and grouping C, where grouping A and grouping B have been regrouped once, and grouping C has not been regrouped. The grouping obtained after regrouping grouping A is denoted as A1, and the grouping obtained after regrouping grouping B is denoted as B1. The groupings A1 and B1 obtained after regrouping meet the grouping requirements of a high-curvature surface or region and do not need to be regrouped further. At this time, grouping A and A1 can be merged, such as performing a union operation A , the union of grouping B and B1 can be processed as B , the merged grouping A can be processed by sampling k points, and the merged grouping B is sampled by k points to ensure that each grouping in the grouping result contains k points. The k points sampled from the merged grouping A are used as a sampling grouping, and the k points sampled from the merged grouping B are used as a sampling grouping. These two sampling groupings and grouping C are used as the target grouping result, that is, the final grouping result is obtained, and the grouping ends here.

[0043] In step 104, semantic segmentation processing is performed based on the target grouping result.

[0044] In the embodiments of the present application, after the grouping is completed, the points in each grouping of the target grouping result can be subjected to subsequent feature extraction operations to continue the subsequent semantic segmentation processing.

[0045] In the above embodiments, the present application optimizes on the basis of the grouping module in the existing three-dimensional point cloud semantic segmentation model to achieve directional grouping. This grouping method adaptively adjusts the grouping parameters for high-curvature surfaces or regions, optimizing the grouping effect of high-curvature surfaces or regions. Specifically, the present application designs a directional grouping method that can achieve directional grouping in different degrees and directions, and hardly increases the model complexity, and is compatible with the traditional three-dimensional point cloud semantic segmentation process. In addition, the present application provides a parameter adjustment method for adaptive directional grouping, which can adapt to different curvature surfaces or regions.

[0046] Figure 5 is a block diagram of a three-dimensional point cloud semantic segmentation device based on directional grouping and parameter adaptation provided by an embodiment of the present application. As Figure 5 shown, the three-dimensional point cloud semantic segmentation device based on directional grouping and parameter adaptation may include: an initial grouping module 501, a regrouping module 502, an acquisition module 503, and a semantic segmentation processing module 504.

[0047] Among them, the initial grouping module 501 is used to obtain the three-dimensional point cloud to be processed, and perform initial grouping on the three-dimensional point cloud by using the first directional grouping method to obtain m groupings, and each grouping among the m groupings contains k points, where m and k are positive integers.

[0048] The regrouping module 502 is configured to adaptively adjust the parameters of the first directional grouping method when at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions, and use the first directional grouping method to regroup at least some of the groups respectively based on the adjusted parameters until the obtained groups meet the grouping requirements for high-curvature surfaces or regions.

[0049] The obtaining module 503 is configured to obtain a target grouping result based on at least some of the groups, the groups obtained after regrouping, and other groups; where the other groups are the groups other than at least some of the m groups.

[0050] The semantic segmentation processing module 504 is configured to perform semantic segmentation processing based on the target grouping result.

[0051] In some embodiments, the regrouping module 502 is configured to: determine the point cloud covariance matrix within each group; determine that at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions according to the eigenvalues obtained from the decomposition of the covariance matrix.

[0052] In some embodiments, the regrouping module 502 is configured to: for each group, determine that the group does not meet the grouping requirements for high-curvature surfaces or regions when the maximum value among the eigenvalues obtained from the decomposition of the covariance matrix within the group is greater than or equal to a threshold. In a possible implementation, the calculation formula for the point cloud covariance matrix within the group is expressed as follows:

[0053] where, is the point cloud covariance matrix within the group; is the coordinate value of the i th point within the group; is the mean value of the point cloud coordinates within the group.

[0054] In some embodiments, the first directional grouping method includes any one of a directional k-nearest neighbor grouping method, a directional sphere query grouping method, and a directional sequential nearest neighbor grouping method; where the directional k-nearest neighbor grouping method means dividing the k points with the closest Euclidean distance to a specified point into a group; the directional sphere query grouping method means dividing the k points within a Euclidean distance of R from a specified point into a group; the directional sequential nearest neighbor grouping method means encoding the three-dimensional point cloud using a space-filling curve and then dividing the k nearest neighbor points on the space-filling curve path into a group.

[0055] In some embodiments, the first directional grouping method includes a directional k-nearest neighbor grouping method or a directional ball query grouping method. In the embodiments of the present application, the regrouping module 502 is configured to: for a grouping that does not meet the grouping requirements of a high-curvature surface or region, based on the maximum value among the eigenvalues obtained from the covariance matrix decomposition within the grouping, adjust the weight of the coordinate component corresponding to the maximum value, so as to serve as an adjustment to the parameters of the first directional grouping method. In a possible implementation manner, the regrouping module 502 is configured to: increase the weight of the coordinate component corresponding to the maximum value by a first value.

[0056] In some embodiments, the first directional grouping method includes a directional sequential nearest neighbor grouping method, and the space-filling curve used is the Z-curve. In the embodiments of the present application, the regrouping module 502 is configured to: for a grouping that does not meet the grouping requirements of a high-curvature surface or region, based on the maximum value among the eigenvalues obtained from the covariance matrix decomposition within the grouping, adjust the number of encoding bits and / or the order parameter of the first directional grouping method. In a possible implementation manner, the regrouping module 502 is configured to: increase the number of encoding bits in the direction of the coordinate component corresponding to the maximum value by a first number of bits, so as to adjust the number of encoding bits and the order parameter of the first directional grouping method.

[0057] In some embodiments, the obtaining module 503 is configured to: perform a merging process on each grouping in at least some groupings and the regrouped grouping obtained therefrom, to obtain each merged grouping associated with at least some groupings; perform a k-point sampling process on each merged grouping respectively, to obtain the corresponding sampled groupings, and use the sampled groupings and other groupings as the target grouping results.

[0058] It should be noted that the foregoing explanations of the embodiments of the 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation are also applicable to the 3D point cloud semantic segmentation device based on directional grouping and parameter adaptation in this embodiment, and will not be elaborated herein.

[0059] According to the embodiments of the present application, the present application also provides an electronic device and a readable storage medium.

[0060] As Figure 6 shown, it is a block diagram of an electronic device according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described herein and / or claimed.

[0061] As shown Figure 6 in the figure, the electronic device includes: one or more processors 601, a memory 602, and interfaces for connecting the components, including a high-speed interface and a low-speed interface. Each component is interconnected using different buses and can be mounted on a common motherboard or otherwise installed as required. The processor can process instructions executed within the electronic device, including instructions stored in the memory or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a set of blade servers, or a multi-processor system). Figure 6 In Figure 6 , a processor 601 is taken as an example.

[0062] The memory 602 is the non-transitory computer-readable storage medium provided in this application. Among them, the memory stores instructions executable by at least one processor, so that the at least one processor executes the three-dimensional point cloud semantic segmentation method based on directional grouping and parameter adaptation provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions, and the computer instructions are used to make a computer execute the three-dimensional point cloud semantic segmentation method based on directional grouping and parameter adaptation provided in this application.

[0063] The memory 602, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the three-dimensional point cloud semantic segmentation method based on directional grouping and parameter adaptation in the embodiments of this application (for example, Figure 5 the initial grouping module 501, the regrouping module 502, the acquisition module 503, and the semantic segmentation processing module 504 shown in the figure). The processor 601 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions, and modules stored in the memory 602, that is, implements the three-dimensional point cloud semantic segmentation method based on directional grouping and parameter adaptation in the above method embodiments.

[0064] The memory 602 may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function. The data storage area may store data created according to the use of the electronic device and the like. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 602 may optionally include a memory remotely provided with respect to the processor 601, and these remote memories may be connected to the electronic device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0065] The electronic device may further include: an input device 603 and an output device 604. The processor 601, the memory 602, the input device 603, and the output device 604 may be connected through a bus or other means. Figure 6 Taking the connection through the bus as an example.

[0066] The input device 603 may receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the electronic device, such as input devices including a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 604 may include a display device, an auxiliary lighting device (e.g., an LED), and a haptic feedback device (e.g., a vibration motor), etc. The display device may include but is not limited to a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0067] Various embodiments of the systems and techniques described herein may be implemented in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and / or interpreted on a programmable system including at least one programmable processor, the programmable processor may be a dedicated or general-purpose programmable processor, and may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0068] These computing procedures (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0069] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0070] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), the Internet, and blockchain networks.

[0071] A computer system may include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0072] It should be understood that various forms of processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in this application can be executed in parallel, sequentially or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved, and no limitations are imposed herein.

[0073] The above specific embodiments do not constitute a limitation on the protection scope of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims

1. A 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation, characterized in that Including the following steps: Obtain the three-dimensional point cloud to be processed, and perform initial grouping on the three-dimensional point cloud using a first directional grouping method to obtain m groups, where each group among the m groups includes k points, and m and k are positive integers; In the case that at least some of the m groups do not meet the grouping requirements of the high-curvature surface or region, adaptively adjust the parameters of the first directional grouping method, and use the first directional grouping method to re-group the at least some groups respectively based on the adjusted parameters until the obtained groups meet the grouping requirements of the high-curvature surface or region; Obtain a target grouping result based on the at least some groups, the groups obtained after re-grouping, and other groups; where the other groups are the groups among the m groups except the at least some groups; Perform semantic segmentation processing based on the target grouping result.

2. The method according to claim 1, wherein Determining that at least some of the m groups do not meet the grouping requirements of the high-curvature surface or region includes: Determine the covariance matrix of the point cloud within each group; Based on the eigenvalues obtained from the decomposition of the covariance matrix, determine that at least some of the m groups do not meet the grouping requirements of the high-curvature surface or region.

3. The method according to claim 2, wherein The determining that at least some of the m groups do not meet the grouping requirements of the high-curvature surface or region based on the eigenvalues obtained from the decomposition of the covariance matrix includes: For each group, in the case that the maximum value among the eigenvalues obtained from the decomposition of the covariance matrix within the group is greater than or equal to a threshold, determine that the group does not meet the grouping requirements of the high-curvature surface or region.

4. The method according to claim 2, wherein The calculation formula of the covariance matrix of the point cloud within the group is expressed as follows: Among them, is the covariance matrix of the point cloud within the said group; is the coordinate value of the i th point within the said group; is the mean value of the point cloud coordinates within the said group.

5. The method according to any one of claims 1-4, characterized in that, The first directional grouping method includes any one of a directional k-nearest neighbor grouping method, a directional sphere query grouping method, and a directional sequential nearest neighbor grouping method; where The directional k-nearest neighbor grouping method means dividing the k points with the closest Euclidean distance to the specified point into one group; The directional sphere query grouping method means dividing the k points within a Euclidean distance of R from the specified point into one group; The directional sequential nearest neighbor grouping method means that after encoding the three-dimensional point cloud using a space-filling curve, dividing the k nearest neighbor points on the space-filling curve path into one group.

6. The method according to claim 5, characterized in that, The first directional grouping method includes the directional k-nearest neighbor grouping method or the directional sphere query grouping method; The adaptively adjusting the parameters of the first directional grouping method includes: For the groups that do not meet the grouping requirements of the high-curvature surface or region, based on the maximum value among the eigenvalues obtained from the decomposition of the covariance matrix within the group, adjust the weight of the coordinate component corresponding to the maximum value as the adjustment of the parameters of the first directional grouping method.

7. The method according to claim 6, characterized in that, The adjusting the weight of the coordinate component corresponding to the maximum value includes: Increase the weight of the coordinate component corresponding to the maximum value by a first value.

8. The method according to claim 5, characterized in that, The first directional grouping method includes the directional sequential nearest neighbor grouping method, and the space-filling curve used is the Z curve; The adaptively adjusting the parameters of the first directional grouping method includes: For a grouping that does not meet the grouping requirements of the high-curvature surface or region, based on the maximum value among the eigenvalues obtained from the covariance matrix decomposition within the grouping, adjust the encoding bits and / or order parameters of the first directional grouping method.

9. The method according to claim 8, wherein The adjustment of the encoding bits and / or order parameters of the first directional grouping method includes: Increase the encoding bits of the coordinate component direction corresponding to the maximum value by a first number of bits to adjust the encoding bits and order parameters of the first directional grouping method.

10. The method according to claim 1, wherein The obtaining of the target grouping result based on the at least partial groupings, the groupings obtained after regrouping, and the other groupings includes: Perform a merging process on each grouping in the at least partial groupings and the groupings obtained after regrouping them to obtain the merged groupings associated with the at least partial groupings; Perform a k-point sampling process on each of the merged groupings to obtain the corresponding sampled groupings, and use the sampled groupings and the other groupings as the target grouping result.

Images