A 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation

Abstract

This application proposes a 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation. The method includes: first, initial grouping of the point cloud; if at least some groups in the initial grouping do not meet the grouping requirements for high-curvature surfaces or regions, adaptively adjusting the parameters of the directional grouping method; and then regrouping based on the adjusted parameters. After grouping, the resulting target groupings are subjected to semantic segmentation. This application can adaptively adjust the grouping parameters for high-curvature surfaces or regions, thereby obtaining more reasonable grouping results, which is helpful for subsequent feature extraction and has a positive impact on the final segmentation result.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation. Background Technology

[0002] 3D semantic segmentation is fundamental for localization, navigation, path planning, and motion control. High-quality 3D semantic segmentation results are a key prerequisite for the successful application of autonomous driving and robot navigation. In deep learning-based 3D point cloud semantic segmentation models, the grouping layer is mainly used for feature extraction and aggregation, and is an important component of the 3D point cloud semantic segmentation model.

[0003] However, the grouping methods used in 3D point cloud understanding in related technologies cannot effectively address the problems of high curvature surfaces or regions. Summary of the Invention

[0004] This application provides a method and apparatus for semantic segmentation of three-dimensional point clouds based on directional grouping and parameter adaptation.

[0005] In a first aspect, embodiments of this application provide a 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation, including:

[0006] A 3D point cloud to be processed is acquired, and the 3D point cloud is initially grouped using a first directional grouping method to obtain m groups. Each of the m groups includes k points, where m and k are positive integers.

[0007] If at least some of the m groups do not meet the grouping requirements for high curvature surfaces or regions, the parameters of the first directional grouping method are adaptively adjusted, and the at least some groups are regrouped based on the adjusted parameters using the first directional grouping method until the resulting groups meet the grouping requirements for high curvature surfaces or regions.

[0008] Based on the at least some groupings, the groupings obtained after regrouping, and other groupings, a target grouping result is obtained; wherein, the other groupings are the groupings other than the at least some groupings among the m groupings;

[0009] Semantic segmentation is performed based on the target grouping results.

[0010] Secondly, embodiments of this application provide a 3D point cloud semantic segmentation device based on directional grouping and parameter adaptation, comprising:

[0011] An initial grouping module is used to acquire the 3D point cloud to be processed and to perform initial grouping of the 3D point cloud using a first directional grouping method to obtain m groups. Each of the m groups includes k points, where m and k are positive integers.

[0012] The regrouping module is used to adaptively adjust the parameters of the first directional grouping method when at least some of the groups in the m groups do not meet the grouping requirements of the high curvature surface or region, and then regroup the at least some groups based on the adjusted parameters using the first directional grouping method until the resulting groups meet the grouping requirements of the high curvature surface or region.

[0013] The acquisition module is used to obtain a target grouping result based on the at least some groupings, the groupings obtained after regrouping, and other groupings; wherein, the other groupings are the groupings other than the at least some groupings among the m groupings;

[0014] The semantic segmentation processing module is used to perform semantic segmentation processing based on the target grouping results.

[0015] Thirdly, embodiments of this application provide an electronic device, including:

[0016] At least one processor;

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect above.

[0019] Fourthly, embodiments of this application provide a storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect above.

[0020] Fifthly, embodiments of this application provide a program product, including at least one of a program and instructions, wherein when the program and instructions are executed by a processor, they implement the steps of the method described in the first aspect.

[0021] According to the technical solution of this application, the grouping parameters of high curvature surfaces or regions can be adaptively adjusted to obtain more reasonable grouping results, which helps in subsequent feature extraction and has a positive impact on the final segmentation results.

[0022] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0023] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0024] Figure 1 This is a flowchart illustrating the 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation provided in the embodiments of this application.

[0025] Figure 2 This is a schematic diagram of the directional k-nearest neighbor grouping results provided in the embodiments of this application;

[0026] Figure 3 This is a schematic diagram of the directional ball query grouping results provided in an embodiment of this application;

[0027] Figure 4(a) shows an example of the directional serialization nearest neighbor grouping results provided in the embodiments of this application. Figure 1 ;

[0028] Figure 4(b) shows an example of the directional serialization nearest neighbor grouping results provided in the embodiments of this application. Figure 2 ;

[0029] Figure 4(c) shows an example of the directional serialization nearest neighbor grouping results provided in the embodiments of this application. Figure 3 ;

[0030] Figure 4(d) is an example of the directional serialization nearest neighbor grouping results provided in the embodiments of this application;

[0031] Figure 5 A block diagram of a 3D point cloud semantic segmentation device based on directional grouping and parameter adaptation provided in an embodiment of this application;

[0032] Figure 6 This is a block diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0033] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0034] The embodiments described in this application are not exhaustive, but merely illustrative of some embodiments, and are not intended to limit the scope of protection of this application. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined with each other. For example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0035] In each embodiment of this application, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0036] The terminology used in the embodiments of this application is for the purpose of describing specific embodiments only and is not intended to limit the scope of this application.

[0037] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0038] It is worth noting that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0039] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0040] The following describes, with reference to the accompanying drawings, a method and apparatus for semantic segmentation of 3D point clouds based on directional grouping and parameter adaptation, according to embodiments of this application.

[0041] It should be noted that the execution entity of the 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation in this application embodiment can be a 3D point cloud semantic segmentation device based on directional grouping and parameter adaptation. This device can be implemented by software and / or hardware, and can be configured in an electronic device. For example, the electronic device can include, but is not limited to, a terminal, a server, etc.

[0042] Figure 1 This is a flowchart illustrating the 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation provided in an embodiment of this application. Figure 1 As shown, the 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation may include, but is not limited to, the following steps.

[0043] In step 101, the three-dimensional point cloud to be processed is obtained, and the three-dimensional point cloud is initially grouped using the first directional grouping method to obtain m groups. Each of the m groups includes k points, where m and k are positive integers.

[0044] In some embodiments, when obtaining the 3D point cloud to be processed, the 3D point cloud can be initially grouped using a first directional grouping method. For example, using the complete 3D point cloud to be processed... For example, the coordinates of the points and characteristics , ( The 3D point cloud can be initially grouped using the first directional grouping method, i.e., sampled point cloud, with a total of m points (which can be denoted as designated points or center points). Each sampled point's group contains k points, and the shape of the grouped point cloud is as follows: , where d is the feature dimension of the point cloud.

[0045] In some embodiments, the first directional grouping method may include any one of the directional k-nearest neighbor grouping method, the directional ball query grouping method, and the directional serialized nearest neighbor grouping method. The directional k-nearest neighbor grouping method may refer to grouping a specified point with its k nearest points in terms of weighted distance. That is, the directional k-nearest neighbor grouping selects the k nearest points in terms of weighted distance to the specified point and groups them together. Figure 2 The diagram illustrates the directional k-nearest neighbor grouping results. This grouping strategy yields accurate nearest neighbor grouping, which helps ensure model performance. For example, to achieve directional k-nearest neighbor grouping, weights a, b, and c can be assigned to the x, y, and z components respectively, and a weighted distance can be used as the metric. Optionally, the formula for calculating this weighted distance is as follows:

[0046]

[0047] in, , , Representing the sampling center point respectively coordinates, , , Representing neighboring points respectively coordinates, , , Represent , , Component weights For point With point The weighted distance.

[0048] In embodiments of this application, the directional ball query grouping method may refer to grouping a specified point with k points within a weighted distance R. For example, if you want to... China For directional sphere query sampling centered on the target, it is first necessary to calculate separately Weighted distance to the remaining points Then randomly select k points are used to achieve grouping. For example... Figure 3 As shown, this is a schematic diagram of the grouping results for a directional ball query. Considering the geometric meaning, grouping a directional ball query is equivalent to filtering out results based on... Centered at the sphere, the semi-principal axis lengths are respectively , , Points within the ellipsoid, therefore, directional spherical query groups can also be called "ellipsoidal query groups".

[0049] In embodiments of this application, the directional serialization nearest neighbor grouping method can refer to encoding a 3D point cloud using a space-filling curve and then grouping the k nearest points along the path of the space-filling curve into a group. Optionally, the space-filling curve used in this directional serialization nearest neighbor grouping method can be a directional Z-curve or a Hilbert curve. It should be noted that the space-filling curve is a path through each point in a high-dimensional discrete space and can maintain spatial proximity relationships to a certain extent. Mathematically, this space-filling curve can be represented by a bijective function. Serialization encoding is performed on the point cloud, that is, the 3D point cloud coordinates are converted into serialized integer codes. First, project the 3D point cloud onto a grid with a size of [size missing]. Encode it in the discrete space: Since points that are nearest neighbors on the path of a spatial filling curve also have a certain proximity relationship in space, this proximity relationship can be used to approximate the spatial proximity relationship, providing the possibility for efficient neighborhood search of point clouds. After encoding the 3D point cloud using a spatial filling curve, the k nearest neighbors on the path of the spatial filling curve can be grouped together.

[0050] As an example, the space-filling curve used in this directional serialization nearest neighbor grouping method can be a directional Z-curve. The core idea of ​​the directional Z-curve is to encode multidimensional coordinates into a one-dimensional value by alternatingly cutting space, where the lower three bits of the encoded value are composed of interleaved binary low-order bits of the x, y, and z coordinates. For Z-curve-based grouping methods, by changing the number of binary bits in the x, y, and z alternations, more points can be sampled in a certain direction. For example, changing "x(1bit)-y(1bit)-z(1bit)" to "x(1bit)-y(1bit)-z(2bit)" allows sampling more points in the z direction than in the x and y directions. Figure 4(a) shows an example of the directional serialization nearest neighbor grouping result based on Z-curve encoding (xyz order), Figure 4(b) shows an example of the directional serialization nearest neighbor grouping result based on Z-curve encoding (xyyz order), Figure 4(c) shows an example of the directional serialization nearest neighbor grouping result based on Z-curve encoding (xxyz order), and Figure 4(d) shows an example of the directional serialization nearest neighbor grouping result based on Z-curve encoding (xyzz order).

[0051] In step 102, if at least some of the m groups do not meet the grouping requirements for high curvature surfaces or regions, the parameters of the first directional grouping method are adaptively adjusted, and the first directional grouping method is used to regroup at least some of the groups based on the adjusted parameters until the resulting groups meet the grouping requirements for high curvature surfaces or regions.

[0052] In some embodiments, it can be determined whether any of the m groups does not meet the grouping requirements for high-curvature surfaces or regions. Optionally, if at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions, the parameters of the first directional grouping method can be adaptively adjusted, and the groups can be regrouped according to the adjusted parameters. Optionally, for the group currently to be regrouped, the points used by the group in the initial grouping can continue to be used as the center point, and the first directional grouping method can be used to sample the surrounding points of the center point based on the adjusted parameters to achieve regrouping of the current group. It is worth noting that the 3D point cloud may be divided into one or more different groups.

[0053] In some embodiments, an optional implementation of determining that at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions includes: determining the point cloud covariance matrix within each group; and determining that at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions based on the eigenvalues ​​of the covariance matrix decomposition. For example, a group not meeting the grouping requirements for high-curvature surfaces or regions may include: the maximum value among the eigenvalues ​​of the point cloud covariance matrix decomposition within the group being greater than or equal to a threshold. That is: for each group, if the maximum value among the eigenvalues ​​of the point cloud covariance matrix decomposition within the current group is greater than or equal to the threshold, then the current group does not meet the grouping requirements for high-curvature surfaces or regions, and the parameters of the first directional grouping method need to be adjusted to regroup the points within the current group; if the maximum value among the eigenvalues ​​of the point cloud covariance matrix decomposition within the current group is less than the threshold, then the current group is considered to meet the grouping requirements for high-curvature surfaces or regions, and there is no need to regroup the current group. Optionally, as an example, the formula for calculating the point cloud covariance matrix within a group is expressed as follows:

[0054]

[0055] in, The point cloud covariance matrix within the group; For the first in the group i The coordinates of each point; This represents the average coordinates of the point cloud within the group.

[0056] For example, the covariance matrix can be decomposed into eigenvalues, as shown in the following formula:

[0057]

[0058] in, It is an orthogonal matrix whose column vectors are the eigenvectors of the covariance matrix; It is a diagonal matrix, and the elements on the diagonal are eigenvalues.

[0059] In the embodiments of this application, for each group, if the maximum value among the eigenvalues ​​of the covariance matrix decomposition within that group is greater than or equal to a threshold, the point cloud within that group can be regrouped using a first directional grouping method. A larger receptive field is set for the axis (or coordinate component) corresponding to the maximum eigenvalue, while the receptive fields of the other two axes (or other two coordinate components) remain unchanged. Optionally, a method for obtaining a larger receptive field is as follows:

[0060] 1) For directional k-nearest neighbor grouping method or directional spherical query grouping method

[0061] For groups that do not meet the grouping requirements for high-curvature surfaces or regions, the weights of the coordinate components corresponding to the maximum value in the covariance matrix decomposition within that group can be adjusted as parameters for the first directional grouping method. As an example, the weight of the coordinate component corresponding to the maximum value can be increased by a first value.

[0062] For example, suppose that the maximum value of the eigenvalues ​​in the covariance matrix decomposition of a certain group A among m groups is greater than a threshold, and the coordinate component corresponding to the maximum eigenvalue is the y-component. Then, the weight b of the y-component can be adjusted. For example, the initial value of weight b is 1, and a first value can be added to the value of weight b. For example, the first value can be a fixed value (such as 1). This application does not specifically limit the method of adjusting the weight of the coordinate component, and other methods can also be used to adjust the weight. For example, the size of the weight b of the y-component can be adjusted based on the pre-set mapping relationship between the weight value and the adjustment value. For example, the larger the maximum eigenvalue, the larger the adjustment range of weight b. After the adjustment of the weight b of the y-component is completed, the point cloud in group A can be regrouped based on the size of weight a, weight c, and the adjusted weight b. For example, sampling can continue to be performed in the point cloud around the center point of group A to obtain a new group A1. After regrouping, it can be further determined whether the regrouped groups meet the grouping requirements of high curvature surfaces or regions. If not, the parameters of the first directional grouping method are adaptively adjusted until the obtained groups meet the grouping requirements of high curvature surfaces or regions.

[0063] 2) For the directional serialized nearest neighbor grouping method, the space-filling curve used is the directional Z-curve.

[0064] For groups that do not meet the grouping requirements for high-curvature surfaces or regions, the encoding bit depth and / or order parameters of the first directional grouping method can be adjusted based on the maximum value among the eigenvalues ​​of the covariance matrix decomposition within that group. As an example, the encoding bit depth of the coordinate component direction corresponding to the maximum value can be increased by a first number of bits to adjust the encoding bit depth and order parameters of the first directional grouping method. For instance, the first number can be 1, or it can be other values, such as being determined based on the specific magnitude of the maximum value—for example, the larger the maximum value, the larger the first number, but this is not limited to this.

[0065] For example, suppose that the maximum value of the eigenvalues ​​in the covariance matrix decomposition of a certain group A among m groups is greater than a threshold, and the coordinate component corresponding to this maximum eigenvalue is the y-component. Then, the encoding bit depth and / or encoding order in the y-component direction can be adjusted. With the initial encoding order being xyz and each component direction having 1 bit depth, when it is determined that the point cloud within group A needs to be regrouped, the encoding bit depth of the y-component corresponding to the maximum value can be changed from 1 bit to 2 bits, and the encoding order changed to xyyz. The adjusted grouping parameters can then be used to regroup the point cloud within group A. For example, sampling can continue in the point cloud near the center point of group A to obtain a new group A1. After regrouping, it can be determined whether the regrouped groups meet the grouping requirements for high-curvature surfaces or regions. If not, the parameters of the first directional grouping method are adaptively adjusted until the obtained groups meet the grouping requirements for high-curvature surfaces or regions.

[0066] In step 103, the target grouping result is obtained based on at least some grouping, the grouping obtained after regrouping, and other grouping; wherein, other grouping refers to the grouping other than at least some grouping among the m groupings.

[0067] In some embodiments, each group in at least a portion of the groups can be merged with the groups obtained from their regrouping to obtain merged groups associated with at least a portion of the groups; each merged group is sampled at k points to obtain corresponding sampled groups, and each sampled group and other groups are used as the target grouping result.

[0068] For example, consider m groups, including group A, group B, and group C. Groups A and B have been regrouped once, while group C has not been regrouped. The regrouped group A is denoted as A1, and the regrouped group B is denoted as B1. The regrouped groups A1 and B1 meet the grouping requirements for high-curvature surfaces or regions, and further regrouping is unnecessary. In this case, groups A and A1 can be merged, such as by performing a union operation. Groups B and B1 can be combined into a union. You can merge group A Perform sampling processing on k points, and merge group B. Perform sampling at k points to ensure that each group in the grouping result includes k points. This will be done from the merged group A. The k points sampled are taken as a sampling group, and will be merged from group B. The k points sampled are taken as a sampling group. These two sampling groups and group C are taken as the target grouping result, which is the final grouping result. The grouping ends here.

[0069] In step 104, semantic segmentation is performed based on the target grouping results.

[0070] In the embodiments of this application, after the grouping is completed, the points in each group of the target grouping result can be subjected to subsequent feature extraction operations in order to continue the subsequent semantic segmentation processing.

[0071] In the above embodiments, this application achieves directional grouping by optimizing the grouping module in the existing 3D point cloud semantic segmentation model. This grouping method optimizes the grouping effect for high-curvature surfaces or regions by adaptively adjusting the grouping parameters. Specifically, this application designs a directional grouping method that can achieve directional grouping of different degrees and directions with almost no increase in model complexity, and is compatible with traditional 3D point cloud semantic segmentation processes. Furthermore, this application provides an adaptive directional grouping parameter adjustment method that can adapt to surfaces or regions with different curvatures.

[0072] Figure 5 This is a block diagram of a 3D point cloud semantic segmentation device based on directional grouping and parameter adaptation, provided in an embodiment of this application. Figure 5 As shown, the 3D point cloud semantic segmentation device based on directional grouping and parameter adaptation may include: an initial grouping module 501, a regrouping module 502, an acquisition module 503, and a semantic segmentation processing module 504.

[0073] The initial grouping module 501 is used to acquire the three-dimensional point cloud to be processed and to perform initial grouping of the three-dimensional point cloud using the first directional grouping method to obtain m groups. Each of the m groups includes k points, where m and k are positive integers.

[0074] The regrouping module 502 is used to adaptively adjust the parameters of the first directional grouping method when at least some of the groups in the m groups do not meet the grouping requirements of the high curvature surface or region, and then regroup the at least some groups based on the adjusted parameters using the first directional grouping method until the resulting groups meet the grouping requirements of the high curvature surface or region.

[0075] The acquisition module 503 is used to obtain the target grouping result based on at least some grouping, the grouping obtained after regrouping, and other grouping; wherein, other grouping refers to the grouping other than at least some grouping among the m groupings.

[0076] The semantic segmentation processing module 504 is used to perform semantic segmentation processing based on the target grouping results.

[0077] In some embodiments, the regrouping module 502 is used to: determine the point cloud covariance matrix within each group; and determine, based on the eigenvalues ​​of the covariance matrix decomposition, that at least some of the m groups do not meet the grouping requirements for high curvature surfaces or regions.

[0078] In some embodiments, the regrouping module 502 is configured to: for each group, determine that the group does not meet the grouping requirements for high-curvature surfaces or regions if the maximum value among the eigenvalues ​​of the covariance matrix decomposition within the group is greater than or equal to a threshold. In one possible implementation, the formula for calculating the point cloud covariance matrix within the group is expressed as follows:

[0079]

[0080] in, The point cloud covariance matrix within the group; For the first in the group i The coordinates of each point; This represents the average coordinates of the point cloud within the group.

[0081] In some embodiments, the first directional grouping method includes any one of the directional k-nearest neighbor grouping method, the directional ball query grouping method, and the directional serialized nearest neighbor grouping method; wherein, the directional k-nearest neighbor grouping method refers to grouping a specified point with the k points closest to it by weighted distance; the directional ball query grouping method refers to grouping a specified point with the k points within a weighted distance R; and the directional serialized nearest neighbor grouping method refers to grouping the k nearest points on the path of the space-filling curve into a group after encoding the 3D point cloud using a space-filling curve.

[0082] In some embodiments, the first directional grouping method includes a directional k-nearest neighbor grouping method or a directional spherical query grouping method. In embodiments of this application, the regrouping module 502 is used to: for groups that do not meet the grouping requirements of high-curvature surfaces or regions, adjust the weights of the coordinate components corresponding to the maximum value in the eigenvalues ​​of the covariance matrix decomposition within the group, as an adjustment to the parameters of the first directional grouping method. In one possible implementation, the regrouping module 502 is used to: increase the weight of the coordinate component corresponding to the maximum value by a first value.

[0083] In some embodiments, the first directional grouping method includes a directional serialized nearest neighbor grouping method, and the space-filling curve used is a directional Z-curve. In embodiments of this application, the regrouping module 502 is configured to: for groups that do not meet the grouping requirements of high-curvature surfaces or regions, adjust the encoding bit length and / or order parameters of the first directional grouping method based on the maximum value among the eigenvalues ​​of the covariance matrix decomposition within the group. In one possible implementation, the regrouping module 502 is configured to: increase the encoding bit length of the coordinate component direction corresponding to the maximum value by a first number of bits to adjust the encoding bit length and order parameters of the first directional grouping method.

[0084] In some embodiments, the acquisition module 503 is configured to: merge each group in at least a portion of the groups with the groups obtained by regrouping them to obtain each merged group associated with at least a portion of the groups; perform k-point sampling processing on each merged group to obtain each corresponding sampled group; and use each sampled group and other groups as the target grouping result.

[0085] It should be noted that the foregoing explanation of the embodiment of the 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation also applies to the 3D point cloud semantic segmentation device based on directional grouping and parameter adaptation in this embodiment, and will not be repeated here.

[0086] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0087] like Figure 6 The diagram shown is a block diagram of an electronic device according to an embodiment of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0088] like Figure 6As shown, the electronic device includes one or more processors 601, a memory 602, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required. The processors can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 6 Take the 601 processor as an example.

[0089] The memory 602 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor to cause the at least one processor to execute the 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to execute the 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation provided in this application.

[0090] Memory 602, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation in the embodiments of this application (e.g., attached...). Figure 5 The initial grouping module 501, regrouping module 502, acquisition module 503, and semantic segmentation processing module 504 are shown. The processor 601 executes various server functions and data processing by running non-transient software programs, instructions, and modules stored in the memory 602, thereby realizing the 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation in the above method embodiments.

[0091] Memory 602 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device. Furthermore, memory 602 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 602 may optionally include memory remotely located relative to processor 601, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0092] The electronic device may also include an input device 603 and an output device 604. The processor 601, memory 602, input device 603, and output device 604 can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0093] Input device 603 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the electronic device, such as touch screens, keypads, mice, trackpads, touchpads, joysticks, one or more mouse buttons, trackballs, joysticks, etc. Output device 604 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The display device may include, but is not limited to, liquid crystal displays (LCDs), light-emitting diode (LED) displays, and plasma displays. In some embodiments, the display device may be a touch screen.

[0094] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0095] These computational programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0096] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0097] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet, and blockchain networks.

[0098] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0099] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0100] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A 3D point cloud semantic segmentation method based on directional grouping and parameter adaptation, characterized in that, Includes the following steps: A 3D point cloud to be processed is acquired, and the 3D point cloud is initially grouped using a first directional grouping method to obtain m groups. Each of the m groups includes k points, where m and k are positive integers. If at least some of the m groups do not meet the grouping requirements for high curvature surfaces or regions, the parameters of the first directional grouping method are adaptively adjusted, and the at least some groups are regrouped based on the adjusted parameters using the first directional grouping method until the resulting groups meet the grouping requirements for high curvature surfaces or regions. Based on the at least some groupings, the groupings obtained after regrouping, and other groupings, a target grouping result is obtained; wherein, the other groupings are the groupings other than the at least some groupings among the m groupings; Semantic segmentation is performed based on the target grouping results; Wherein, the first directional grouping method includes any one of the directional k-nearest neighbor grouping method, the directional sphere query grouping method, and the directional serialized nearest neighbor grouping method; when the first directional grouping method includes the directional k-nearest neighbor grouping method or the directional sphere query grouping method, the adaptive adjustment of the parameters of the first directional grouping method includes: for groups that do not meet the grouping requirements of the high curvature surface or region, adjusting the weight of the coordinate component corresponding to the maximum value in the eigenvalues ​​of the covariance matrix decomposition within the group, as an adjustment of the parameters of the first directional grouping method; When the first directional grouping method includes the directional serialized nearest neighbor grouping method, the space-filling curve used is the directional Z-curve; the adaptive adjustment of the parameters of the first directional grouping method includes: for groups that do not meet the grouping requirements of the high curvature surface or region, adjusting the encoding bit length and / or order parameters of the first directional grouping method based on the maximum value of the eigenvalues ​​of the covariance matrix decomposition within the group.

2. The method according to claim 1, characterized in that, Determining that at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions includes: Determine the point cloud covariance matrix within each group; Based on the eigenvalues ​​of the covariance matrix decomposition, it is determined that at least some of the m groups do not meet the grouping requirements for high curvature surfaces or regions.

3. The method according to claim 2, characterized in that, The step of determining, based on the eigenvalues ​​of the covariance matrix decomposition, that at least some of the m groups do not meet the grouping requirements for high-curvature surfaces or regions includes: For each group, if the maximum value of the eigenvalues ​​of the covariance matrix decomposition within the group is greater than or equal to a threshold, the group is determined to not meet the grouping requirements of the high curvature surface or region.

4. The method according to claim 2, characterized in that, The formula for calculating the point cloud covariance matrix within the group is as follows: in, The point cloud covariance matrix within the group; For the first in the group i The coordinates of each point; The mean of the point cloud coordinates within the group is given.

5. The method according to any one of claims 1-4, characterized in that, in, The directional k-nearest neighbor grouping method refers to grouping a specified point with the k points that are closest to it by weighted distance. The directional ball query grouping method refers to grouping a specified point with k points within a weighted distance R into a group; The directional serialized nearest neighbor grouping method refers to encoding a 3D point cloud using a space-filling curve, and then grouping the k nearest points along the path of the space-filling curve into a group.

6. The method according to claim 1, characterized in that, The adjustment of the weights of the coordinate components corresponding to the maximum value includes: Increase the weight of the coordinate component corresponding to the maximum value by the first value.

7. The method according to claim 1, characterized in that, The adjustment of the encoding bit length and / or order parameters of the first directional grouping method includes: Increase the number of bits in the encoding of the coordinate component direction corresponding to the maximum value by a first number of bits to adjust the encoding bit length and order parameters of the first directional grouping method.

8. The method according to claim 1, characterized in that, The process of obtaining the target grouping result based on the at least partial grouping, the grouping obtained after regrouping, and other groupings includes: Each group in the at least partial grouping is merged with the grouping obtained by regrouping to obtain each merged group associated with the at least partial grouping; Each merged group is sampled at k points to obtain corresponding sampled groups, and each sampled group and the other groups are used as the target group result.

Images