Topology-aware point cloud segmentation network construction method, segmentation method and device

Abstract

Embodiments of the present application provide a topology-aware point cloud segmentation network construction method, a segmentation method and an apparatus. The topology-aware point cloud segmentation network construction method comprises: constructing a total loss function corresponding to a point cloud segmentation network, the total loss function being a weighted sum of a cross-entropy loss term and a topology loss term; adding the total loss function to a to-be-trained point cloud segmentation network to constitute a topology loss constraint branch of the to-be-trained point cloud segmentation network; training the to-be-trained point cloud segmentation network using training data and optimizing parameters of the to-be-trained point cloud segmentation network through the topology loss constraint branch to obtain a target point cloud segmentation network. The technical solution of the embodiments of the present application can improve the accuracy of the point cloud segmentation network in topology structure segmentation and ensure the segmentation effect of the point cloud segmentation network.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a method, method and apparatus for constructing a point cloud segmentation network based on topology awareness. Background Technology

[0002] In computer 3D vision, semantic segmentation of point clouds is a crucial task. The problem of edge extraction from point clouds can be transformed into a semantic segmentation problem through edge detection. Current technologies employ numerous end-to-end deep learning-based 3D point cloud segmentation networks that directly learn discriminative features from point cloud data, achieving relatively good segmentation results. However, these networks are still prone to errors in fine-grained segmentation, particularly topological errors. Therefore, improving the accuracy of point cloud segmentation networks in topological segmentation and ensuring consistent segmentation performance has become a pressing technical challenge. Summary of the Invention

[0003] The embodiments of this application provide a topology-aware point cloud segmentation network construction method, segmentation method, and apparatus, which can at least to some extent improve the accuracy of point cloud segmentation network segmentation in terms of topology structure and ensure the segmentation effect of point cloud segmentation network.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to one aspect of the embodiments of this application, a method for constructing a topology-aware point cloud segmentation network is provided, the method comprising:

[0006] Construct the total loss function corresponding to the point cloud segmentation network, wherein the total loss function is a weighted sum of the cross-entropy loss term and the topology loss term;

[0007] The total loss function is added to the point cloud segmentation network to be trained to form the topology loss constraint branch of the point cloud segmentation network to be trained.

[0008] The point cloud segmentation network to be trained is trained using training data, and the parameters of the point cloud segmentation network to be trained are optimized through the topology loss constraint branch to obtain the target point cloud segmentation network.

[0009] According to one aspect of the embodiments of this application, a topology-aware point cloud segmentation method is provided, the method comprising:

[0010] Feature extraction is performed on the identified point cloud data to obtain the corresponding point-by-point features;

[0011] The point-by-point features are input into a pre-trained target point cloud segmentation network so that the target point cloud segmentation network outputs the segmentation result corresponding to the point cloud data. The target point cloud segmentation network is constructed using the topology-aware point cloud segmentation network construction method described in the above embodiments.

[0012] According to one aspect of the embodiments of this application, a topology-aware point cloud segmentation network construction apparatus is provided, the apparatus comprising:

[0013] The construction module is used to construct the total loss function corresponding to the point cloud segmentation network. The total loss function is a weighted sum of the cross-entropy loss term and the topology loss term.

[0014] An addition module is used to add the total loss function to the point cloud segmentation network to be trained, so as to form the topology loss constraint branch of the point cloud segmentation network to be trained;

[0015] The training module is used to train the point cloud segmentation network to be trained using training data, and to optimize the parameters of the point cloud segmentation network to be trained using the topology loss constraint branch, so as to obtain the target point cloud segmentation network.

[0016] According to one aspect of the embodiments of this application, a topology-aware point cloud segmentation apparatus is provided, the apparatus comprising:

[0017] The extraction module is used to extract features from the identified point cloud data and obtain the corresponding point-by-point features;

[0018] The processing module is used to input the point-by-point features into a pre-trained target point cloud segmentation network, so that the target point cloud segmentation network outputs the segmentation result corresponding to the point cloud data. The target point cloud segmentation network is constructed using the topology-aware point cloud segmentation network construction method described in the above embodiments.

[0019] According to one aspect of the embodiments of this application, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the topology-aware point cloud segmentation network construction and point cloud segmentation method as described in the above embodiments.

[0020] According to one aspect of the embodiments of this application, an electronic device is provided, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the topology-aware point cloud segmentation network construction and point cloud segmentation method as described in the above embodiments.

[0021] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the topology-aware point cloud segmentation network construction and point cloud segmentation method provided in the above embodiments.

[0022] In some embodiments of this application, a total loss function corresponding to the point cloud segmentation network is constructed. This total loss function is a weighted sum of cross-entropy loss and topology loss terms. This total loss function is added to the point cloud segmentation network to be trained to form a topology loss constraint branch. Training data is then used to train the point cloud segmentation network to optimize its parameters through the topology loss constraint branch. Based on the topology of the segmentation results output by the point cloud segmentation network, the parameters of the point cloud segmentation network are adjusted to obtain the target point cloud segmentation network. Therefore, by constructing the topology loss constraint branch, the parameters of the point cloud segmentation network can be optimized, thereby improving the accuracy of the segmentation results output by the point cloud segmentation network in terms of topology, and thus ensuring the segmentation effect of the point cloud segmentation network.

[0023] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0025] Figure 1 A flowchart illustrating a topology-aware point cloud segmentation network construction method according to an embodiment of this application is shown.

[0026] Figure 2 A schematic diagram of the structure of a target point cloud segmentation network according to an embodiment of this application is shown;

[0027] Figure 3 A block diagram of a topology-aware point cloud segmentation network construction apparatus according to an embodiment of this application is shown.

[0028] Figure 4A block diagram of a topology-aware point cloud segmentation apparatus according to an embodiment of this application is shown;

[0029] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0030] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0031] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0032] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0033] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0034] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0035] Figure 1 A flowchart illustrating a topology-aware point cloud segmentation network construction method according to an embodiment of this application is shown. This method can be executed by a server, which can be a physical server or a cloud server. (Refer to...) Figure 1 As shown, the topology-aware point cloud segmentation network construction method includes at least steps S110 to S130, which are described in detail below:

[0036] In step S110, a total loss function corresponding to the point cloud segmentation network is constructed. The total loss function is a weighted sum of the cross-entropy loss term and the topology loss term.

[0037] Specifically, in one embodiment of this application, constructing the total loss function corresponding to the point cloud segmentation network includes:

[0038] Based on the simplex, simplex complex, chain group, edge operator and homology group in algebraic topology, a persistent homology is constructed, and a persistence graph is used to describe the homology group changes corresponding to the 3D point cloud data.

[0039] For a fixed simplicial complex, a topological loss is constructed using a filtering sequence of hyperlevel sets, and the EMD distance is used to measure the similarity between two 2D point sets. The EMD distance between 2D point sets S1 and S2 is calculated using the following formula:

[0040]

[0041] Where Φ: S1→S2 is a bijection, and EMD distance measures the minimum cost of transforming point set S1 into S2.

[0042] For k-persistent graph PD k yes The multiset, viewed as a sequence from the filtered sequence to Mapping of point sets:

[0043] PD k : PD k Each point {b} in i ,d i} corresponds to a k-dimensional topological feature, when the scale parameter α = b i This topological feature is generated when the scale parameter α = d. i This topological feature is generated at that time;

[0044] The topology loss term for the i-th binary classification is calculated using the following formula:

[0045]

[0046] in, For the persistence graph PD corresponding to the point set f k (f) and the persistence graph PD corresponding to the point set g k The optimal match for (g);

[0047] The total loss function for the point cloud segmentation network is constructed based on the topology loss term and the cross-entropy loss term, as shown in the following formula:

[0048] L(f,G)=L ce (f seg,G)+λL topo (f topo ,G),

[0049]

[0050] Where G represents the real label, G i For the true label of the i-th item, L ce f is the cross-entropy loss term. seg To address the probability output of point cloud segmentation networks for multi-class classification, f topo For the topological loss constraint branch, the binary classification probability output for all C classes, L topo λ is the average of the topology loss for all categories, and λ is the weight of the topology loss.

[0051] In this embodiment, the real labels can be pre-set by those skilled in the art. For example, for the point cloud data corresponding to the table, the real labels can be divided into the tabletop, table legs, etc.

[0052] In an exemplary embodiment of this application, when PD k (f) and PD k When the number of points in (g) is different, if PD k (f) The number of points is greater than PD k The number of points in (g) then applies to PD. k (f) The closest to (1,0) and PD k (g) Match the same number of points as (1,0), and match the remaining points with the diagonal.

[0053] If PD k (f) Points less than or equal to PD k The number of points in (g) is then PD k All points in (f) are matched with (1,0), and the extra real marker points are matched with the diagonal.

[0054] It should be noted that this matching algorithm is more efficient than the optimal matching algorithm for any point set, as it only requires adjustments to the PD. k Sort the distances from the points in (f) to (1,0) with a time complexity of O(n log n).

[0055] Please continue to refer to this. Figure 1 In step S120, the total loss function is added to the point cloud segmentation network to be trained to form the topology loss constraint branch of the point cloud segmentation network to be trained.

[0056] In this embodiment, by adding the total loss function to the point cloud segmentation network to be trained, a topology loss constraint branch is formed. This topology loss constraint function can adjust the weights of the original segmentation structure. In this way, the topology loss can be easily and quickly added to any existing segmentation network that can provide point-by-point prediction for training, thereby correcting the topology structure of the segmentation error.

[0057] In step S130, the point cloud segmentation network to be trained is trained using training data, and the parameters of the point cloud segmentation network to be trained are optimized through the topology loss constraint branch to obtain the target point cloud segmentation network.

[0058] In one embodiment of this application, the parameters of the point cloud segmentation network to be trained are optimized through the topological loss constraint branch, including:

[0059] Define a persistence graph PD k (f) Mapping h from persistent points to points in the original point cloud k :

[0060] h k :{b i ,d i}→(cp(σ b ),cp(σ d )),

[0061] Among them, {b i ,d i} is the persistence graph PD k The point in (f), σ b and σ d These are the key simplexes that lead to the birth and extinction of the corresponding topological features, cp(σ) b ), cp(σ d To determine the simplex σ b and σ d Key points regarding function values;

[0062] Calculating topological loss for persistent graph PD k Each point {b} in (f) i ,d i The gradient of} is calculated, and this gradient is then applied along the corresponding keypoint {b}. i ,d i Backpropagation is performed to optimize the parameters of the point cloud segmentation network, where the gradient of the topology loss is calculated using the following formula:

[0063]

[0064] in, Represents the topological loss Ltopo The gradient calculation of the i-th term, cb(p) and cd(p) represent the birth keypoint and death keypoint of the persistent point p in the original point cloud, respectively, f(·) is the output value of the neural network, and w is the parameter of the neural network.

[0065] Please refer to Figure 2 Based on the foregoing embodiments, this application also provides a topology-aware point cloud segmentation method, which includes:

[0066] Feature extraction is performed on the identified point cloud data to obtain the corresponding point-by-point features;

[0067] The point-by-point features are input into a pre-trained target point cloud segmentation network so that the target point cloud segmentation network outputs the segmentation result corresponding to the point cloud data. The target point cloud segmentation network is constructed using the topology-aware point cloud segmentation network construction method described in the above embodiments.

[0068] In this embodiment, combined with Figure 2 As shown, feature extraction is performed on the point cloud data using an encoder-decoder to obtain corresponding point-by-point features. These point-by-point features are then input into the target point cloud segmentation model, which includes a first branch and a second branch. The first branch is a topological constraint branch, which further extracts topologically corresponding features through multiple MLP layers. A sigmoid layer is used to perform binary classification probability prediction for each class. The probability output f of this first branch is... topo It participates in the calculation of topological loss.

[0069] The second branch uses multiple MLP layers to convert the output f of the topology constraint branch. topo The weights are multiplied by the output of the last fully connected layer, and then a Softmax layer is used to calculate the score f for each category. seg And calculate the cross-entropy loss.

[0070] After adopting the method provided in the above embodiments, topology loss constraint can effectively improve the segmentation effect of the original 3D point cloud segmentation network, and can also effectively correct topological errors such as breaks, and can completely segment some connected component parts, providing support for the application of accurate point cloud segmentation in artificial intelligence.

[0071] The following describes an apparatus embodiment of this application, which can be used to execute the topology-aware point cloud segmentation network construction method and the topology-aware point cloud segmentation method described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the topology-aware point cloud segmentation network construction method and the topology-aware point cloud segmentation method described above in this application.

[0072] Figure 3 A block diagram of a topology-aware point cloud segmentation network construction apparatus according to an embodiment of this application is shown.

[0073] Reference Figure 3 As shown, a topology-aware point cloud segmentation network construction apparatus according to an embodiment of this application includes:

[0074] The construction module 310 is used to construct the total loss function corresponding to the point cloud segmentation network, wherein the total loss function is a weighted sum of the cross-entropy loss term and the topology loss term;

[0075] Add module 320, used to add the total loss function to the point cloud segmentation network to be trained, so as to form the topology loss constraint branch of the point cloud segmentation network to be trained;

[0076] The training module 330 is used to train the point cloud segmentation network to be trained using training data, and to optimize the parameters of the point cloud segmentation network to be trained using the topology loss constraint branch, so as to obtain the target point cloud segmentation network.

[0077] In one embodiment of this application, the construction module 310 is used for:

[0078] Based on the simplex, simplex complex, chain group, edge operator and homology group in algebraic topology, a persistent homology is constructed, and a persistence graph is used to describe the homology group changes corresponding to the 3D point cloud data.

[0079] For a fixed simplicial complex, a topological loss is constructed using a filtering sequence of hyperlevel sets, and the EMD distance is used to measure the similarity between two 2D point sets. The EMD distance between 2D point sets S1 and S2 is calculated using the following formula:

[0080]

[0081] Where Φ: S1→S2 is a bijection, and EMD distance measures the minimum cost of transforming point set S1 into S2.

[0082] For k-persistent graph PD k yes The multiset, viewed as a sequence from the filtered sequence to Mapping of point sets:

[0083] PD k : PD k Each point {b} in i ,d i} corresponds to a k-dimensional topological feature, when the scale parameter α = b iThis topological feature is generated when the scale parameter α = d. i This topological feature is generated at that time;

[0084] The topology loss term for the i-th binary classification is calculated using the following formula:

[0085]

[0086] in, For the persistence graph PD corresponding to the point set f k (f) and the persistence graph PD corresponding to the point set g k The optimal match for (g);

[0087] The total loss function for the point cloud segmentation network is constructed based on the topology loss term and the cross-entropy loss term, as shown in the following formula:

[0088] L(f,G)=L ce (f seg ,G)+λL topo (f topo ,G),

[0089]

[0090] Where G represents the real label, G i For the true label of the i-th item, L ce f is the cross-entropy loss term. seg To address the probability output of point cloud segmentation networks for multi-class classification, f topo For the topological loss constraint branch, the binary classification probability output for all C classes, L topo λ is the average of the topology loss for all categories, and λ is the weight of the topology loss.

[0091] In one embodiment of this application, the construction module 310 is used to: when PD k (f) and PD k When the number of points in (g) is different, if PD k (f) The number of points is greater than PD k The number of points in (g) then applies to PD. k (f) The closest to (1,0) and PD k (g) Match the same number of points as (1,0), and match the remaining points with the diagonal.

[0092] If PD k (f) Points less than or equal to PD k The number of points in (g) is then PD k All points in (f) are matched with (1,0), and the extra real marker points are matched with the diagonal.

[0093] In one embodiment of this application, the training module 330 is used for:

[0094] Define a persistence graph PD k (f) Mapping h from persistent points to points in the original point cloud k :

[0095] h k :{b i ,d i}→(cp(σ b ),cp(σ d )),

[0096] Among them, {b i ,d i} is the persistence graph PD k The point in (f), σ b and σ d These are the key simplexes that lead to the birth and extinction of the corresponding topological features, cp(σ) b ), cp(σ d To determine the simplex σ b and σ d Key points regarding function values;

[0097] Calculating topological loss for persistent graph PD k Each point {b} in (f) i ,d i The gradient of} is calculated, and this gradient is then applied along the corresponding keypoint {b}. i ,d i Backpropagation is performed to optimize the parameters of the point cloud segmentation network, where the gradient of the topology loss is calculated using the following formula:

[0098]

[0099] in, Represents the topological loss L topo The gradient calculation of the i-th term, cb(p) and cd(p) represent the birth keypoint and death keypoint of the persistent point p in the original point cloud, respectively, f(·) is the output value of the neural network, and w is the parameter of the neural network.

[0100] Figure 4 A block diagram of a topology-aware point cloud segmentation apparatus according to an embodiment of this application is shown.

[0101] Reference Figure 4 As shown, a topology-aware point cloud segmentation apparatus according to an embodiment of this application includes:

[0102] The extraction module 410 is used to extract features from the identified point cloud data and obtain the corresponding point-by-point features;

[0103] The processing module 420 is used to input the point-by-point features into a pre-trained target point cloud segmentation network, so that the target point cloud segmentation network outputs the segmentation result corresponding to the point cloud data. The target point cloud segmentation network is constructed using the topology-aware point cloud segmentation network construction method described in the above embodiments.

[0104] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0105] It should be noted that, Figure 5 The computer system of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0106] like Figure 5 As shown, the computer system includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 502 or programs loaded from storage portion 508 into Random Access Memory (RAM) 503, such as performing the methods described in the above embodiments. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An Input / Output (I / O) interface 505 is also connected to the bus 504.

[0107] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. Removable media 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 510 as needed so that computer programs read from them can be installed into storage section 508 as needed.

[0108] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs various functions defined in the system of this application.

[0109] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0110] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0111] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0112] In another aspect, this application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods described in the above embodiments.

[0113] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0114] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0115] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0116] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for constructing a point cloud segmentation network based on topology awareness, characterized in that, include: Construct the total loss function corresponding to the point cloud segmentation network, wherein the total loss function is a weighted sum of the cross-entropy loss term and the topology loss term; The total loss function is added to the point cloud segmentation network to be trained to form the topology loss constraint branch of the point cloud segmentation network to be trained. The point cloud segmentation network to be trained is trained using training data, and the parameters of the point cloud segmentation network to be trained are optimized through the topology loss constraint branch to obtain the target point cloud segmentation network. The total loss function for constructing the point cloud segmentation network includes: Based on the simplex, simplex complex, chain group, edge operator and homology group in algebraic topology, a persistent homology is constructed, and a persistence graph is used to describe the homology group changes corresponding to the 3D point cloud data. For a fixed simplicial complex, a topological loss is constructed using a filtering sequence of hyperlevel sets, and the EMD distance is used to measure the similarity between two 2D point sets. The EMD distance between 2D point sets S1 and S2 is calculated using the following formula: Where Φ: S1→S2 is a bijection, and EMD distance measures the minimum cost of transforming point set S1 into S2. For k-persistent graph PD k yes The multiset, viewed as a sequence from the filtered sequence to Mapping of point sets: PD k Each point {b} in i ,d i } corresponds to a k-dimensional topological feature, when the scale parameter α = b i This topological feature is generated when the scale parameter α = d. i This topological feature is generated at that time; The topology loss term for the i-th binary classification is calculated using the following formula: in, For the persistence graph PD corresponding to the point set f k (f) and the persistence graph PD corresponding to the point set g k The optimal match for (g); The total loss function for the point cloud segmentation network is constructed based on the topology loss term and the cross-entropy loss term, as shown in the following formula: L(f,G)=L ce (f seg ,G)+λL topo (f topo ,G), Where G represents the real label, G i For the true label of the i-th item, L ce f is the cross-entropy loss term. seg To address the probability output of point cloud segmentation networks for multi-class classification, f topo For the topological loss constraint branch, the binary classification probability output for all C classes, L topo λ is the average of the topology loss for all categories, and λ is the weight of the topology loss.

2. The method according to claim 1, characterized in that, When PD k (f) and PD k When the number of points in (g) is different, if PD k (f) The number of points is greater than PD k The number of points in (g) then applies to PD. k (f) The closest to (1,0) and PD k (g) Match the same number of points as (1,0), and match the remaining points with the diagonal. If PD k (f) Points less than or equal to PD k The number of points in (g) is then PD k All points in (f) are matched with (1,0), and the extra real marker points are matched with the diagonal.

3. The method according to claim 1, characterized in that, The parameters of the point cloud segmentation network to be trained are optimized through the topological loss constraint branch, including: Define a persistence graph PD k (f) Mapping h from persistent points to points in the original point cloud k : h k :{b i ,d i }→(cp(σ b ),cp(σ d )), Among them, {b i ,d i } is the persistence graph PD k The point in (f), σ b and σ d These are the key simplexes that lead to the birth and extinction of the corresponding topological features, cp(σ) b ), cp(σ d To determine the simplex σ b and σ d Key points regarding function values; Calculating topological loss for persistent graph PD k Each point {b} in (f) i ,d i The gradient of} is calculated, and this gradient is then applied along the corresponding keypoint {b}. i ,d i Backpropagation is performed to optimize the parameters of the point cloud segmentation network, where the gradient of the topology loss is calculated using the following formula: in, Represents the topological loss L topo The gradient calculation of the i-th term, cb(p) and cd(p) represent the birth keypoint and death keypoint of the persistent point p in the original point cloud, respectively, f(·) is the output value of the neural network, and w is the parameter of the neural network.

4. A point cloud segmentation method based on topology awareness, characterized in that, include: Feature extraction is performed on the identified point cloud data to obtain the corresponding point-by-point features; The point-by-point features are input into a pre-trained target point cloud segmentation network so that the target point cloud segmentation network outputs the segmentation result corresponding to the point cloud data. The target point cloud segmentation network is constructed using the topology-aware point cloud segmentation network construction method as described in any one of claims 1-3.

5. A topology-aware point cloud segmentation network construction device, characterized in that, The method for constructing a topology-aware point cloud segmentation network as described in any one of claims 1-3 includes: The construction module is used to construct the total loss function corresponding to the point cloud segmentation network. The total loss function is a weighted sum of the cross-entropy loss term and the topology loss term. An addition module is used to add the total loss function to the point cloud segmentation network to be trained, so as to form the topology loss constraint branch of the point cloud segmentation network to be trained; The training module is used to train the point cloud segmentation network to be trained using training data, and to optimize the parameters of the point cloud segmentation network to be trained using the topology loss constraint branch, so as to obtain the target point cloud segmentation network.

6. A point cloud segmentation device based on topology awareness, characterized in that, include: The extraction module is used to extract features from the identified point cloud data and obtain the corresponding point-by-point features; The processing module is used to input the point-by-point features into a pre-trained target point cloud segmentation network, so that the target point cloud segmentation network outputs the segmentation result corresponding to the point cloud data, wherein the target point cloud segmentation network is constructed using the topology-aware point cloud segmentation network construction method as described in any one of claims 1-3.

7. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the topology-aware point cloud segmentation network construction method as described in any one of claims 1 to 3 and the topology-aware point cloud segmentation method as described in claim 4.

8. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method for constructing a topology-aware point cloud segmentation network as described in any one of claims 1 to 3 and the method for segmenting a topology-aware point cloud as described in claim 4.

Images