Target contour extraction method, device and system based on SCE-Net and storage medium

Abstract

The invention discloses a target contour extraction method, device and system based on SCE-Net and a storage medium, and relates to the technical field of image processing. The method comprises the following steps: acquiring original point cloud data, constructing an SCE-Net network comprising a point cloud target segmentation module and a point cloud target contour extraction module which are connected in series, and inputting the original point cloud data into the point cloud target segmentation module to obtain different types of point cloud individual data; and processing different types of point cloud individual data through a point cloud target contour extraction module to obtain a point cloud target contour extraction result of the whole scene. According to the method provided by the embodiment of the invention, the segmentation accuracy is ensured, the contour extraction fineness is improved, the whole network can directly output segmentation and contour extraction results from the original point cloud data, the manual intervention is reduced, and the automation degree is improved. Accurate target segmentation is realized through the segmentation classifier, and the point cloud target contour of the whole scene is accurately extracted through the contour classifier.

Description

Technical Field

[0001] The present application belongs to the field of image processing technology, and in particular, relates to a target contour extraction method, device, system and storage medium based on SCE-Net. Background Art

[0002] With the continuous development of three-dimensional sensors, the collection of point cloud data has become easier and easier. In recent years, with the development of deep learning technology, especially convolutional neural network technology, it has shown great advantages in the field of image processing. Using deep learning technology to process point cloud data provides a new solution for point cloud data processing tasks.

[0003] However, the point cloud's inherent disorder, non-structure, density inconsistency, and information incompleteness also pose obstacles to the use of deep learning technology to process point cloud data. First, most existing technologies tend to target small-scale point clouds, and many technologies that process large-scale point cloud data also require cutting point clouds, which may cause the loss of overall point cloud information; in addition, mutual occlusion of scanned objects can easily lead to information loss, which also has a great impact on point cloud data processing; in addition, the noise in point cloud data will lead to a reduction in the accuracy of tasks such as shape recognition. In addition, for the same model, rotating different angles still represents the same object, but the network recognition results may differ due to the rotation of the point cloud data.

[0004] In addition, with the continuous advancement of point cloud data acquisition technology and the deepening of research on methods for processing point cloud data based on deep learning technology, tasks related to point cloud data processing are constantly being proposed, mainly including 3D shape classification, 3D target detection and tracking, 3D point cloud segmentation, 3D point cloud registration, 3D reconstruction and 6-DOF posture estimation. To solve these tasks, some deep learning technologies voxelize point cloud data or convert them into 2D views for analysis. However, these technologies have certain limitations. The performance of deep learning technologies based on voxels for point cloud data analysis is often limited by the resolution of voxels. Although the methods for analyzing point cloud data based on 2D images have superior performance in completing shape classification and retrieval tasks, they still have certain difficulties when facing complex tasks. Feature-based deep learning technologies are limited by the ability of manually extracted features to represent point cloud data.

[0005] Most of the traditional methods for identifying target contour points on point cloud data are based on prior knowledge or local features of point clouds, such as covariance matrix, curvature, normal vector, etc. The results obtained by these methods are often over-simplified models of actual geometry. Therefore, traditional 3D reconstruction technology for point clouds often has good effects on a specific category of data, but is not very universal. At present, the research on point cloud target contour extraction is still focused on a certain entity such as industrial parts, and the point cloud data of a scene containing multiple objects is not used as the research object of point cloud target contour extraction technology. In addition, there are few related technologies that fully combine point cloud target contour extraction technology with point cloud target segmentation technology, which has certain disadvantages in solving the problem of point cloud target contour extraction for scene point clouds. Summary of the invention

[0006] The purpose of this application is to provide a target contour extraction method, device, system and storage medium based on SCE-Net, which aims to realize the function of extracting point cloud target contour from scene point cloud data in view of the shortcomings of existing point cloud target contour extraction technology.

[0007] In order to achieve the above objectives, the solution of this application is:

[0008] In a first aspect, an embodiment of the present application provides a target contour extraction method based on SCE-Net, comprising:

[0009] Acquire original point cloud data; construct an SCE-Net network, the SCE-Net network includes a point cloud target segmentation module and a point cloud target contour extraction module, and the point cloud target segmentation module and the point cloud target contour extraction module are connected in series;

[0010] The point cloud target segmentation module includes a first multi-layer perceptron embedding layer, a first encoder, a first decoder and a segmentation classifier connected in sequence; the first encoder includes an N-layer first sub-encoder, and the first decoder includes an N-layer first sub-decoder;

[0011] Inputting the original point cloud data into the first multi-layer perceptron embedding layer for processing to obtain point cloud data features; inputting the point cloud data features into the first encoder for processing to obtain point cloud data features of different dimensions of point cloud data of different scales;

[0012] Input point cloud data features of different dimensions of point cloud data of different scales into the first decoder for processing, and use the point cloud data features output by the first sub-decoder of the previous layer and the point cloud data features input by the first sub-encoder of the corresponding layer as input to obtain point cloud data features of all points; input the point cloud data features of all points into the segmentation classifier for processing to obtain individual point cloud data of different categories;

[0013] The point cloud target contour extraction module includes a second multi-layer perceptron embedding layer, a second encoder, a second decoder and a contour classifier connected in sequence; the second encoder includes N layers of second sub-encoders, and the second decoder includes N layers of second sub-decoders;

[0014] The point cloud individual data of different categories are embedded into the feature space through the second multi-layer perceptron embedding layer, and the point cloud individual data of different categories are processed through the second encoder and the second decoder to obtain the composite point cloud data features of each point in the point cloud individual data of different categories;

[0015] The composite point cloud data features are used through the contour classifier to perform binary classification on all points in the point cloud individual data of different categories, and the point cloud target contour extraction result of the entire scene is obtained.

[0016] According to the above method of the embodiment of the present application, the following additional technical features may also be provided:

[0017] Furthermore, the original point cloud data is input into the first multi-layer perceptron embedding layer for processing to obtain point cloud data features, including:

[0018] A nonlinear mapping is established in the first multi-layer perceptron embedding layer. The eigenvalues ​​of the original point cloud data are extracted through the first multi-layer perceptron embedding layer, and the eigenvalues ​​of the original point cloud data are mapped to the feature space, and the point cloud data features are output. The formula is as follows:

[0019] Γ: R N×M →R N×C

[0020] Where N represents the number of points in the original point cloud data, M represents the dimension of the original point cloud data features, and C represents the feature dimension required for inputting the first encoder for point cloud data feature extraction.

[0021] Furthermore, the point cloud data features are input into the first encoder for processing to obtain point cloud data features of different dimensions of point cloud data of different scales, including:

[0022] The point cloud data features are processed by downsampling in the first encoder, the first encoder includes N layers of first sub-encoders, the feature dimension of the point cloud data input by each layer of the first sub-encoder is C″, and the feature dimension of the point cloud data output by each layer of the first sub-encoder is 2×C″;

[0023] The point cloud data scale input by the first sub-encoder of each layer is N′, and the point cloud data scale output by the first sub-encoder of each layer is N′ / S, where S represents the reduction ratio of the point cloud data scale during the downsampling process. After the downsampling is completed, the point cloud data features are aggregated through graph convolution technology and point cloud data features of different dimensions of point cloud data of different scales are extracted.

[0024] Furthermore, point cloud data features of different dimensions of point cloud data of different scales are input into the first decoder for processing, and the first sub-decoder of each layer in the first decoder takes the point cloud data features output by the first sub-decoder of the previous layer and the point cloud data features input by the first sub-encoder of the corresponding layer as input to obtain the point cloud data features of all points, including:

[0025] In the first decoder, point cloud data features of different dimensions of point cloud data of different scales are processed by upsampling. The first decoder includes N layers of first sub-decoders. The dimension of point cloud data features input by each layer of the first sub-decoder is C″′, and the dimension of point cloud data features output by each layer of the first sub-decoder is C″′ / 2.

[0026] The point cloud data scale input to the first sub-decoder of each layer is N″, and the point cloud data scale output by the first sub-decoder of each layer is N″×S, where S is the point cloud data scale magnification during the upsampling process. After the upsampling is completed, the graph convolution technology is used to aggregate the point cloud data features of different dimensions of point cloud data of different scales and extract the point cloud data features of all points.

[0027] Furthermore, the point cloud data features of all points are input into the segmentation classifier for processing to obtain point cloud individual data of different categories, including:

[0028] In the segmentation classifier, the semantic label of each point in the point cloud data features of all points is predicted, points with the same semantic label are divided into the same category, and points with different semantic labels are segmented to obtain individual point cloud data of different categories.

[0029] Furthermore, in the second sub-decoder of the point cloud target contour extraction module, except for the second sub-decoder of the last layer, the second sub-decoders of each layer take the features output by the second sub-decoder of the previous layer and the input point cloud data features of the second sub-encoder of the corresponding layer as input, and the last layer decoder fuses the point cloud data features of all points obtained by the point cloud target segmentation module, the point cloud data features input by the second sub-encoder of the corresponding layer, and the features output by the second sub-decoder of the previous layer as input to obtain the composite point cloud data features of each point in different categories of point cloud individual data.

[0030] Furthermore, a composite loss function is introduced into the point cloud target contour extraction module. The composite loss function includes a chamfer distance loss function and a weighted binary classification cross entropy loss function. The formula of the chamfer distance loss function is as follows:

[0031]

[0032] Among them, L CD represents the chamfer distance loss function, n gIndicates the number of contour points of the actual point cloud target, S 1 Represents the contour point set of the actual point cloud target, p i Indicates S 1 The coordinates of the i-th contour point in S 2 Represents the set of contour points of the predicted point cloud target, p j Indicates S 2 The coordinates of the i-th contour point in n p Indicates the number of contour points of the predicted point cloud target;

[0033] The weighted binary classification cross entropy loss function formula is as follows:

[0034]

[0035] Among them, L CE represents the weighted binary classification cross entropy loss function, Q i Represents the unique hot encoding of the contour point identification label of each point in the point cloud data, W i Represents the loss calculation weight matrix of the i-th point in the point cloud data, W i ={1, n g1 / n p1},n g1 Indicates the number of non-contour points in the actual point cloud data, n p1 Indicates the number of contour points in the actual point cloud data, Y i Represents the calculation result set of the i-th point in the point cloud data,

[0036] The formula of the composite loss function is as follows:

[0037] L=αL CD +βL CE

[0038] Among them, L represents the composite loss function, α and β represent L CD and L CE The weight of , α+β=1.

[0039] In a second aspect, an embodiment of the present application provides a target contour extraction device based on SCE-Net, comprising:

[0040] A data acquisition module is configured to acquire raw point cloud data;

[0041] A network construction module is configured to construct an SCE-Net network, wherein the SCE-Net network includes a point cloud target segmentation module and a point cloud target contour extraction module, wherein the point cloud target segmentation module and the point cloud target contour extraction module are connected in series, wherein the point cloud target segmentation module includes a first multi-layer perceptron embedding layer, a first encoder, a first decoder, and a segmentation classifier connected in sequence; the first encoder includes N layers of first sub-encoders, and the first decoder includes N layers of first sub-decoders; the point cloud target contour extraction module includes a second multi-layer perceptron embedding layer, a second encoder, a second decoder, and a contour classifier connected in sequence; the second encoder includes N layers of second sub-encoders, and the second decoder includes N layers of second sub-decoders;

[0042] The data processing module is configured to input the original point cloud data into the first multi-layer perceptron embedding layer for processing to obtain point cloud data features; input the point cloud data features into the first encoder for processing to obtain point cloud data features of different dimensions of point cloud data of different scales; input the point cloud data features of different dimensions of point cloud data of different scales into the first decoder for processing, and each layer of the first sub-decoder in the first decoder uses the point cloud data features output by the first sub-decoder of the previous layer and the point cloud data features input by the first sub-encoder of the corresponding layer as input to obtain point cloud data features of all points; input the point cloud data features of all points into the segmentation classifier for processing to obtain point cloud individual data of different categories; embed point cloud individual data of different categories into the feature space through the second multi-layer perceptron embedding layer, process point cloud individual data of different categories through the second encoder and the second decoder, and obtain composite point cloud data features of each point in the point cloud individual data of different categories; perform binary classification on all points in point cloud individual data of different categories using the composite point cloud data features through the contour classifier to obtain the point cloud target contour extraction result of the entire scene.

[0043] In a third aspect, an embodiment of the present application provides a target contour extraction system based on SCE-Net, the system including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement a target contour extraction method based on SCE-Net as provided in the first aspect of the embodiment of the present application.

[0044] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, it is used to implement the target contour extraction method based on SCE-Net as in the first aspect of the embodiment of the present application.

[0045] Compared with the prior art, the target contour extraction method based on SCE-Net provided in the embodiment of the present application has the following beneficial technical effects:

[0046] The embodiment of the present application obtains the original point cloud data, thereby ensuring that the starting point of the analysis is the three-dimensional data of the real world, providing rich spatial information for subsequent processing. It is applicable to point cloud data from various sources, such as laser scanning, radar scanning or three-dimensional reconstruction, and enhances the versatility and applicability of the method.

[0047] The embodiment of the present application constructs an SCE-Net network, connects the point cloud target segmentation module and the point cloud target contour extraction module in series, and realizes a coarse-to-fine processing flow, which not only ensures the accuracy of segmentation, but also improves the precision of contour extraction. The entire network can directly output segmentation and contour extraction results from the original point cloud data, reduces manual intervention, and improves the degree of automation.

[0048] The point cloud target segmentation module of the embodiment of the present application preliminarily extracts the global and local features of the point cloud data through the first multi-layer perceptron embedding layer, laying the foundation for subsequent processing, and performs multi-scale feature extraction through the first encoder. Through encoders at different levels, the different scale features of the point cloud data are captured, thereby enhancing the network's processing capabilities for targets of different scales; the multi-scale features of the first encoder are integrated through the first decoder, which helps to restore spatial information and improve segmentation accuracy; through the segmentation classifier, the category of each point in the point cloud is determined based on the extracted features to achieve accurate target segmentation.

[0049] The point cloud target contour extraction module of the embodiment of the present application further embeds the segmented point cloud data into a feature space that is more suitable for contour extraction through a second multi-layer perceptron embedding layer, thereby optimizing data representation for subsequent processing; performs a more refined encoding-decoding process through a second encoder and a second decoder, extracts and fuses features at different levels, generates composite point cloud data features for each point, and further enhances the judgment of contours; performs binary classification through a contour classifier, uses the composite point cloud data features for binary classification, and determines whether each point belongs to the target contour, thereby accurately extracting the point cloud target contour of the entire scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of the process of the target contour extraction method based on SCE-Net in an embodiment of the present application is shown;

[0051] Figure 2 The structural block diagram of the target contour extraction device based on SCE-Net in an embodiment of the present application is shown;

[0052] Figure 3 A structural block diagram of a computer device according to an embodiment of the present application is shown;

[0053] Figure 4 The SCE-Net network structure diagram of the target contour extraction method based on SCE-Net in an embodiment of the present application is shown;

[0054] Figure 5 The structure diagram of the GCN module of the target contour extraction method based on SCE-Net in the embodiment of the present application is shown;

[0055] Figure 6 The structure diagram of the GAF module of the target contour extraction method based on SCE-Net in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0056] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some structures related to the present application are shown in the accompanying drawings, rather than all structures. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0057] The terms "including" and "having" and any variations thereof in this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0058] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0059] like Figure 1 As shown, the embodiment of the present application provides a target contour extraction method based on SCE-Net, comprising the following steps:

[0060] Step 101, obtaining original point cloud data.

[0061] Obtaining raw point cloud data is a basic step in the field of image processing technology. The following is an extended description of the methods for obtaining raw point cloud data. Common ways to obtain raw point cloud data include laser radar, structured light scanning, time-of-flight sensors, stereo vision, and ultrasonic sensors, among which:

[0062] LiDAR measures the distance between an object and the sensor by emitting laser pulses and receiving reflected signals through vehicle-mounted LiDAR, drone-mounted LiDAR, or ground-based laser scanners. The time required for the laser pulse to be emitted and returned is calculated and converted into distance, thereby generating 3D point cloud data.

[0063] Structured light scanning is to project a known pattern of light onto the surface of an object through a desktop 3D scanner or an industrial-grade structured light scanner, and use a camera to capture the changes in the reflected light pattern. The three-dimensional shape of the object is calculated based on the deformation of the pattern.

[0064] Time of Flight (ToF) sensors calculate distance and generate point cloud data by emitting light pulses from smartphones, AR / VR devices, or robot navigation sensors and measuring the time it takes for the light to travel from the source to the source.

[0065] Stereo Vision uses two or more cameras to capture images of the same scene from different angles through a stereo camera or drone photography system, calculates the depth information of the object by comparing the parallax in the image, and generates a three-dimensional point cloud.

[0066] Ultrasonic sensors emit high-frequency sound waves through ultrasonic rangefinders and receive sound waves reflected from objects, calculate the flight time of the sound waves to measure distance and generate point cloud data.

[0067] Step 102, constructing an SCE-Net network, the SCE-Net network includes a point cloud target segmentation module and a point cloud target contour extraction module, the point cloud target segmentation module and the point cloud target contour extraction module are connected in series; wherein the point cloud target segmentation module includes a first multi-layer perceptron embedding layer, a first encoder, a first decoder and a segmentation classifier connected in sequence; the first encoder includes N layers of first sub-encoders, and the first decoder includes N layers of first sub-decoders; the point cloud target contour extraction module includes a second multi-layer perceptron embedding layer, a second encoder, a second decoder and a contour classifier connected in sequence; the second encoder includes N layers of second sub-encoders, and the second decoder includes N layers of second sub-decoders.

[0068] The structure of the SCE-Net (Segmentation-Contour Extraction Network, SCE-Net) in the embodiment of the present application is as follows: Figure 4As shown, the network consists of two modules: a point cloud target segmentation module and a point cloud target contour extraction module, both of which adopt an encoder-decoder structure. In order to develop an end-to-end network that directly completes the tasks of point cloud target segmentation and point cloud target contour extraction, the embodiment of the present application uses the output of the point cloud target segmentation module as the input of the point cloud target contour extraction module through the SCE-Net network, thereby connecting the two modules in series.

[0069] Step 103, inputting the original point cloud data into the first multi-layer perceptron embedding layer for processing to obtain point cloud data features; inputting the point cloud data features into the first encoder for processing to obtain point cloud data features of different dimensions of point cloud data of different scales.

[0070] In the point cloud target segmentation module, research has found that the AF-GCN network uses a network architecture that combines point cloud Transformer and U-Net, and the module proposed by it for extracting point cloud data features based on graph attention filters has sufficient experimental evidence to prove that it has good performance in point cloud semantic segmentation and point cloud data feature extraction. Therefore, the point cloud target segmentation module mainly refers to the structure of the AF-GCN network to design the theoretical architecture of this module.

[0071] The entire network takes the original point cloud data of the scene as input. In the point cloud target segmentation module, the multi-layer perceptron embedding layer is first used to map the feature values ​​of the original point cloud data into the feature space and finally output the point cloud data features.

[0072] The multi-layer perceptron embedding layer establishes a nonlinear mapping:

[0073] The formula is as follows:

[0074] Γ: R N×M →R N×C

[0075] Where N represents the number of points in the original point cloud data, M represents the dimension of the original point cloud data features, and C represents the feature dimension required for inputting the first encoder for point cloud data feature extraction.

[0076] The point cloud data features are processed by downsampling in the first encoder, the first encoder includes N layers of first sub-encoders, the feature dimension of the point cloud data input by each layer of the first sub-encoder is C″, and the feature dimension of the point cloud data output by each layer of the first sub-encoder is 2×C″;

[0077] The point cloud data scale input by the first sub-encoder of each layer is N′, and the point cloud data scale output by the first sub-encoder of each layer is N′ / S, where S represents the reduction ratio of the point cloud data scale during the downsampling process. After the downsampling is completed, the point cloud data features are aggregated through graph convolution technology and point cloud data features of different dimensions of point cloud data of different scales are extracted.

[0078] like Figure 5 As shown, this is a structural diagram of the GCN (Graph Convolutional Networks, GCN) module in the first encoder of an embodiment of the present application. The structure extracts point cloud data features according to the coordinates of the points and the graph structure of the point cloud data, respectively, and fuses the two features through convolution and multi-layer perceptron (MultiLayer Perceptron, MLP) to obtain the features of a graph composed of neighboring points of a certain point in the point cloud data, and finally obtains higher-level and more abstract features of the point through pooling aggregation.

[0079] Then, the encoder of the point cloud object segmentation module uses downsampling and graph convolution techniques to extract point cloud data features of different dimensions of point cloud data of different scales from the point cloud data features.

[0080] Step 104, input point cloud data features of different dimensions of point cloud data of different scales into the first decoder for processing, and each layer of the first sub-decoder in the first decoder uses the point cloud data features output by the first sub-decoder of the previous layer and the point cloud data features input by the first sub-encoder of the corresponding layer as input to obtain the point cloud data features of all points.

[0081] The decoder of the point cloud target segmentation module takes features of different dimensions of point cloud data of different scales as input and aggregates them with each other. Each layer of the decoder takes the point cloud data features output by the decoder of the previous layer and the input point cloud data features of the encoder of the corresponding layer as input. Finally, the decoder outputs the point cloud data features of all points in the scene. This feature will be retained and applied as prior knowledge to the decoder of the point cloud target contour extraction module.

[0082] like Figure 6 As shown, this is a structural diagram of the GAF (Graph Attention Filter, GAF) module of the decoder in an embodiment of the present application. The structure of this module is basically the same as that of the GCN module. The difference is that the GAF module generates an attention weight matrix based on the spatial distance between the midpoints of the point cloud data when extracting the features of the graph structure. Through the attention mechanism, while applying graph convolution to extract the features of the point cloud data, it suppresses the useless information provided by the features from distant points.

[0083] In addition, in ordinary GCN and GAF ​​modules, the BallQuery algorithm is used to find the neighboring points of each point in the point cloud data to build a graph structure. In the corresponding modules of upsampling and downsampling, the BallQuery algorithm is used to find the neighboring points of each point in the point cloud data before sampling to build a graph structure.

[0084] The first layer decoder of this module takes the high-dimensional features extracted by the last layer encoder as input, and aggregates different point cloud data features. Each layer of the remaining decoders takes the point cloud data features output by the previous layer decoder and the input point cloud data features of the encoder at the corresponding level as input. The dimension of the point cloud data features input to each layer of the decoder is C″′, and the dimension of the point cloud data features output by each decoder is C″′ / 2. The scale of the point cloud data input to each layer of the decoder is N″, and the scale of the point cloud data output by each layer of the decoder is N″×S, where S is the magnification of the point cloud data scale during the upsampling process. Finally, the decoder outputs the spatial features of each point in the scene, which will be retained and applied as prior knowledge to the decoder of the point cloud target contour extraction module. After the upsampling is completed, the graph convolution technology is used to aggregate the point cloud data features of different dimensions of point cloud data of different scales and extract the point cloud data features of all points.

[0085] Step 105 , input the point cloud data features of all points into the segmentation classifier for processing to obtain point cloud individual data of different categories.

[0086] For the point cloud target segmentation module, the embodiment of the present application uses multi-classification cross entropy as the loss function of the point cloud target segmentation module. In the process of calculating the loss function, the output of the point cloud target segmentation module is defined as:

[0087]

[0088] Where n represents the number of points in the point cloud data, c represents the number of categories of individual points after point cloud target segmentation, and Y i Represents the calculation set of different categories of the i-th point in the point cloud data, Represents the calculation result corresponding to the mth category of the ith point in the point cloud target segmentation module.

[0089] The embodiment of the present application uses multi-classification cross entropy as the loss function of the point cloud target segmentation module. The loss function first uses the softmax function to normalize the calculation results of the network, then takes the logarithm operation on the normalized result, and finally calculates according to the following formula:

[0090]

[0091] Where Q i A unique encoding representing the segmentation class label of each point in the point cloud data.

[0092] Finally, after the segmentation classifier predicts semantic labels for each point in the scene, the scene will be segmented into point cloud individuals of different categories. At the end of the point cloud target segmentation module, the segmentation classifier predicts semantic labels for each point in the scene based on the spatial features of each point in the scene, and the point cloud scene will be segmented into point cloud individuals of different categories.

[0093] Step 106, embedding different categories of point cloud individual data into the feature space through the second multi-layer perceptron embedding layer, processing different categories of point cloud individual data through the second encoder and the second decoder, and obtaining composite point cloud data features of each point in the different categories of point cloud individual data.

[0094] In the point cloud target contour extraction module, a network architecture similar to that of the point cloud target segmentation module is adopted. However, compared with the point cloud target segmentation module, the point cloud target contour extraction module needs to extract more comprehensive and important point cloud data features. Therefore, in order to aggregate the point cloud data features more finely and comprehensively, the point cloud target contour extraction module uses the attention pooling (AP) mechanism to complete the pooling of point cloud features. Therefore, the graph convolutional networks (GCN) and graph attention filters (GAF) in this module are replaced by attention pooling-graph convolutional networks (AP-GCN) and attention pooling-graph attention filters (AP-GAF).

[0095] In the point cloud data used for point cloud object contour extraction, the number of non-contour points often far exceeds the number of contour points. This imbalance makes it easy for the model to favor a large number of non-contour points during training, thereby neglecting the learning of contour points. If the training loss is calculated directly through ordinary binary cross entropy as in the point cloud object segmentation module, due to the large number of non-contour points, the model tends to pay too much attention to non-contour points, and then overfits the non-contour points. In order to pursue higher accuracy, it tends to identify all points in the scene point cloud as non-contour points.

[0096] In view of the above problems, a method that can be used is to use weighted binary cross entropy as the loss function and give a higher weight to the loss of contour points during the training process, so that the model pays more attention to contour points during the learning process. However, it is found in the experimental process of the embodiment of the present application that only using weighted binary cross entropy can alleviate the problem caused by the imbalance of positive and negative samples to a certain extent, but it cannot completely avoid the occurrence of overfitting during the training process.

[0097] In addition, due to the serious imbalance of positive and negative samples in the point cloud data used by the point cloud target contour extraction module, there is a problem of overfitting of samples with a larger proportion during the learning process. To address this problem, the embodiment of the present application introduces a composite loss function in the point cloud target contour extraction module, which combines the chamfer distance and the binary classification cross entropy in a weighted summation manner, controls the parameter optimization direction of the model during the learning process, and avoids overfitting problems during training.

[0098] In order to better solve the overfitting problem caused by the imbalance of positive and negative samples in point cloud data and optimize the model learning effect, the embodiment of this application uses two loss functions combined with each other to form a relatively complex composite loss function to adjust the optimization direction of model parameters. The composite loss function designed in this article mainly includes two parts: chamfer distance and weighted binary classification cross entropy.

[0099] Chamfer distance is often used to calculate the loss function of point cloud reconstruction. This method calculates the average value of the minimum distance between each point in the two point clouds and all the points in the other point cloud. Therefore, this method can intuitively display the contour similarity between the point cloud data predicted by the model and the actual point cloud data, and can well measure the effect of point cloud target contour extraction from the perspective of the final recognition result. In the process of calculating the chamfer distance, the set of contour points of the actual point cloud target is defined as:

[0100]

[0101] where n g Indicates the number of contour points in the actual point cloud target, p i Indicates S 1 The coordinates of the i-th contour point in .

[0102] Similarly, the contour point set of the point cloud target predicted by this application is defined as:

[0103] where n p represents the number of contour points in the target of the predicted point cloud, p j Indicates S 2 The coordinates of the jth contour point in .

[0104] Specifically, first based on S 1 A point in i , find S 2 A point in j So that p j to p i The minimum distance is defined as Then based on S 2 A point in j , find S 1 A point in i So that p i to p j The minimum distance is defined as Then find S 1 All points in The average value and S 2 All points in The average value of is summed up as the loss value calculated based on the chamfer distance. The specific calculation process is as follows:

[0105]

[0106] Among them, L CD represents the chamfer distance loss function, n g Indicates the number of contour points of the actual point cloud target, S 1 Represents the contour point set of the actual point cloud target, p i Indicates S 1 The coordinates of the i-th contour point in S 2 Represents the set of contour points of the predicted point cloud target, p j Indicates S 2 The coordinates of the i-th contour point in , np represents the number of contour points of the predicted point cloud target;

[0107] Binary classification cross entropy is widely used in binary classification problems as a method of calculating model training loss. The core concept of this method is to represent the difference between the probability distribution of different categories of samples predicted by the model and the actual label value of the sample. If the predicted probability is closer to the label value, the value of the cross entropy loss function is smaller, and vice versa. The value of the cross entropy loss function is larger. Based on this principle, this method can be used to measure whether the binary classification prediction result is accurate. Since the point cloud data used by the point cloud target contour extraction module has the problem of sample imbalance, the embodiment of the present application sets weight values ​​for the binary cross entropy calculation of different categories, so that the model pays more attention to contour points when learning. When calculating the weighted binary classification cross entropy, the embodiment of the present application defines the output of the point cloud target contour extraction module and the weights of each category as:

[0108]

[0109] Among them, W i Represents the loss calculation weight matrix of the i-th point in the point cloud data, n g1 Indicates the number of non-contour points in the actual point cloud data, n p1 Indicates the number of contour points in the actual point cloud data, Y i Represents the calculation result set of the i-th point in the point cloud data,

[0110] The formula for the loss function based on weighted binary classification cross entropy is as follows:

[0111]

[0112] Among them, L CE represents the weighted binary classification cross entropy loss function, Q i Represents the one-hot encoding of the contour point identification label of each point in the point cloud data.

[0113] In contrast, the chamfer distance compares the point cloud data from the perspective of the sample label predicted by the model to calculate the training loss, and the weighted binary classification cross entropy mainly calculates the training loss based on the model's prediction probability for different categories. The two measure the model prediction results from different perspectives. The point cloud target contour point extraction module loss function of the embodiment of the present application combines the two loss functions to constrain the parameter optimization direction of the model from different dimensions. The composite loss function is mainly implemented by weighted summation of the two loss functions, and the formula is as follows:

[0114] L=αL CD +βL CE

[0115] Among them, L represents the composite loss function, α and β represent L CD and L CE The weight is α+β=1, and in the embodiment of the present application, α=0.3, β=0.7.

[0116] Since the point cloud target contour extraction module and the point cloud target segmentation module use similar module settings, the embodiment of the present application uses the point cloud data features obtained by the decoder in the point cloud target segmentation module as prior knowledge, and merges the point cloud data features extracted by the two modules in the decoder of the point cloud target contour extraction module. The point cloud target segmentation and point cloud target contour extraction modules are more effectively connected to each other by feature sharing, and the point cloud data features extracted by the point cloud target segmentation module are fully utilized to provide richer point cloud data features for point cloud target contour extraction.

[0117] When extracting the point cloud target contour from the scene point cloud data, it is necessary not only to extract the curve of each individual surface, but also to extract the boundaries between different individuals as part of the scene structure. At the same time, the point cloud target segmentation module accurately divides the scene point cloud data into several independent point cloud subsets belonging to different categories, which correspond to different objects or components in the scene. In the segmentation process, accurate segmentation of scene point cloud data often means accurate identification of the boundaries of point cloud individuals. Therefore, in SCE-Net, the point cloud data features extracted by the last layer decoder of the point cloud target segmentation module are applied as prior knowledge to the last layer decoder of the point cloud target contour extraction module, which can not only make the point cloud target contour extraction module more sensitive to the boundaries between point cloud individuals, but also further optimize the model training effect.

[0118] The embodiment of the present application reconstructs the last layer decoder of the point cloud target contour extraction module, and integrates the point cloud data features extracted by the point cloud target segmentation module with the point cloud data features extracted by the point cloud target contour extraction module.

[0119] In the unreconstructed decoder structure of the point cloud target contour extraction module, the feature Featurel output by the decoder of the previous level and the input point cloud data feature Feature2 of the encoder of the corresponding level are taken as input. First, the graph structure built in the upsampling process is used to aggregate Feature1 to generate the features of all points in the upsampled point cloud data, and Feature2 is further processed by a multi-layer perceptron to obtain a higher-level representation of Feature2. The two features are then concatenated and deeply fused through a convolution operation. Finally, the fused features are used as the output of the entire decoder for the classifier to perform classification operations.

[0120] In the reconstructed last layer decoder, the double input head structure of the original last layer decoder is changed to a three input head structure, and the point cloud feature Feature3 extracted by the point cloud data segmentation module is added as a new input. First, after the newly added Feature3 is input into the last layer encoder, it is directly fused with Feature2 by element-by-element addition to increase the feature information of Feature2, and then the added features are fused with Feature1 by linking and convolution, and finally the fused point cloud data features of the three are output for the contour classifier to classify.

[0121] By applying the point cloud data features extracted by the last encoder of the point cloud target segmentation module as prior knowledge to the last encoder of the point cloud target contour extraction module, not only can the two modules be connected together to make the network structure more coherent, but also the amount of information of the point cloud data features extracted by the point cloud target contour extraction module is increased, and the final effect of point cloud target contour extraction is improved. In addition, after the prior knowledge is input into the decoder and directly merged with Feature2, the input and output feature dimensions of the point cloud target extraction module are not changed. The point cloud target segmentation module and the point cloud target contour extraction module can not only be trained jointly, but also split and trained separately, which adds more flexibility to the experiment and allows the design of multiple training strategies according to research needs.

[0122] The point cloud target contour extraction module takes the point cloud data of different categories of point cloud individuals as the input of the module. First, the multi-layer perceptron embedding layer is used to embed the point coordinates of different individuals into the feature space. Then, the encoder in the point cloud target contour extraction module further learns the features of different dimensions of point cloud data of different scales, and then the decoder aggregates the different point cloud data features. Except for the last layer decoder, each layer decoder still takes the features output by the previous layer decoder and the input point cloud data features of the corresponding level encoder. The last layer decoder takes the point cloud data features extracted by the point cloud target segmentation module, the input point cloud data features of the corresponding level encoder, and the point cloud data features extracted by the previous layer decoder as input and fuses the three together, and finally outputs the composite point cloud data features of each point in the individual.

[0123] In the SCE-Net network, graph convolution extracts the features of a point and its surrounding points based on the neighborhood of a point in the point cloud. It then aggregates the features of all points in its neighborhood in the graph structure through a pooling layer, and finally combines the aggregated features with the features of the point itself.

[0124] Among the existing pooling methods, recent work related to point cloud data processing often uses average pooling or maximum pooling to aggregate point cloud data features. However, both methods have certain limitations. Aggregating features using the maximum pooling method will select the maximum value of the features among neighboring points as the representative feature of all points in the neighborhood. This method can intuitively identify the most important features, but at the same time it will ignore other point cloud data features except the maximum value, resulting in the loss of most point cloud data features. However, for the point cloud target contour extraction module, the maximum pooling method is not sophisticated and comprehensive enough, and often ignores some point cloud data features that contribute greatly to the extraction of point cloud target contours. Aggregating features using the average pooling method will calculate the average value of the features of a certain point in the point cloud data and its neighboring points to obtain a unified feature representation. Although this method takes into account all point cloud data features, it cannot distinguish the different features of the point cloud data according to their importance, and cannot highlight more important point cloud data features. In contrast, the weighted pooling method can not only take into account the characteristics of all points in the neighborhood of the point cloud data by controlling the pooling weights, but also focus on learning important point cloud data features by controlling the weight values, which is more suitable for application in the point cloud target contour extraction module.

[0125] The embodiment of the present application dynamically assigns attention weights to the point cloud data features of neighboring points of each point in the point cloud data in the pooling layer, distinguishes the different feature values ​​of each point according to their importance, and strengthens the learning of important point cloud data features when the features are aggregated. In terms of designing the strategy for pooling weight allocation, this paper is inspired by previous work and integrates the attention mechanism into the pooling layer. During the training process, the weight allocation strategy of the pooling layer is continuously learned and optimized, rather than using a fixed weight allocation strategy to allocate point cloud data feature weights and aggregate point cloud data features, making the aggregation of point cloud data features more flexible.

[0126] The attention pooling layer takes the point cloud data feature set of k neighboring points in the neighborhood of each point in the point cloud data as input. In order to identify the important features in the point cloud data features, it is first necessary to evaluate the importance of the features of different points. The feature formula in the point cloud data is as follows:

[0127]

[0128] where F i Represents the set of feature vectors corresponding to all neighboring points in the neighborhood of the i-th point in the point cloud data, Represents the eigenvector corresponding to the mth neighboring point in the neighborhood of the point, and k represents the number of neighboring points in the neighborhood of the point.

[0129] The embodiment of the present application implements the evaluation of the importance of point cloud data features based on linear mapping. For the feature vectors of all points in the neighborhood of a certain point, linear mapping is used to calculate the corresponding importance coefficients for the feature vectors of the neighboring points of each point. The weight matrix of the linear mapping can be obtained and continuously optimized during the learning process to ensure that more important features in the point cloud data are found more accurately. After completing the importance evaluation, the importance coefficients of the point cloud data features are normalized using the softmax function and the attention weight matrix is ​​finally calculated to ensure that in the subsequent weight mapping process, each eigenvalue can receive appropriate attention according to its importance. After the attention weight matrix is ​​calculated, the point cloud data features are multiplied element by element by the attention weight matrix to obtain the point cloud data features after the attention weights are assigned. The formula for attention weight allocation is as follows:

[0130]

[0131] in is the point cloud data feature after weight mapping processing, Represents the linear mapping used to calculate the attention matrix.

[0132] After completing the attention weight allocation, it is necessary to sum each feature value of all neighboring points according to the graph structure to complete the feature aggregation of all neighboring points. In addition, in order to fully learn the features of the aggregated point cloud data, it is necessary to use MLP to conduct a more in-depth analysis of the aggregated point cloud data features and extract more abstract point cloud data features. The pooling formula based on the attention mechanism is as follows:

[0133]

[0134] in, Represents the set of point cloud feature vectors obtained after feature aggregation.

[0135] In contrast, the point cloud data feature extraction module based on the graph attention filter used in SCE-Net mainly uses the spatial distance between each point in the point cloud data and each neighboring point in its neighborhood as the main basis for calculating the attention weight, so that the entire network can suppress useless information from distant points when extracting features; the attention pooling mechanism mainly uses the importance of features of different dimensions in the point cloud data features as the main basis for calculating the attention weight, so that the entire network focuses on more important point cloud data features when aggregating features. The two adjust the focus of model learning from different dimensions respectively. The attention pooling mechanism proposed in the embodiment of the present application serves as a supplement to the graph attention filter. While fully aggregating the data features of all neighboring points, it further strengthens the network's learning of important point cloud data features.

[0136] Step 107, using a contour classifier to perform binary classification on all points in different categories of point cloud individual data using the composite point cloud data features, to obtain a point cloud target contour extraction result for the entire scene.

[0137] Finally, the contour classifier uses the composite point cloud data features of each point in the point cloud data to perform binary classification on all points in the point cloud individual, and integrates the classification results in different individual point cloud data to finally obtain the point cloud target contour extraction result of the entire scene.

[0138] In order to verify the effect of the method provided in the embodiment of the present application, experiments were carried out on the training and testing of SCE-Net on the S3DIS dataset. By manually improving the S3DIS dataset and annotating the contours, the contour extraction effect of a room in the dataset was obtained. It can be concluded that the network has achieved good results in point cloud target contour extraction. The SCE-Net proposed in the embodiment of the present application basically has the ability to identify contour points in point cloud targets.

[0139] The experimental results of SCE-Net model testing on the S3DIS dataset are shown in the following table:

[0140] Table 1

[0141] Accuracy Accuracy Recall F1 score 0.88 0.29 0.68 0.40

[0142] It can be concluded that combining point cloud target segmentation technology with point cloud target contour extraction technology has a better effect on recognizing contour points of point cloud data collected based on actual scenes.

[0143] like Figure 2 As shown, the embodiment of the present application provides a target contour extraction device based on SCE-Net, including a data acquisition module 201, a network construction module 202 and a data processing module 203, wherein:

[0144] A data acquisition module 201 is configured to acquire original point cloud data;

[0145] The network construction module 202 is configured to construct an SCE-Net network, wherein the SCE-Net network includes a point cloud target segmentation module and a point cloud target contour extraction module, wherein the point cloud target segmentation module and the point cloud target contour extraction module are connected in series, wherein the point cloud target segmentation module includes a first multi-layer perceptron embedding layer, a first encoder, a first decoder, and a segmentation classifier connected in sequence; the first encoder includes N layers of first sub-encoders, and the first decoder includes N layers of first sub-decoders; the point cloud target contour extraction module includes a second multi-layer perceptron embedding layer, a second encoder, a second decoder, and a contour classifier connected in sequence; the second encoder includes N layers of second sub-encoders, and the second decoder includes N layers of second sub-decoders;

[0146] The data processing module 203 is configured to input the original point cloud data into the first multi-layer perceptron embedding layer for processing to obtain point cloud data features; input the point cloud data features into the first encoder for processing to obtain point cloud data features of different dimensions of point cloud data of different scales; input the point cloud data features of different dimensions of point cloud data of different scales into the first decoder for processing, and each layer of the first sub-decoder in the first decoder uses the point cloud data features output by the first sub-decoder of the previous layer and the point cloud data features input by the first sub-encoder of the corresponding layer as input to obtain point cloud data features of all points; input the point cloud data features of all points into the segmentation classifier for processing to obtain point cloud individual data of different categories; embed point cloud individual data of different categories into the feature space through the second multi-layer perceptron embedding layer, process point cloud individual data of different categories through the second encoder and the second decoder, and obtain composite point cloud data features of each point in the point cloud individual data of different categories; perform binary classification on all points in point cloud individual data of different categories using the composite point cloud data features through the contour classifier to obtain the point cloud target contour extraction result of the entire scene.

[0147] The target contour extraction device based on SCE-Net in the embodiment of the present application can be a computer device, or a component in the computer device, such as an integrated circuit or a chip. The computer device can be a terminal, or other devices other than a terminal. Exemplarily, the computer device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted computer device, a mobile Internet device (Mobile Internet Device, MID), an ultra-mobile personal computer (Ultra-Mobile Personal Computer, UMPC), a netbook or a personal digital assistant (Personal Digital Assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (Personal Computer, PC), etc., which is not specifically limited in the embodiment of the present application.

[0148] The target contour extraction device based on SCE-Net provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented in the embodiment of the target contour extraction method based on SCE-Net are not repeated here.

[0149] The present application also provides a computer device, such as Figure 3As shown, the computer device includes a processor 301 and a memory 302. The memory 302 stores programs or instructions that can be executed on the processor 301. When the program or instruction is executed by the processor 301, the various steps of the above-mentioned SCE-Net-based target contour extraction method are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0150] It should be noted that the computer device in the embodiment of the present application includes the above-mentioned mobile computer device and non-mobile computer device.

[0151] The memory 302 can be used to store software programs and various data. The memory 302 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 302 may include a volatile memory or a non-volatile memory, or the memory 302 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 302 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0152] The processor 301 may include one or more processing units; optionally, the processor 301 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 301.

[0153] The embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned SCE-Net-based target contour extraction method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0154] The embodiment of the present application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned SCE-Net-based target contour extraction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0155] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0156] The embodiment of the present application also provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned SCE-Net-based target contour extraction method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0157] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0158] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A target contour extraction method based on SCE-Net, characterized in that: The method comprises: Acquire original point cloud data; construct an SCE-Net network, wherein the SCE-Net network includes a point cloud target segmentation module and a point cloud target contour extraction module, and the point cloud target segmentation module and the point cloud target contour extraction module are connected in series; The point cloud target segmentation module includes a first multi-layer perceptron embedding layer, a first encoder, a first decoder and a segmentation classifier connected in sequence; the first encoder includes an N-layer first sub-encoder, and the first decoder includes an N-layer first sub-decoder; Inputting the original point cloud data into the first multi-layer perceptron embedding layer for processing to obtain point cloud data features; inputting the point cloud data features into the first encoder for processing to obtain point cloud data features of different dimensions of point cloud data of different scales; Inputting point cloud data features of different dimensions of the point cloud data of different scales into the first decoder for processing, the first sub-decoder of each layer in the first decoder takes the point cloud data features output by the first sub-decoder of the previous layer and the point cloud data features input by the first sub-encoder of the corresponding layer as input, to obtain point cloud data features of all points; inputting the point cloud data features of all points into the segmentation classifier for processing, to obtain point cloud individual data of different categories; The point cloud target contour extraction module includes a second multi-layer perceptron embedding layer, a second encoder, a second decoder and a contour classifier connected in sequence; the second encoder includes N layers of second sub-encoders, and the second decoder includes N layers of second sub-decoders; The point cloud individual data of different categories are embedded into the feature space through the second multi-layer perceptron embedding layer, and the point cloud individual data of different categories are processed through the second encoder and the second decoder to obtain the composite point cloud data features of each point in the point cloud individual data of different categories; The composite point cloud data features are used by the contour classifier to perform binary classification on all points in different categories of point cloud individual data, thereby obtaining a point cloud target contour extraction result of the entire scene.

2. The target contour extraction method based on SCE-Net as claimed in claim 1, characterized in that: The inputting the original point cloud data into the first multi-layer perceptron embedding layer for processing to obtain point cloud data features includes: A nonlinear mapping is established in the first multi-layer perceptron embedding layer, and the feature values ​​of the original point cloud data are extracted through the first multi-layer perceptron embedding layer, and the feature values ​​of the original point cloud data are mapped to the feature space, and the point cloud data features are output. The formula is as follows: C: R N×M →R N×C Where N represents the number of points in the original point cloud data, M represents the dimension of the original point cloud data features, and C represents the feature dimension required for inputting the first encoder for point cloud data feature extraction.

3. The target contour extraction method based on SCE-Net as claimed in claim 1, characterized in that: The step of inputting the point cloud data features into the first encoder for processing to obtain point cloud data features of different dimensions of point cloud data of different scales includes: The point cloud data features are processed by downsampling in the first encoder, the first encoder includes N layers of first sub-encoders, the point cloud data feature dimension input by each layer of the first sub-encoder is C″, and the point cloud data feature dimension output by each layer of the first sub-encoder is 2×C″; The point cloud data scale input by the first sub-encoder of each layer is N′, and the point cloud data scale output by the first sub-encoder of each layer is N′ / S, where S represents the reduction ratio of the point cloud data scale during the downsampling process. After the downsampling is completed, the point cloud data features are aggregated through graph convolution technology and point cloud data features of different dimensions of point cloud data of different scales are extracted.

4. The target contour extraction method based on SCE-Net as claimed in claim 1, characterized in that: The point cloud data features of different dimensions of the point cloud data of different scales are input into the first decoder for processing, and the first sub-decoder of each layer in the first decoder takes the point cloud data features output by the first sub-decoder of the previous layer and the point cloud data features input by the first sub-encoder of the corresponding layer as input to obtain the point cloud data features of all points, including: In the first decoder, the point cloud data features of different dimensions of the point cloud data of different scales are processed by upsampling, the first decoder includes N layers of first sub-decoders, the dimension of the point cloud data features input by each layer of the first sub-decoder is C″′, and the dimension of the point cloud data features output by each layer of the first sub-decoder is C″′ / 2; The point cloud data scale input to the first sub-decoder of each layer is N″, and the point cloud data scale output by the first sub-decoder of each layer is N″×S, where S is the point cloud data scale magnification during the upsampling process. After the upsampling is completed, the graph convolution technology is used to aggregate the point cloud data features of different dimensions of point cloud data of different scales and extract the point cloud data features of all points.

5. The target contour extraction method based on SCE-Net as claimed in claim 1, characterized in that: The point cloud data features of all points are input into the segmentation classifier for processing to obtain point cloud individual data of different categories, including: In the segmentation classifier, the semantic label of each point in the point cloud data features of all points is predicted, points with the same semantic label are divided into the same category, and points with different semantic labels are segmented to obtain individual point cloud data of different categories.

6. The target contour extraction method based on SCE-Net as claimed in claim 1, characterized in that: In the second sub-decoder of the point cloud target contour extraction module, except for the second sub-decoder of the last layer, the second sub-decoders of each layer take the features output by the second sub-decoder of the previous layer and the input point cloud data features of the second sub-encoder of the corresponding layer as input, and the last layer decoder fuses the point cloud data features of all points obtained by the point cloud target segmentation module, the point cloud data features input by the second sub-encoder of the corresponding layer, and the features output by the second sub-decoder of the previous layer as input to obtain the composite point cloud data features of each point in different categories of point cloud individual data.

7. The target contour extraction method based on SCE-Net as claimed in claim 1, characterized in that: A composite loss function is introduced into the point cloud target contour extraction module, and the composite loss function includes a chamfer distance loss function and a weighted binary classification cross entropy loss function, wherein the chamfer distance loss function formula is as follows: Among them, L CD represents the chamfer distance loss function, n g represents the number of contour points of the actual point cloud target, S1 represents the contour point set of the actual point cloud target, and p i represents the coordinates of the i-th contour point in S1, S2 represents the contour point set of the predicted point cloud target, and p j represents the coordinates of the i-th contour point in S2, n p Indicates the number of contour points of the predicted point cloud target; The weighted binary classification cross entropy loss function formula is as follows: Among them, L CE represents the weighted binary classification cross entropy loss function, Q i Represents the unique hot encoding of the contour point identification label of each point in the point cloud data, W i Represents the loss calculation weight matrix of the i-th point in the point cloud data, W i ={1, n g1 / m p1 },n g1 Indicates the number of non-contour points in the actual point cloud data, n p1 Indicates the number of contour points in the actual point cloud data, Y i Represents the calculation result set of the i-th point in the point cloud data, The composite loss function formula is as follows: L=αL CD +βL CE Among them, L represents the composite loss function, α and β represent L CD and L CE The weight of , α+β=1.

8. A target contour extraction device based on SCE-Net, characterized in that: The device comprises: A data acquisition module is configured to acquire raw point cloud data; A network construction module is configured to construct an SCE-Net network, wherein the SCE-Net network includes a point cloud target segmentation module and a point cloud target contour extraction module, wherein the point cloud target segmentation module and the point cloud target contour extraction module are connected in series, wherein the point cloud target segmentation module includes a first multi-layer perceptron embedding layer, a first encoder, a first decoder, and a segmentation classifier connected in sequence; the first encoder includes N layers of first sub-encoders, and the first decoder includes N layers of first sub-decoders; the point cloud target contour extraction module includes a second multi-layer perceptron embedding layer, a second encoder, a second decoder, and a contour classifier connected in sequence; the second encoder includes N layers of second sub-encoders, and the second decoder includes N layers of second sub-decoders; The data processing module is configured to input the original point cloud data into the first multi-layer perceptron embedding layer for processing to obtain point cloud data features; input the point cloud data features into the first encoder for processing to obtain point cloud data features of different dimensions of point cloud data of different scales; input the point cloud data features of different dimensions of point cloud data of different scales into the first decoder for processing, and the first sub-decoder of each layer in the first decoder uses the point cloud data features output by the first sub-decoder of the previous layer and the point cloud data features input by the first sub-encoder of the corresponding layer as input to obtain point cloud data features of all points; input the point cloud data features of all points into the segmentation classifier for processing to obtain point cloud individual data of different categories; embed the point cloud individual data of different categories into the feature space through the second multi-layer perceptron embedding layer, process the point cloud individual data of different categories through the second encoder and the second decoder, and obtain the composite point cloud data features of each point in the point cloud individual data of different categories; perform binary classification on all points in the point cloud individual data of different categories using the composite point cloud data features through the contour classifier to obtain the point cloud target contour extraction result of the entire scene.

9. A target contour extraction system based on SCE-Net, the system comprising a processor and a memory, wherein a computer program is stored in the memory, characterized in that: The computer program is loaded and executed by the processor to implement the target contour extraction method based on SCE-Net as described in any one of claims 1 to 7.

10. A computer-readable storage medium, wherein a computer program is stored in the storage medium, characterized in that: When the computer program is executed by a processor, it is used to implement the target contour extraction method based on SCE-Net as described in any one of claims 1 to 7.

Images